Python Data Engineer – Airflow & AWS

Freelancing Opportunity

Job Title: Python Data Engineer – Airflow & AWS

Job Summary

Looking for a Python Data Engineer to support and enhance Airflow-based data pipelines. The role is hands-on and focused on Python coding, updating parent/base classes, AWS Glue-based data validation, EC2 monitoring, and AWS Genie alerts for pipeline failures.

Key Responsibilities

• Write and maintain Python code for data pipelines.
• Refactor and update parent/base classes and shared logic.
• Develop, monitor, and troubleshoot Apache Airflow DAGs.
• Run AWS Glue Crawlers for schema discovery and data validation.
• Monitor EC2 instances supporting pipelines.
• Configure AWS Genie notifications for pipeline failures.
• Perform production support, data quality checks, and root-cause analysis.

Required Skills

• Strong Python (OOP, reusable code).
• Apache Airflow (DAGs, scheduling, retries).
• AWS Glue Crawlers & Data Catalog.
• AWS EC2 monitoring.
• AWS Genie or similar alerting tools.
• Git, Linux basics, logging & debugging.

Tech Stack

Python | Apache Airflow | AWS Glue | EC2 | S3 | AWS Genie | Git | Linux

Job Types: Part-time, Freelance
Contract length: 12 months

Pay: ₹30,000.00 - ₹40,000.00 per month

Benefits:

• Work from home

Experience:

• total work: 7 years (Required)

Shift availability:

• Night Shift (Required)
• Overnight Shift (Required)

Work Location: Remote