Azure Data Engineer (Airflow Mandatory)

Job Title: Azure Data Engineer (Airflow Mandatory)

Experience: 5+ years

Location: Remote

Employment Type: Full-time Contract

Timing: 3-11 PM IST

Role Summary

We are looking for an experienced Azure Data Engineer with strong Airflow orchestration experience to design, build, and maintain scalable data pipelines on Azure. The role requires hands-on expertise in Python, PySpark, SQL/PLSQL, and Azure data services, with Airflow as a core, mandatory skill.

Key Responsibilities

Design and build end-to-end data pipelines using Apache Airflow for orchestration

Develop scalable ETL/ELT workflows using Python and PySpark

Implement data ingestion from multiple sources (RDBMS, APIs, files, cloud storage)

Optimize and maintain SQL / PL-SQL queries for performance and reliability

Work extensively with Azure Data Services for batch and/or streaming workloads

Implement data quality checks, logging, monitoring, and alerting

Collaborate with analytics, reporting, and business teams

Follow best practices for security, performance, and cost optimization

Support production pipelines and troubleshoot data issues

Mandatory Technical Skills

Apache Airflow (Mandatory)

DAG design, scheduling, sensors, operators

Dependency management, retries, SLAs, and monitoring

Python – strong hands-on development experience

PySpark – data transformation and processing

SQL / PL-SQL – complex queries, optimization, analytical functions

Azure Data Engineering Services, including:

Azure Data Factory

Azure Data Lake Storage (ADLS Gen2)

Azure Databricks

Azure Synapse Analytics (preferred)

Experience with data modeling (fact/dimension, curated layers)

Good to Have

CI/CD for data pipelines (Azure DevOps, GitHub Actions)

Experience with monitoring tools (Datadog, Azure Monitor, Log Analytics)

Exposure to Delta Lake / Parquet formats

Understanding of data governance and security in Azure

Experience in cloud cost optimization

Experience & Qualifications

5+ years of hands-on experience in Data Engineering

Proven experience building production-grade pipelines on Azure

Strong problem-solving and debugging skills

Ability to work independently and in cross-functional teams

Why Join Us

Work on real-world, large-scale data platforms

Ownership of end-to-end data pipelines

Opportunity to work with modern cloud and data technologies

Collaborative and engineering-driven culture

Job Types: Contractual / Temporary, Freelance
Contract length: 6 months

Work Location: Remote