Senior Data Engineer

We are looking for a Senior Data Engineer to design, build, and maintain enterprise-scale data platforms on Azure. You will own end-to-end data pipelines from raw ingestion to curated, analytics-ready gold layers while driving data quality standards, mentoring junior engineers, and collaborating directly with product and AI teams.

Pipeline Design & Architecture

• Architect and manage scalable ETL/ELT pipelines using Medallion Architecture (Bronze Silver Gold)
• Design ADF pipelines, data models, and schemas optimised for performance and maintainability
• Build near real-time and batch ingestion solutions across structured and semi-structured sources

Data Governance & Quality

• Implement data quality checks, validation frameworks, and lineage tracking across pipeline layers
• Enforce security controls, access policies, and compliance requirements on ADLS Gen 2 and Azure SQL
• Monitor pipeline SLAs, diagnose performance bottlenecks, and own incident resolution

Engineering Excellence

• Drive CI/CD adoption for data pipelines using Azure DevOps or equivalent tooling
• Define and enforce coding standards, peer-review practices, and documentation norms
• Contribute to Al agent development initiatives as a data platform subject-matter expert

Collaboration & Mentorship

• Partner with data scientists, analysts, and product managers to translate requirements into reliable data products
• Mentor junior engineers through code reviews, design discussions, and structured knowledge sharing
• Participate in Agile ceremonies and contribute to sprint planning, backlog refinement, and technical estimation

EXPERIENCE & REQUIREMENTS

Must-Have

• 5-6 years of hands-on Data Engineering experience in production environments
• Deep expertise in Azure ecosystem — ADLS Gen 2, ADF, Azure SQL Database, and Delta Lake
• Strong command of PySpark, Python, and SQL for large-scale data transformation
• Proven track record building and operating Medallion Architecture (Bronze/Silver/Gold) on Azure
• Experience designing data warehouses and data lake solutions at enterprise scale
• Familiarity with CI/CD pipelines, version control (Git), and DevOps practices in a data context

Nice-to-Have

• Exposure to AWS or GCP in addition to Azure
• Experience supporting or building Al/ML data pipelines and feature stores
• Knowledge of streaming frameworks — Kafka, Event Hubs, or Spark Structured Streaming
• Azure certifications (DP-203, AZ-900 or equivalent)