Azure Data Engineer | 5+ years

Job Summary

We are seeking a skilled and experienced Data Engineer with strong expertise in Azure Databricks, Spark/PySpark, and Big Data technologies. The ideal candidate will design, develop, and optimize scalable data pipelines while ensuring performance, reliability, and data quality across large datasets. This role requires strong technical capability along with excellent communication and interpersonal skills to collaborate effectively with cross-functional teams and business stakeholders.

Key Responsibilities

• Design, build, and maintain scalable ETL/ELT pipelines using Azure Databricks, Spark, and PySpark.
• Develop and optimize big data solutions leveraging Hadoop ecosystem tools.
• Work closely with data architects, analysts, and business teams to understand requirements and deliver robust solutions.
• Implement data quality checks, validation processes, and automation frameworks.
• Tune Spark jobs and optimize SQL queries to ensure efficient processing of large datasets.
• Integrate data from multiple sources while ensuring consistency, accuracy, and reliability.
• Deploy, monitor, and troubleshoot data workflows in production environments.
• Follow best practices for coding standards, documentation, and version control.

Required Skills & Qualifications

• 5+ years of overall IT experience with 3+ years in Data Engineering.
• Strong hands-on experience with Azure Databricks and distributed processing frameworks.
• Expertise in Apache Spark and PySpark for large-scale data processing.
• Solid understanding of Big Data concepts and technologies such as Hadoop, HDFS, Hive, etc.
• Advanced proficiency in SQL, including complex query writing and query optimization.
• Strong understanding of ETL/ELT principles and data modeling fundamentals.
• Experience working in cloud environments, preferably Microsoft Azure.
• Excellent communication, analytical thinking, and problem-solving skills.

Preferred Skills (Nice to Have)

• Experience with Azure Data Factory, Azure Synapse, or other Azure analytics services.
• Knowledge of CI/CD pipelines and DevOps practices.
• Hands-on experience with Git, automation tools, and continuous integration workflows.

Job Types: Full-time, Permanent

Pay: ₹556,957.88 - ₹1,943,725.92 per year

Application Question(s):

• Mention your last working date

Experience:

• Azure: 5 years (Preferred)
• Azure Databricks: 5 years (Preferred)
• Apache Spark: 5 years (Preferred)
• Pyspark: 5 years (Preferred)
• SQL: 5 years (Preferred)
• ETL: 5 years (Preferred)
• Azure Synapse: 5 years (Preferred)
• Data Engineer: 5 years (Preferred)

Work Location: In person