Data Engineer (ETL)

Data Engineer (ETL)

Experience: 7–8 Years
Location: Remote
Shift: 11:00 AM – 9:00 PM IST
Openings: 1

Job Summary

We are looking for an experienced Data Engineer with 7–8 years of strong hands-on experience in ETL/ELT, Python, PySpark, Azure Databricks, Snowflake, and SQL.

The ideal candidate will have a strong background in cloud data engineering and should also have experience with Terraform, CI/CD, Azure Functions, Service Bus, and AI/ML/LLM technologies.

Key Responsibilities

• Design, develop, and optimize scalable ETL/ELT data pipelines.
• Build robust data solutions using Python, PySpark, Azure Databricks, and Snowflake.
• Develop, troubleshoot, and optimize complex SQL queries and data transformations.
• Work with Azure Databricks and Unity Catalog for data processing, governance, and security.
• Implement cloud infrastructure using Terraform and Infrastructure as Code practices.
• Build and maintain CI/CD pipelines using GitHub Actions, Git, Artifactory, Sonar, and related tools.
• Integrate Azure Functions, Azure Service Bus, and other Azure cloud services.
• Monitor and optimize pipeline performance, scalability, reliability, and cost.
• Implement data quality, security, and governance best practices.
• Explore and leverage AI/ML and LLM capabilities within data engineering solutions.
• Collaborate with cross-functional teams in an Agile environment.

Required Skills

• 7–8 years of professional Data Engineering experience.
• Strong ETL/ELT development experience.
• Excellent Python programming skills.
• Strong experience with Apache Spark, PySpark, and Spark SQL.
• Hands-on experience with Azure Databricks and Unity Catalog.
• Strong experience with Snowflake.
• Advanced SQL development and query optimization skills.
• Experience with Terraform / Infrastructure as Code.
• Experience with CI/CD tools and practices, particularly:
• GitHub Actions
• Git
• Artifactory
• Sonar
• Hands-on experience with:
• Azure Functions
• Azure Service Bus
• Other Azure services
• Practical experience with AI/ML and LLM technologies.

Preferred Qualifications

• Experience working with Azure cloud-native data platforms.
• Strong understanding of data architecture and modern data engineering patterns.
• Experience with data orchestration, monitoring, performance tuning, and optimization.
• Experience working in Agile/Scrum environments.

Pay: ₹130,000.00 - ₹160,000.00 per month

Work Location: Remote