Role: Senior Engineer - Azure Data-bricks
Experience: 5–8 years
Location: Chennai, Kochi, Mumbai
Work Mode : On-site

About the Role

We are seeking a highly skilled Senior Engineer with strong expertise in Azure Databricks, ETL development, and cloud-based data engineering. The ideal candidate will have hands-on experience in building scalable data pipelines, integrating with Azure Data Lake Storage, and working with tools like Apache NiFi. This role requires strong technical capabilities, problem-solving skills, and the ability to work collaboratively with cross-functional teams.

Key Responsibilities

• 
  Design, build, and manage scalable ETL pipelines using Azure Databricks and PySpark.

• 
  Schedule and orchestrate tasks on Databricks within end-to-end workflows.

• 
  Perform regular application maintenance, health checks, and incident restoration.

• 
  Optimize code, workflows, and system performance for high-traffic, load-balanced services.

• 
  Automate routine operational tasks to improve system reliability and efficiency.

• 
  Work closely with IT teams, business stakeholders, and occasionally interact directly with customers.

• 
  Participate in Agile ceremonies such as standups, demos, grooming, and solutioning sessions.

Required Technical Skill Set

• 
  Azure Databricks: Hands-on experience building scalable ETL pipelines and integrating with Azure Data Lake Storage.

• 
  ETL Development: Strong expertise in developing and maintaining large-scale data integration workflows.

• 
  Azure Data Lake Storage: Proficiency in using ADLS within cloud data architectures.

• 
  PySpark: Strong working knowledge of PySpark for data transformation and processing.

• 
  Apache NiFi: Secondary expertise in data ingestion, transformation, and flow management.

• 
  Azure Cloud Services: Familiarity with Azure Synapse, Azure SQL, and related data engineering services.

Must-Have Competencies

• 
  Strong hands-on experience with Azure Databricks for ETL and big-data pipelines.

• 
  Proficiency with Azure Data Lake Storage and cloud-based data architectures.

• 
  Experience working with Apache NiFi for data ingestion and workflow orchestration.

• 
  Strong working knowledge of PySpark.

• 
  Excellent problem-solving skills for optimizing and troubleshooting ETL workflows.

Good-to-Have Competencies

• 
  Experience with additional Azure services such as Azure Synapse, Azure Data Factory, and Azure Functions.

• 
  Understanding of streaming data pipelines and real-time data processing.

• 
  Ability to work effectively with data scientists, analysts, and cross-functional teams.

• 
  Strong communication skills to present and explain technical solutions.