ADF Data Engineer

Overview:

We are seeking an experienced Azure Data Engineer to join our team in a hybrid Developer. This role focuses on enhancing and supporting existing Data & Analytics solutions by leveraging Azure Data Engineering technologies. The engineer will work on developing, maintaining, and deploying IT products and solutions that serve various business users, with a strong emphasis on performance, scalability, and reliability.

Must-Have Skills:

• 
  ADF

• 
  Databricks

Key Responsibilities:

• Incident classification and prioritization
  
• Log analysis and trend identification
  
• Coordination with Subject Matter Experts (SMEs)
  
• Escalation of unresolved or complex issues
  
• Root cause analysis and permanent resolution implementation
  
• Stakeholder communication and status updates
  
• Resolution of complex and major incidents
  
• Code reviews (Per week 2 per individual) to ensure adherence to standards and optimize performance
  
• Bug fixing of recurring or critical issues identified during operations
  
• Gold layer tasks, including enhancements and performance tuning.

• 
  Design, develop, and support data pipelines and solutions using Azure data engineering services.

• 
  Implement data flow and ETL techniques leveraging Azure Data Factory, Databricks, and Synapse.

• 
  Cleanse, transform, and enrich datasets using Databricks notebooks and PySpark.

• 
  Orchestrate and automate workflows across services and systems.

• 
  Collaborate with business and technical teams to deliver robust and scalable data solutions.

• 
  Work in a support role to resolve incidents, handle change/service requests, and monitor performance.

• 
  Contribute to CI/CD pipeline implementation using Azure DevOps.

Technical Requirements:

• 
  6 to 8 years of experience in IT and Azure data engineering technologies.

• 
  Strong experience in Azure Databricks, Azure Synapse, and ADLS Gen2.

• 
  Proficient in Python, PySpark, and SQL.

• 
  Experience with file formats such as JSON and Parquet.

• 
  Working knowledge of database systems, with a preference for Teradata and Snowflake.

• 
  Hands-on experience with Azure DevOps and CI/CD pipeline deployments.

• 
  Understanding of Data Warehousing concepts and data modeling best practices.

• 
  Familiarity with SNOW (ServiceNow) for incident and change management.

Non-Technical Requirements:

• 
  Ability to work independently and collaboratively in virtual teams across geographies.

• 
  Strong analytical and problem-solving skills.

• 
  Experience in Agile development practices, including estimation, testing, and deployment.

• 
  Effective task and time management with the ability to prioritize under pressure.

• 
  Clear communication and documentation skills for project updates and technical processes.

Technologies:

• 
  Azure Data Factory

• 
  Azure Databricks

• 
  Azure Synapse Analytics

• 
  Unity Catalog

• 
  PySpark / SQL

• 
  Azure Data Lake Storage (ADLS), Blob Storage

• 
  Azure DevOps (CI/CD pipelines)

Nice-to-Have:

• 
  Experience with Business Intelligence tools, preferably Power BI

• 
  DP-203 certification (Azure Data Engineer Associate)