Job Description: Azure Data Engineering (ADF, Python, SQL & DW) – Lead Programmer Analyst
We are looking for a highly skilled and experienced Azure Data Engineering (ADF, Python, SQL & DW) as a Lead Programmer Analyst.
Technical Expertise:
-
Experience should have 5 Years to 7 Years of Azure Data Engineering.
-
Strong expertise in Azure Data Factory (ADF) – pipelines, activities, triggers, linked services, and integration runtimes.
-
Proficiency in Python for scripting, data wrangling, and automation.
-
Experience with Azure Data Lake Storage (ADLS), Azure SQL, Synapse Analytics, or Databricks.
-
Strong knowledge of SQL (queries, stored procedures, performance tuning).
-
Understanding of data modeling, data warehousing, and data integration patterns.
-
Experience working in CI/CD environments (Azure DevOps preferred).
-
Familiarity with logging, monitoring, and debugging ADF pipelines.
-
Knowledge in Databricks
-
Understanding on the SDLC.
-
Understanding on the Agile methodologies.
-
Communication with customer and producing the Daily status report.
-
Should have good oral and written communication.
-
Should be a good team player.
-
Should be proactive and adaptive.
Key Responsibilities:
-
Design, develop, and maintain ADF pipelines for data ingestion, transformation, and orchestration.
-
Integrate Python scripts within ADF pipelines for advanced data processing and automation.
-
Work with Azure services such as Data Lake, Synapse Analytics, Databricks, Event Hub, and Key Vault.
-
Build and optimize ETL/ELT solutions for batch and real-time processing.
-
Collaborate with data engineers, analysts, and business teams to understand requirements and deliver solutions.
-
Implement best practices for data security, governance, and performance optimization.
-
Perform troubleshooting, monitoring, and tuning of pipelines for reliability and efficiency.
-
Prepare and maintain technical documentation for all development activities.