Data Engineer

Job Description :

Requirements:

A minimum of 6+ years of experience in managing and developing solutions on the Azure data platform.

Extensive experience with Azure Data Factory, SQL Server database design, and ETL processes.

Primary Skills :

• Proven experience in Microsoft Fabric for data integration, orchestration, and analytics.
• Strong proficiency in PySpark for scalable data transformation and processing.
• Expertise in Azure Databricks and Spark SQL for advanced ETL and data modeling.
• Strong hands-on knowledge of Azure Synapse Analytics for warehousing and analytical workloads.
• Proficiency in SQL and relational databases, with focus on performance tuning and query optimization.
• Solid understanding of ETL/ELT processes, Delta Lake, and data lakehouse architectures.
• Knowledge of Azure Data Factory, Azure DevOps, and CI/CD pipelines for deployment automation.
• Understanding of data governance, security, and compliance (Key Vault, RBAC, Managed Identities).

*

Key Responsibilities

• Design, develop, and maintain scalable, end-to-end data pipelines using Microsoft Fabric, PySpark, and Databricks.
• Build and optimize data models and ETL workflows to support analytics, reporting, and machine learning.
• Integrate data from relational databases, APIs, and flat files into Azure Data Lake (ADLS) and Synapse.
• Implement Delta Lake architecture to ensure ACID compliance and maintain historical data versions.
• Collaborate with data architects, analysts, and business teams to translate business requirements into technical solutions.
• Optimize Spark and SQL performance for high-volume data workloads.
• Implement monitoring, alerting, and logging solutions using Azure Monitor and Log Analytics.
• Contribute to data governance and security standards, ensuring data quality and reliability across environments.
• Mentor junior engineers and promote best practices in modern data engineering and Azure cloud technologies.

Preferred Skills

• Experience with Power BI or similar visualization tools for integrated analytics.
• Exposure to real-time data processing and event-driven architectures (Kafka, Event Hubs).
• Familiarity with CI/CD best practices using Azure DevOps and Git-based workflows.
• Strong problem-solving, analytical, and communication skills.

Email: aswini.u@applogiq.org
Phone: 96293 11599

Job Type: Full-time

Work Location: In person