Azure Data Engineer

Hiring Azure Data engineer with 5+ years of experience for PAN India

Required Technical Skillset - Azure, Databricks, Data build tool (DBT)

Job Description

Role Overview:

We are seeking a Data Engineer with strong expertise in dbt (Data Build Tool) as the primary skill and Databricks as a secondary capability. The role focuses on building scalable data transformation workflows, ensuring data quality, and enabling analytics through modern data stack practices.

Shape

Key Responsibilities

Data Modeling and Transformation:

Design, build, and maintain robust and scalable data models using dbt and SQL. Create reusable transformations and ensure data is structured for efficient analysis.

Pipeline Development:

Develop and optimize ELT pipelines, leveraging dbt for transformations and Databricks for compute and orchestration when needed.

Quality Assurance:

Implement and manage data quality tests within dbt to ensure data integrity, accuracy, and consistency.

Documentation and Governance:

Maintain comprehensive documentation for dbt models, data lineage, and dependencies.

CI/CD and Version Control:

Apply software engineering best practices using Git and set up CI/CD pipelines for automated dbt project deployments.

Collaboration:

Partner with data analysts, data scientists, and business stakeholders to deliver clean, reliable datasets for BI and ML use cases.

Shape

Essential Skills

Technical Skills:

dbt (Primary):

Hands-on expertise with dbt Core and dbt Cloud, including writing models, tests, macros, and using the CLI.

SQL:

Expert-level proficiency in writing complex, efficient SQL queries for data transformation.

Databricks (Secondary):

Experience with Azure Databricks for data processing and orchestration. Familiarity with Spark is a plus.

Azure Data Platform:

Knowledge of Azure Synapse Analytics, Azure Data Lake Storage, and Azure Data Factory.

Programming:

Proficiency in Python for scripting and pipeline automation. Familiarity with Jinja for dbt templating.

Data Warehousing:

Strong understanding of dimensional modeling, star/snowflake schemas, and data warehousing concepts.

Version Control & DevOps:

Experience with Git and CI/CD tools (Azure DevOps or similar).