Data Engineer

We are looking for an experienced Data Engineer with strong expertise in Databricks to design, build, and maintain scalable data pipelines and data platforms. The ideal candidate will work closely with data scientists, analysts, and business stakeholders to deliver high-quality, reliable, and performant data solutions.

Key Responsibilities

Design, develop, and maintain end-to-end data pipelines using Databricks

Build and optimize ETL/ELT workflows for structured and unstructured data

Develop scalable data processing solutions using Apache Spark (PySpark / Spark SQL)

Implement data ingestion frameworks from multiple data sources (RDBMS, APIs, files, streaming sources)

Perform data transformations, validations, and quality checks

Optimize Spark jobs for performance, cost, and scalability

Work with Delta Lake for data versioning, ACID transactions, and time travel

Collaborate with data scientists and analysts to support analytics and machine learning workloads

Ensure data security, governance, and compliance best practices

Troubleshoot and resolve data pipeline and performance issues

Document data models, pipelines, and technical designs

Technical Skills Required

Mandatory Skills

5–7 years of experience as a Data Engineer

Strong hands-on experience with Databricks

Proficiency in Apache Spark (PySpark preferred)

Strong SQL skills for data transformation and analysis

Experience with Delta Lake

Experience working with cloud platforms (AWS / Azure / GCP)

Knowledge of data warehousing concepts and data modeling

Experience with ETL tools and frameworks

Good to Have

Experience with streaming technologies (Kafka, Spark Streaming)

Knowledge of Python for data engineering

Familiarity with orchestration tools (Airflow, Azure Data Factory, etc.)

Experience with CI/CD for data pipelines

Exposure to machine learning pipelines is a plus

Job Type: Full-time

Pay: ₹700,000.00 - ₹1,000,000.00 per year

Work Location: In person