Senior Data Engineer

Senior Data Engineer (Lead)

Bengaluru, Karnataka
Full-Time | Hybrid (3 Days Office / 2 Days Remote)
Experience Required: 6–8 Years

Job Summary

We are looking for a highly skilled Senior Data Engineer (Lead) to drive large-scale data modernization and analytics transformation initiatives. The ideal candidate will be a hands-on technical leader with strong expertise in Databricks, Apache Spark, Python, and AWS.

You will lead the migration from legacy data platforms to a modern Lakehouse architecture while ensuring high performance, scalability, and enterprise-grade data governance.

Key Responsibilities

• Design and implement scalable data architecture using Databricks and Apache Spark
• Lead migration from legacy warehouse platforms to Lakehouse architecture
• Develop robust ETL/ELT pipelines using Python and advanced SQL
• Build batch and real-time data pipelines using Spark, Kafka/Kinesis
• Optimize performance, storage formats, and data models
• Implement orchestration using Apache Airflow
• Manage CI/CD pipelines and infrastructure using Jenkins and Terraform
• Ensure enterprise data governance, lineage, and KPI alignment
• Implement monitoring, observability, and SLA frameworks
• Mentor and guide senior and mid-level engineers

Required Skills & Qualifications

• 6–8+ years of experience in Data Engineering
• Strong hands-on expertise in Databricks and Apache Spark
• Advanced Python programming skills
• Strong SQL knowledge (complex joins, window functions, performance tuning)
• Solid AWS experience (S3, Lambda, EMR)
• Experience with Kafka, Kinesis, or Spark Streaming
• Knowledge of Docker and Kubernetes
• Experience with Terraform or CloudFormation
• Strong communication and stakeholder management skills

Preferred Certifications (At least one required)

• Databricks Certified Data Engineer – Professional
• AWS Solutions Architect – Associate or Professional

Additional Requirements

• Immediate to 30 days’ notice period preferred
• Minimum 1.5 years tenure with previous employers
• Local candidates preferred
• Comfortable working in a hybrid model
• Ability to work with US-based stakeholders (late evening overlap required)

Job Type: Full-time

Pay: From ₹2,400,000.00 per year

Application Question(s):

• How many years of hands-on experience do you have in Data Engineering
• Do you have 6 to 8 years of experience in Data Engineering, distributed systems, and platform architecture with clear technical ownership
• Have you maintained a minimum tenure of 1.5 years with each previous employer
• How many years of hands-on experience do you have with Databricks
• How many years of experience do you have working with Apache Spark

Work Location: In person