Senior Data Engineer (SWF)

Job Title: Senior Data Engineer (AWS | Python | ETL | QuickSight)

Location: Pune
Experience: 7-9 years

About the Role

We are looking for a Senior Data Engineer with strong expertise in building scalable ETL pipelines, performing data transformations using AWS Glue and Athena, and developing interactive reporting solutions with Amazon QuickSight.

The ideal candidate should have hands-on experience with AWS Cloud services, Python, and infrastructure automation using Terraform, along with experience in containerization and CI/CD pipelines via GitHub workflows.

Key Responsibilities

• Design, develop, and maintain robust ETL jobs to extract, transform, and load data from diverse sources into the data lake and warehouse.
• Implement data transformation pipelines using AWS Glue, integrating with Athena for analytical querying and optimization.
• Develop and automate QuickSight dashboards and reports for business insights and KPI tracking.
• Write clean, efficient, and maintainable Python code for data processing and orchestration.
• Containerize data applications and jobs using Docker for portability and scalability.
• Write and maintain unit and integration tests to ensure code reliability and data quality.
• Deploy and manage infrastructure and applications using GitHub Actions workflows for CI/CD.
• Work extensively with AWS services including IAM, S3, Lambda, Snowflake, Lake Formation, and DynamoDB.
• Manage and automate cloud infrastructure using Terraform for consistent, version-controlled deployments.
• Collaborate with data scientists, analysts, and business stakeholders to design end-to-end analytical solutions.

Required Skills & Experience

• 6+ years of experience in data engineering or related field.
• Strong proficiency in Python, including data manipulation and automation.
• Hands-on experience with AWS Glue, Athena, and QuickSight.
• Deep understanding of ETL design principles and best practices.
• Experience with Docker for containerization.
• Proficient in AWS CLI and managing AWS resources.
• Familiarity with Terraform for infrastructure as code.
• Experience writing unit and integration tests using frameworks like pytest.
• Experience with GitHub Actions or other CI/CD tools for deployment automation.
• Strong problem-solving and analytical skills with attention to detail.

Good to Have

• Prior experience in the automotive domain or working with vehicle data and telematics systems.
• Familiarity with data governance and security frameworks in AWS.

Education

• Bachelor’s degree in computer science, Information Technology, or a related field.