Key Responsibilities

• Design, develop, and maintain scalable data pipelines using modern ETL/ELT frameworks.
• Implement and optimize data workflows using tools like Apache Airflow, Apache Spark, and Apache Kafka.
• Build and maintain cloud-based data infrastructure on platforms like AWS, GCP, or Azure.
• Develop and implement data lakehouse solutions, leveraging technologies such as Delta Lake, Apache Iceberg, or Apache Hudi.
• Design and implement Data Mesh architectures to enable decentralized data ownership and domain-driven design.
• Develop automated data quality checks and monitoring systems to ensure reliability and accuracy.
• Collaborate with data scientists, analysts, and cross-functional teams to understand data requirements and implement appropriate solutions.
• Optimize data warehouse performance, ensure efficient storage and retrieval, and maintain data documentation.
• Implement data security, compliance, and governance measures to align with industry standards.
• Troubleshoot data pipeline issues, perform root cause analysis, and ensure timely resolution.

Technical Requirements

• 5+ years of experience in data engineering or similar roles.
• Proficiency in Python, SQL, and shell scripting.
• Strong experience with ETL/ELT tools (Airflow, Informatica, Talend, dbt).
• Expertise in big data technologies (Hadoop, Spark, Hive, Presto, Trino).
• Experience with stream processing (Kafka, Kinesis, Apache Flink).
• Proficiency in cloud platforms (GCP/Azure services).
• Deep understanding of data modeling, data warehouse, and data lakehouse concepts.
• Experience implementing Data Mesh architecture principles.
• Knowledge of data versioning, governance, and security best practices.
• Experience with version control systems (Git) and CI/CD pipelines.

Job Type: Full-time

Pay: Rs150,000.00 - Rs450,000.00 per month

Work Location: In person