Senior Data Engineer

Job Description

We are seeking a highly skilled Senior Data Engineer with hands-on experience in designing and implementing large-scale, distributed data processing systems. The candidate will play a key role in developing and optimizing data pipelines that enable data-driven decision-making across the organization.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines for both batch and streaming data processing.
• Work with distributed data processing technologies such as Hadoop, Apache Spark, Apache Flink, and Apache Kafka to handle large-volume, real-time, and batch data workloads.
• Collaborate closely with data scientists, analysts, and product teams to understand data requirements and deliver structured, high-quality datasets.
• Build and manage ETL/ELT workflows ensuring data accuracy, quality, and performance.
• Implement data integration frameworks, monitor data flow, and ensure reliability and consistency across systems.
• Write efficient, reusable, and modular code in Java and Python to support data transformations, automation, and monitoring.
• Optimize data storage and retrieval mechanisms for better performance and scalability.
• Ensure data governance, security, and compliance within the data platform.
• Provide technical guidance and mentorship to junior engineers when needed.

Technical Skills Required

• Programming Languages: Strong proficiency in Java and Python.
• Big Data Ecosystem: Proven experience with Hadoop, Spark, Flink, Kafka, and related distributed computing frameworks.
• Data Pipeline Development: Experience building both batch and real-time (stream) data processing pipelines.
• Data Storage Systems: Familiarity with HDFS, Hive, HBase, or other NoSQL systems.
• Version Control & CI/CD: Working knowledge of Git, Jenkins, or equivalent tools.
• Cloud Platforms (Optional): Exposure to AWS/GCP/Azure data services is desirable.
• Domain Knowledge: Experience in AMI (Advanced Metering Infrastructure) or Energy/Utilities domain will be an added advantage.

Job Types: Full-time, Permanent

Pay: Up to ₹1,200,000.00 per year

Work Location: In person