Data Engineer

We are seeking an experienced Data Engineer to design, build, and optimize scalable data pipelines and data infrastructure solutions. This role involves working with cloud platforms, big data frameworks, and streaming technologies to enable both batch and real-time data processing for business insights and analytics.

Key Responsibilities:

• Design, implement, and maintain scalable data pipelines using AWS cloud services and other modern data tools.
• Develop batch and stream processing workflows with Apache Spark and Apache Kafka.
• Build and optimize ETL/ELT processes using Python and Spark for data ingestion, transformation, and integration.
• Design data models and schemas to support efficient data storage and retrieval.
• Implement infrastructure as code (IaC) using Terraform for provisioning and managing cloud resources.
• Collaborate with data scientists, analysts, and other engineering teams to deliver reliable and performant data solutions.
• Ensure data quality, consistency, and security across pipelines and storage systems.