Platform Data Engineer

Job Title: Platform Data Engineer

Location: Dubai, UAE

Responsibilities:

Design, build, and maintain scalable data and compute platforms to support data product development.
Manage and optimize distributed compute environments (e.g., Spark) to ensure efficient processing of large-scale data workloads.
Develop, deploy, and maintain workflow orchestration pipelines using Airflow for reliable data processing and scheduling.
Operate and maintain streaming data platforms (e.g., Kafka) to enable real-time data ingestion and processing.
Ensure high availability, reliability, and performance of platform infrastructure and services.
Monitor platform health and implement improvements for scalability, security, and cost efficiency.
Collaborate with data engineers, analysts, and product teams to enable efficient development and deployment of data products.
Troubleshoot platform issues and implement solutions to improve system stability and operational efficiency.

Requirements:

Proven experience in platform engineering, data infrastructure, or data engineering roles.
Strong hands-on experience with distributed computing frameworks such as Apache Spark.
Experience with workflow orchestration tools (e.g., Apache Airflow) for managing data pipelines.
Knowledge of streaming technologies such as Apache Kafka for real-time data processing.
Understanding of scalable data platform architecture and infrastructure management.
Experience ensuring system reliability, high availability, and performance optimization.
Familiarity with security best practices and cost optimization for data platforms.
Strong problem-solving skills and the ability to manage end-to-end platform operations.