Pyspark Developer

ValueLabs is Hiring: Data Engineer PySpark | Cloudera Data Platform (CDP)

Location: Dubai
Experience: 5+ Years
Employment Type: Full-Time

About ValueLabs

ValueLabs is a global technology company committed to delivering innovative solutions and exceptional client experiences. We are expanding our data engineering team in Dubai and are looking for a skilled Data Engineer with strong expertise in PySpark and the Cloudera Data Platform (CDP).

Role Overview

As a Data Engineer, you will design and maintain scalable ETL pipelines, ensuring high data quality, performance, and reliability across our data ecosystem.

Key Responsibilities

• Build and optimize ETL pipelines using PySpark on CDP
• Ingest data from RDBMS, APIs, and file systems into data lakes/warehouses
• Cleanse and transform large datasets to support analytics
• Tune PySpark code and Cloudera components for performance
• Implement data quality checks and validation routines
• Automate workflows using Apache Oozie, Airflow, or similar tools
• Monitor pipeline health and troubleshoot issues
• Collaborate with cross-functional teams to meet data requirements
• Document processes, code, and configurations

Required Qualifications

• Bachelors/Masters in Computer Science, Data Engineering, or related field
• Minimum 5 years of experience in Data Engineering roles

Technical Skills

• PySpark (RDDs, DataFrames, optimization)
• Cloudera Data Platform (Cloudera Manager, Hive, Impala, HDFS, HBase)
• Data Warehousing & SQL (Hive, Impala)
• Big Data Tools (Hadoop, Kafka)
• Workflow Orchestration (Oozie, Airflow)
• Linux scripting and automation

Soft Skills

• Strong analytical and problem-solving abilities
• Excellent communication skills
• Team player with the ability to work independently
• Detail-oriented with a focus on data accuracy