cloudera data engineer

data analyst

data engineer

data scientist

senior data engineer

senior data engineer pyspark python

D4 Insight

<div>Location: Dubai 
Experience: 8+ Years 
Work Mode: Hybrid (3 Days Onsite)
<h3 class="jobSectionHeader">
Role Overview:</h3>

We are seeking a highly experienced Senior Data Engineer with expertise in PySpark, Python, Hadoop, and large-scale data engineering pipelines.

The role involves building secure and scalable data platforms, enabling advanced analytics, and supporting machine learning workflows.
<h3 class="jobSectionHeader">
Key Responsibilities:</h3>
<ul><li>Acquire business requirements and perform exploratory data analysis (EDA).</li>
<li>Build scalable and high-performance data pipelines using Python, Spark, and Hadoop.</li>
<li>Perform feature engineering and support machine learning data workflows.</li>
<li>Collaborate with Data Scientists, Analytics teams, and stakeholders in Agile environments.</li>
<li>Optimize distributed data processing and ensure data quality and governance.</li>
</ul><h3 class="jobSectionHeader">Required Skills:</h3>
<ul><li>Strong expertise in Python, PySpark, Spark optimization, Hadoop, and SQL</li>
<li>Experience in enterprise data engineering and ML pipelines</li>
<li>Familiarity with Git, Agile methodologies, and cloud-native data environments</li>
<li>Strong analytical and communication skills</li>
</ul>
 Apply Now: [email protected]</div>

Senior Data Engineer – PySpark, Python

<div>
 
 Location:
 
 Dubai
 
 
 
 Experience:
 
 8+ Years
 
 
 
 Work Mode:
 
 Hybrid (3 Days Onsite)
 
 
 
 Role Overview
 
 
 
 We are seeking a highly experienced
 
 Senior Data Engineer
 
 with expertise in
 
 PySpark, Python, Hadoop, and large-scale data engineering pipelines
 
 .
 
 
 The role involves building secure and scalable data platforms, enabling advanced analytics, and supporting machine learning workflows.
 
 
 
 Key Responsibilities
 
 
 
 <ul>
 <li>
 Acquire business requirements and perform exploratory data analysis (EDA).
 </li>
 <li>
 Build scalable and high-performance data pipelines using Python, Spark, and Hadoop.
 </li>
 <li>
 Perform feature engineering and support machine learning data workflows.
 </li>
 <li>
 Collaborate with Data Scientists, Analytics teams, and stakeholders in Agile environments.
 </li>
 <li>
 Optimize distributed data processing and ensure data quality and governance.
 
 
 </li>
 </ul>
 
 Required Skills
 
 
 
 <ul>
 <li>
 Strong expertise in Python, PySpark, Spark optimization, Hadoop, and SQL
 </li>
 <li>
 Experience in enterprise data engineering and ML pipelines
 </li>
 <li>
 Familiarity with Git, Agile methodologies, and cloud-native data environments
 </li>
 <li>
 Strong analytical and communication skills
 
 
 </li>
 </ul>
 📩
 
 Apply Now: careers@d4insight.com
 
</div>

GSSTech Group

<div>We are looking for a highly skilled Senior Data Engineer with strong expertise in PySpark, Python, and Big Data technologies to design, build, and optimize scalable data platforms and pipelines. The ideal candidate will work closely with Data Scientists, Business Analysts, and Analytics Delivery teams to develop high-performance analytics solutions and support advanced data science initiatives. 
Requirements 
 
Key Responsibilities<ul><li>
Collaborate with Analytics Delivery Leads and Lead Data Engineers to understand business requirements and deliver impactful data solutions.</li><li>
Work closely with Data Scientists and cross-functional teams to solve complex business problems through data engineering solutions.</li><li>
Manage data onboarding, data access, and stakeholder coordination for analytics initiatives.</li><li>
Design, develop, and maintain scalable, secure, and high-performance data pipelines.</li><li>
Acquire, ingest, process, and transform large-scale structured and unstructured datasets.</li><li>
Implement data engineering best practices for building reliable and production-ready data platforms.</li><li>
Perform data wrangling, cleansing, transformation, and feature engineering for machine learning and analytics use cases.</li><li>
Design and develop modular data pipelines to generate reusable features and modeling datasets.</li><li>
Build and optimize data architectures supporting advanced analytics and machine learning workloads.</li><li>
Contribute to data platform design by selecting appropriate technologies across Big Data, SQL, and NoSQL ecosystems.</li><li>
Ensure data quality, integrity, governance, security, and scalability across the data lifecycle.</li><li>
Participate in Agile squads and collaborate effectively with stakeholders across business and technology teams.</li><li>
Contribute to enterprise data architecture strategy and roadmap aligned with business objectives.</li></ul>
Required Technical SkillsCore Data Engineering<ul><li>
Strong experience in Data Engineering and Big Data solutions.</li><li>
Expert-level proficiency in PySpark.</li><li>
Strong programming experience in Python.</li><li>
Experience building large-scale distributed data processing pipelines.</li></ul>
Data Processing &amp; ETL<ul><li>
Data Ingestion</li><li>
Data Transformation</li><li>
Data Wrangling</li><li>
Data Preparation</li><li>
Data Modeling</li><li>
Feature Engineering</li><li>
ETL/ELT Development</li><li>
Batch Processing</li><li>
Data Pipeline Development</li></ul>
Big Data Technologies<ul><li>
Apache Spark / PySpark</li><li>
Distributed Data Processing Frameworks</li><li>
Big Data Ecosystems</li></ul>
Database Technologies<ul><li>
Strong SQL expertise</li><li>
Experience with Relational Databases</li><li>
Experience with NoSQL Databases</li><li>
Data Warehousing Concepts</li></ul>
Cloud &amp; Analytics Platforms (Preferred)<ul><li>
Azure Data Platform</li><li>
AWS Data Services</li><li>
Google Cloud Data Services</li><li>
Databricks</li><li>
Hadoop Ecosystem</li></ul>
Machine Learning Data Support<ul><li>
Feature Engineering for ML Models</li><li>
Data Preparation for Analytics and AI Use Cases</li><li>
Building Modeling and Feature Tables</li><li>
Supporting Data Science Workloads</li></ul>
Engineering Best Practices<ul><li>
Software Engineering Principles</li><li>
Data Pipeline Optimization</li><li>
Data Quality Management</li><li>
Performance Tuning</li><li>
Scalability &amp; Security</li><li>
Version Control (Git)</li><li>
CI/CD Concepts</li><li>
Documentation Standards</li></ul>
Required Soft Skills<ul><li>
Strong stakeholder management and communication skills.</li><li>
Experience working with Business Analysts, Data Scientists, and Product Teams.</li><li>
Ability to translate business requirements into scalable data solutions.</li><li>
Strong analytical and problem-solving skills.</li><li>
Experience working in Agile/Scrum environments.</li><li>
Ability to work independently and within cross-functional teams.</li></ul>
Preferred Qualifications<ul><li>
Experience working in enterprise-scale data platforms.</li><li>
Experience supporting machine learning and advanced analytics initiatives.</li><li>
Exposure to cloud-native data engineering solutions.</li><li>
Experience designing modern data architectures and data lake environments.</li></ul></div>

Senior Data Engineer

emaratech

<div>About the Role
We are seeking a highly skilled and experienced Data Lake Cloud Engineer with a proven track record of designing, implementing, and maintaining large-scale cloud-based data lake platforms. This role requires a professional who can take ownership of our current data lake ecosystem, optimize its performance, and drive future enhancements with minimal oversight. The ideal candidate will have at least 5 years of hands-on experience in building enterprise-grade data lakes, strong cloud architecture expertise, and the ability to work with cutting-edge data ingestion, processing, and analytics tools. 
Key Responsibilities 
<ul><li>
Take ownership of the existing enterprise data lake platform, ensuring scalability, reliability, and performance.</li><li>
Lead the design, architecture, and implementation of cloud-native data lake solutions and integrations.</li><li>
Manage and optimize data ingestion pipelines on Oracle OCI, using tools such as Apache NiFi, Kafka, Batch Processing of data, Data captures, and or CSV.</li><li>
Design and implement pipelines for network data ingestion and file formats (e.g., Parquet, Avro, OCR, etc.), ensuring efficient storage, processing, and retrieval.</li><li>
Build, configure, and tune query engines such as Trino (Presto), Spark, and Hive for efficient analytics and reporting.</li><li>
Implement and maintain metadata management, data governance, and security frameworks.</li><li>
Monitor and troubleshoot system performance, ensuring SLAs are met for ingestion, processing, and query workloads.</li><li>
Automate platform deployment, monitoring, and maintenance with Infrastructure-as-Code (Terraform, CloudFormation, etc.).</li><li>
Collaborate with data engineers, analysts, and business teams to understand data requirements and deliver solutions that maximize data accessibility and usability.</li><li>
Keep the data platform up to date with the latest open-source and cloud-agnostic technologies, implementing upgrades and enhancements where needed.</li></ul> 
Requirements 
5+ years of proven, hands-on experience implementing and managing large-scale data lakes in the cloud (OCI). Strong expertise in: 
<ul><li>
Data ingestion &amp; orchestration: Apache NiFi, Apache Kafka, CSV, and others</li><li>
Data processing frameworks: Apache Spark, PySpark, Trino (Presto), Hive, Flink.</li><li>
Storage &amp; lakehouse architectures: Delta Lake, Apache Hudi, Iceberg, and cloud-native object storage (S3).</li><li>
Query &amp; analytics tools: Trino/Presto, SparkSQL, Metabase, or Apache Superset.</li><li>
Experience with data lake file formats such as Apache Parquet, Avro, ORC, CSV, etc. including ingestion, parsing, and analytics within a data lake.</li><li>
Solid understanding of data governance, lineage, cataloging, and security frameworks (Apache Atlas).</li><li>
Experience with CI/CD and IaC (ArgoCD, Terraform, Ansible) for automated deployments.</li><li>
Hands-on experience with cloud security best practices, including IAM, encryption, and network security.</li><li>
Strong proficiency in Python or Java for data engineering and automation tasks.</li><li>
Proven ability to work independently, quickly understand existing environments, and deliver results without extensive training.</li></ul> 
Preferred Skills 
<ul><li>
Exposure to machine learning workflows integrated with data lakes.</li><li>
Experience with real-time streaming data pipelines.</li><li>
Familiarity with containerization and orchestration (Docker, Kubernetes).</li><li>
Knowledge of cost optimization strategies in cloud-based data platforms.</li></ul></div>

MultiBank Group

<div>Welcome to MultiBank Group, a global financial pioneer established in 2005 in California and now proudly headquartered in Dubai, UAE. We specialize in delivering cutting-edge trading technology, unparalleled liquidity, and exceptional customer service. Our extensive range of financial products includes Forex, Metals, Shares, Indices, Commodities, and Cryptocurrency CFDs.
Join our thriving community of over 2 million clients across 100 countries, contributing to a daily trading volume exceeding US$ 35 billion. As a heavily regulated institution with oversight from 18+ financial regulators across 5 continents, and recipient of over 80 financial awards, MultiBank Group is devoted to innovation, excellence, and empowering our clients to achieve their financial goals.
Role Overview
We are seeking a Senior Machine Learning Engineer to join our AI team as a technical owner of ML products and infrastructure. This is a deeply hands-on engineering position for someone who builds and scales production AI systems used by real users in real-time environments. The right candidate operates across the full ML lifecycle &mdash; from model design through deployment, optimization, and ongoing performance in production &mdash; and contributes to the technical direction of the AI platform.
Key Responsibilities<ul><li>
Architect and implement robust ML systems in production environments, ensuring scalability, reliability, and performance from day one</li><li>
Build and deploy supervised, unsupervised, deep learning, and generative AI models into live production environments at scale</li><li>
Own technical design for ML pipelines, feature stores, training infrastructure, and inference systems, driving decisions that balance performance, cost, and maintainability</li><li>
Design and deliver RAG systems, fine-tuning pipelines, prompt engineering frameworks, and evaluation pipelines for production-grade LLM applications</li><li>
Implement and maintain CI/CD for ML, model versioning, monitoring, drift detection, and automated retraining pipelines</li><li>
Continuously optimize model performance, inference latency, cost efficiency, and reliability across live systems</li><li>
Collaborate with product managers, engineers, and data teams to translate business problems into scalable, maintainable AI solutions</li><li>
Mentor junior and mid-level ML engineers, establish best practices, and contribute to technical standards across the team</li><li>
Contribute to strategic decisions around data architecture, AI infrastructure, and cloud platform direction</li><li>
Work with mobile attribution and customer engagement data sources including Adjust, MoEngage, and Firebase for ML use cases such as churn prediction, personalization, and campaign optimization</li></ul>
Requirements<ul><li>
7 to 15 or more years of experience in software engineering, data science, or ML engineering</li><li>
Strong background in product companies, scale-ups, or enterprise AI platforms</li><li>
Proven track record of building production-grade AI systems, not solely notebooks or proof-of-concept work</li><li>
Comfortable owning systems end-to-end from data through model through deployment through monitoring</li><li>
Product-first engineering approach, not research-only profiles</li><li>
Advanced Python engineering skills with strong systems thinking and a focus on production quality</li><li>
Comfortable with fast iteration cycles and deploying models into live environments</li><li>
Ability to work directly and confidently with stakeholders and product owners</li><li>
Fintech or financial services experience is an advantage</li></ul>
Technical Skills
Machine Learning and AI: PyTorch, TensorFlow, XGBoost, LightGBM, Hugging Face (Transformers, Datasets, Diffusers)
LLM and GenAI: OpenAI and Anthropic APIs, LangChain, LlamaIndex; RAG architectures with vector DB and retrieval pipelines; embedding models (OpenAI, Cohere, open-source); Pinecone, Weaviate, Milvus, FAISS; fine-tuning via LoRA and PEFT frameworks; evaluation using RAGAS and custom pipelines
MLOps and Production: Docker, Kubernetes, MLflow, Weights and Biases, Airflow, Dagster, Prefect, GitHub Actions, GitLab CI, Evidently AI, Arize, custom observability stacks
Cloud: AWS (SageMaker, EKS, S3, Lambda), Azure ML, Azure Databricks, GCP
Data Stack: Databricks, Spark, PySpark, Delta Lake, Apache Iceberg, Lakehouse architectures
Why Join Us?<ul><li>
Work with one of the world&rsquo;s leading financial derivatives institutions.</li><li>
Competitive salary plus performance-based incentives.</li><li>
Access to a dynamic, international, and fast-growing environment.</li><li>
Strong opportunities for career progression within a global financial group.</li><li>
Be part of a business committed to innovation, excellence, and long-term growth.</li></ul>
Become part of our international community at MultiBank Group, dedicated to excellence, innovation, and shaping the future of finance.
MultiBank Group is an equal opportunity employer. We welcome applications from candidates of all backgrounds and do not discriminate on the basis of nationality, gender, age, religion, or disability.</div>

Senior Machine Learning Engineer

Maaloomatiia Academy

<div>
 
 Job Title
 
 
 
 
 Cloudera Data Engineer
 
 
 
 
 Job Scope
 
 
 
 We are seeking an experienced
 
 Cloudera Data Engineer
 
 with strong hands-on expertise in
 
 Cloudera CDP, Hadoop, Spark, Hive, and HDFS
 
 . The ideal candidate will be responsible for building and maintaining big data pipelines, optimizing data workflows, and supporting production environments to ensure reliability and performance.
 
 
 
 Main Duties And Responsibilities
 
 
 
 <ul>
 <li>
 Design, build, and maintain big data pipelines using Cloudera CDP components.
 </li>
 <li>
 Develop and optimize ETL/ELT workflows using Spark, Hive, and Hadoop ecosystems.
 </li>
 <li>
 Manage and maintain HDFS storage, ensuring high availability and performance.
 </li>
 <li>
 Perform production support activities, including issue resolution, monitoring, and performance tuning.
 </li>
 <li>
 Work with data scientists, analysts, and other engineering teams to support data-driven initiatives.
 </li>
 <li>
 Ensure data governance, security, and compliance across big data environments.
 </li>
 <li>
 Troubleshoot cluster-level issues related to resource utilization, scheduling, and configuration.
 </li>
 <li>
 Participate in solution design, architecture discussions, and technical documentation.
 
 
 
 </li>
 </ul>
 
 Position Requirements
 
 
 
 <ul>
 <li>
 Strong hands-on experience with Cloudera CDP.
 </li>
 <li>
 Proficiency in Hadoop ecosystem: HDFS, YARN, MapReduce, Hive, Impala.
 </li>
 <li>
 Strong experience working with Apache Spark (PySpark or Scala preferred).
 </li>
 <li>
 Solid understanding of distributed systems and big data architecture.
 </li>
 <li>
 Experience with production support and troubleshooting in big data environments.
 </li>
 <li>
 Strong SQL skills and experience optimizing large-scale queries.
 </li>
 <li>
 Familiarity with Linux, shell scripting, and automation tools.
 </li>
 <li>
 Experience with version control (Git) and CI/CD workflows is a plus.
 
 
 
 </li>
 </ul>
 
 Education
 
 
 
 Bachelor's degree in Computer Science or any other related field
 
 
 
 Experience
 
 
 
 <ul>
 <li>
 Experience with cloud platforms (AWS, Azure, GCP) for big data workloads.
 </li>
 <li>
 Exposure to Kafka, NiFi, Oozie, or similar data orchestration tools.
 </li>
 <li>
 Knowledge of Docker, Kubernetes, or containerized environments.
 </li>
 <li>
 Understanding of security, encryption, and data governance concepts.
 </li>
 </ul>
</div>

Cloudera Data Engineer

Seattle Aviation Solutions

<div>
 
 About Seattle Aviation Solutions (SAS)
 
 
 
 
 
 Seattle Aviation Solutions (SAS) is a leading global distributor of aftermarket parts for business and commercial aircraft. Headquartered in Seattle, with operational hubs in Dubai and Turkey, SAS is committed to providing exceptional service and quality products to its clients worldwide. We are leveraging data and AI to revolutionize the way aviation parts are sourced, sold, and managed.
 
 
 
 
 
 Position: AI Associate
 
 
 
 
 
 Are you a technically brilliant and self-directed developer with a passion for working with large datasets and exploring the frontier of AI? Do you enjoy building elegant solutions with Python and want to play a foundational role in shaping a company's data and AI strategy? If so, we invite you to join Seattle Aviation Solutions as an AI Associate.
 
 
 
 
 
 Key Responsibilities
 
 
 
 
 
 - Data Engineering &amp; Pipeline Development: Build and maintain scalable, reliable data pipelines and ETL processes to support analytics, reporting, and AI applications.
 
 
 
 
 
 - AI/ML Enablement: Collaborate with internal stakeholders to identify AI opportunities and develop prototypes that support business functions such as pricing, purchasing optimization, forecasting, and anomaly detection.
 
 
 
 
 
 - Python Development: Write clean, efficient, modular Python code to support data transformation, automation, web scraping, API integrations, and ML model deployment.
 
 
 
 
 
 - Data Architecture: Help define and maintain data infrastructure, schemas, and storage systems to ensure high-performance analytics across structured and unstructured datasets.
 
 
 
 
 
 - Independent Problem Solving: Take ownership of technical challenges, working independently to design and implement end-to-end solutions with minimal oversight.
 
 
 
 
 
 - Tooling and Infrastructure: Set up and manage tools for data analysis, model development, and experimentation (e.g., Jupyter, Airflow, Docker, Git).
 
 
 
 
 
 - Collaboration: Work closely with data and IT leadership and other technical team members to translate high-level ideas into deployable code and functional systems.
 
 
 
 
 
 - R&amp;D: Stay current on AI/ML trends, LLMs, and emerging technologies to help SAS map its AI future and continuously push innovation forward.
 
 
 
 
 
 Personal Attributes
 
 
 
 
 
 - Highly intelligent, curious, and motivated by technical challenges.
 
 
 - Strong problem-solver with a hacker mindset and bias for action.
 
 
 - Self-directed and capable of managing projects independently from concept to deployment.
 
 
 - Excellent communication skills and ability to work across technical and non-technical teams.
 
 
 
 
 
 Qualifications
 
 
 
 
 
 Experience: 2 to 4 years of experience in Python development, data engineering, or AI/ML roles.
 
 
 
 
 
 Skills:
 
 
 - Strong Python programming expertise.
 
 
 - Experience with data processing frameworks (e.g., Pandas, NumPy, SQLAlchemy, PySpark).
 
 
 - Familiarity with cloud environments (e.g., AWS, GCP, or Azure).
 
 
 - Proficiency with SQL and NoSQL databases (e.g., PostgreSQL, MongoDB, Redis).
 
 
 - Experience building and deploying ML models using libraries like scikit-learn, TensorFlow, or PyTorch.
 
 
 - Comfortable with DevOps tools (Git, Docker, CI/CD pipelines) and working in Linux environments.
 
 
 - Understanding of REST APIs and third-party data integrations.
 
 
 - Experience working with large datasets and solving performance bottlenecks.
 
 
 - Comfortable working within the framework of current tech stack (Snowflake, AWS, Fivetran, Tableau).
 
 
 
 
 
 Bonus: Exposure to LLMs, LangChain, Vector Databases (e.g., Pinecone, Weaviate), or NLP applications is highly desirable.
 
 
 
 
 
 Location: Dubai, Night Shift (willingness to work either EST or PST)
 
 
 
 
 
 Job Type: Full-time
 
 
 
 
 
 Benefits
 
 
 
 
 
 - Competitive salary and performance-based incentives
 
 
 - Comprehensive health insurance
 
 
 - Opportunity to define and build the AI roadmap of a global aviation company
 
</div>

Artificial Intelligence Specialist

Explore 1093+ Senior data engineer pyspark python in Dubai this June. Find roles across top companies and apply now to boost your career.

1093 senior data engineer pyspark python jobs In Dubai

Senior Data Engineer – PySpark, Python

Senior Data Engineer – PySpark, Python

Senior Data Engineer

Senior Data Engineer

Senior Machine Learning Engineer

Cloudera Data Engineer

Artificial Intelligence Specialist