data analyst

data engineer

devops engineer

cloudera data engineer

ai/ml engineer

data pipeline engineer

Datamatics Technologies

<div>
 
 Cloudera Data Engineer
 
 
 
 Location:
 
 Remote (UAE Time Zone)
 
 
 Experience:
 
 5+ Years
 
 
 Availability:
 
 Immediate joiners or candidates who can start within 30 days preferred
 
 
 Role Overview
 
 
 We are seeking an experienced
 
 Cloudera Data Engineer
 
 to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.
 
 
 Key Responsibilities
 
 <ul>
 <li>
 
 Design, build, and maintain
 
 ETL/ELT pipelines
 
 using Spark, Hive, Impala, NiFi, and Oozie.
 
 </li>
 <li>
 
 Optimize data pipelines for
 
 performance, reliability, and cost efficiency
 
 across batch and streaming workloads.
 
 </li>
 <li>
 
 Develop
 
 automation scripts and reusable frameworks
 
 for data ingestion, transformation, and cleansing.
 
 </li>
 <li>
 
 Collaborate with Cloudera Administrators to
 
 tune performance
 
 and manage cluster resources efficiently.
 
 </li>
 <li>
 
 Troubleshoot and resolve
 
 data pipeline failures, performance bottlenecks, and data quality issues
 
 .
 
 </li>
 <li>
 
 Monitor
 
 cluster utilization, job performance, and data throughput
 
 using Cloudera Manager.
 
 </li>
 <li>
 
 Work with DevOps/Infrastructure teams to define and maintain
 
 CI/CD pipelines, automation for deployments
 
 , and monitoring of data jobs.
 
 </li>
 <li>
 
 Participate in
 
 incident management, root cause analysis, and preventive maintenance
 
 .
 
 </li>
 <li>
 
 Ensure pipelines meet
 
 SLAs, data quality, and latency requirements
 
 for analytics workloads.
 
 </li>
 </ul>
 
 Requirements
 
 <ul>
 <li>
 
 5+ years of experience as a
 
 Data Engineer
 
 in a Cloudera-based environment.
 
 </li>
 <li>
 
 Strong hands-on experience with
 
 Spark, Hive, Impala, NiFi, and Oozie
 
 .
 
 </li>
 <li>
 
 Solid understanding of
 
 ETL design, performance optimization
 
 , and
 
 data governance
 
 principles.
 
 </li>
 <li>
 
 Experience with
 
 Linux/Unix scripting
 
 and
 
 CI/CD integration
 
 .
 
 </li>
 <li>
 
 Excellent troubleshooting and communication skills.
 
 </li>
 </ul>
</div>

Cloudera Data Engineer

Datamatics Global Services Ltd

<div><h2 class="jobSectionHeader">Cloudera Data Engineer</h2>
Location: Remote (UAE Time Zone) 
Experience: 5+ Years 
Availability: Immediate joiners or candidates who can start within 30 days preferred<h3 class="jobSectionHeader">
Role Overview</h3>
We are seeking an experienced Cloudera Data Engineer to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.<h3 class="jobSectionHeader">
Key Responsibilities</h3><ul><li>
Design, build, and maintain ETL/ELT pipelines using Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Optimize data pipelines for performance, reliability, and cost efficiency across batch and streaming workloads.</li><li>
Develop automation scripts and reusable frameworks for data ingestion, transformation, and cleansing.</li><li>
Collaborate with Cloudera Administrators to tune performance and manage cluster resources efficiently.</li><li>
Troubleshoot and resolve data pipeline failures, performance bottlenecks, and data quality issues.</li><li>
Monitor cluster utilization, job performance, and data throughput using Cloudera Manager.</li><li>
Work with DevOps/Infrastructure teams to define and maintain CI/CD pipelines, automation for deployments, and monitoring of data jobs.</li><li>
Participate in incident management, root cause analysis, and preventive maintenance.</li><li>
Ensure pipelines meet SLAs, data quality, and latency requirements for analytics workloads.</li></ul><h3 class="jobSectionHeader">
Requirements</h3><ul><li>
5+ years of experience as a Data Engineer in a Cloudera-based environment.</li><li>
Strong hands-on experience with Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Solid understanding of ETL design, performance optimization, and data governance principles.</li><li>
Experience with Linux/Unix scripting and CI/CD integration.</li><li>
Excellent troubleshooting and communication skills.</li></ul>
BJoI7cAFtW</div>

HR Ways

About the job Data Engineer - (On-site | Karachi | PKR)Client company introduction: &#65279;The company is a growing consultancy specializing in data analytics, providing high-quality, impactful data solutions to clients based in the UK.Job Description: &#65279;*The Data Engineer *will play a pivotal role in shaping the companys data capabilities, working on projects for UK-based clients. Joining a new and dynamic team, this role offers the opportunity to implement best practices, introduce innovative tools and technologies, and contribute to tasks beyond traditional data engineering. The ideal candidate is proactive, versatile, and eager to make their own mark in a startup-like environment. You will receive mentorship from the UK-based founder, fostering professional growth and the chance to significantly impact the company's success.Key Responsibilities:Design and Implement Data Infrastructure:<ul><li>Develop, maintain, and optimize scalable data pipelines and architectures to support data analytics projects.</li><li>Implement best practices in data warehousing, ETL/ELT processes, and data integration to ensure efficient data flow.</li><li>Introduce and leverage tools and technologies that enhance data processing and storage capabilities.</li></ul>Implement Best Practices and Innovation:<ul><li>Establish data engineering standards and procedures to ensure data integrity, quality, and security.</li><li>Stay abreast of emerging technologies and industry trends, integrating innovative solutions where appropriate.</li><li>Evaluate and introduce tools (e.g., data pipeline frameworks, automation scripts) to improve workflow efficiency.</li></ul>Dynamic Role in a Startup Environment:<ul><li>Contribute to various aspects of the business, such as assisting with technical input for the company website, supporting social media initiatives, or aiding in recruitment efforts as needed.</li><li>Embrace the flexibility of a startup setting, proactively addressing challenges and seizing opportunities to add value.</li></ul>Collaboration and Team Support:<ul><li>Work closely with data analysts, project managers, and other team members to understand data requirements and deliver optimal solutions.</li><li>Participate in team meetings, providing technical insights and contributing to project planning and execution.</li><li>Mentor junior team members, sharing knowledge and fostering a collaborative learning environment.</li></ul>Client Engagement:<ul><li>Occasionally interact with UK-based clients to understand their data needs, provide updates, and ensure satisfaction with data solutions.</li><li>Translate complex technical concepts into clear, understandable language for clients and non-technical stakeholders.</li></ul>Data Quality and Continuous Improvement:<ul><li>Implement data validation and monitoring processes to maintain high data quality and reliability.</li><li>Gather feedback on data solutions and processes, identifying areas for improvement and implementing enhancements.</li><li>Track and report on data engineering metrics to monitor performance and identify optimization opportunities.</li></ul>Qualifications:<ul><li>Bachelor's degree in Computer Science, Information Systems, or a related field.</li><li>Minimum of 3 years of experience in data engineering, preferably within data analytics or consulting environments.</li><li>Proficiency in SQL and programming languages such as Python or Java.</li><li>Experience with cloud-based data services (e.g., Azure Data Factory, Azure SQL Database) and familiarity with Power BI integration.</li><li>Strong understanding of data warehousing concepts, ETL/ELT processes, and best practices in data management.</li><li>Excellent problem-solving skills and the ability to work independently in a dynamic environment.</li><li>Strong communication skills, with the ability to convey complex technical information to both technical and non-technical audiences.</li><li>A proactive mindset with a passion for innovation and continuous improvement.</li></ul>What We Offer:<ul><li>Opportunity to Make Your Mark: Join a growing team where your contributions will have a significant impact on the company's success.</li><li>Mentorship: Work closely with the UK-based founder, gaining valuable insights and advancing your professional development.</li><li>Dynamic Environment: Experience the excitement and challenges of a startup-like setting, where flexibility and innovation are highly valued.</li></ul>Client Exposure: Collaborate with UK-based clients, enhancing your international experience and expanding your professional network&#65279;Other Details:<ul><li>Experience: 3 years or above</li><li>Location: Karachi, Pakistan (on-site)</li><li>&#65279;Timings: 12:00PM - 9:00PM Summer &amp; 1:00PM - 10:00PM Winter</li><li>Working Days: Monday to Friday</li></ul>Apply at jobs@hrways.co (not com)&#65279;About HR Ways: &#65279;&#65279;&#65279;HR Ways is an Award winning Technical Recruitment Firm helping software houses and IT Product companies internationally and locally to find IT Talent. HR Ways is engaged by 300+ Employers worldwide ranging from worlds biggest SaaS Companies to most competitive Startups. We have entities in Dubai, Canada, US, UK, Pakistan, India, Saudi Arabia, Portugal, Brazil and other parts of the world. Join our WhatsApp Channel https://shorturl.at/983az to stay updated or visit www.hrways.co to know more.Job Type: Full-timeWork Location: In person

Data Engineer

Confiz

<div>We are seeking an experienced Azure Data Engineer (5+ years) with strong expertise in data warehousing on lakehouse architecture, data engineering, data quality, and governance. The ideal candidate will design and build scalable data pipelines on Microsoft Fabric, implement data quality frameworks, and operationalize governance processes using Microsoft Purview. This role combines technical delivery with data compliance, ensuring trusted and well-governed data across the organization.
Key Responsibilities
<ul><li>Design and develop batch data pipelines using Fabric Pipelines and PySpark.</li>
<li>Implement data quality rules, validation checks, and monitoring dashboards.</li>
<li>Develop data catalog, lineage, classification, and business glossary in Microsoft Purview.</li>
<li>Enforce data governance policies, standards, and best practices across Microsoft Fabric</li>
<li>Collaborate with DevOps to automate deployments using Azure DevOps.</li>
<li>Integrate data quality in CI/CD pipelines through Azure DevOps.</li>
<li>Ensure compliance with data privacy regulations (e.g., PDPL).</li>
<li>Partner with business and data stewards to improve overall data integrity and usability.</li>
</ul>
Required Skills &amp; Experience
<ul><li>5+ years of experience in data engineering, quality, and governance roles.</li>
<li>Strong hands-on experience with Azure Data Factory, Apache Spark (PySpark), and Azure Synapse/Fabric.</li>
<li>Proven expertise in Microsoft Purview for governance and catalog management.</li>
<li>Proficient in Python and SQL for data processing and validation.</li>
<li>Understanding of data quality dimensions, profiling, and cleansing techniques.</li>
<li>Strong analytical, problem-solving, and communication skills.</li>
<li>Experience with Great Expectations or similar data validation tools is a plus.</li>
</ul>
Preferred Qualifications
<ul><li>Microsoft Certified: Fabric Data Engineer Associate or equivalent certification.</li>
<li>Experience with Microsoft Fabric and AI-driven metadata or quality tools.</li>
</ul>
We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Principal Data Engineer – Data Quality & Governance

Systems Limited

<div>We are seeking a highly skilled Senior Data Engineer to design, build, and optimize scalable data pipelines and data platforms. The ideal candidate should have strong experience with cloud data engineering, ELT/ETL frameworks, and modern data technologies, enabling the business to make data-driven decisions efficiently.<h3 class="jobSectionHeader"> 
</h3>
<h3 class="jobSectionHeader">Responsibilities</h3>
<ul>
<li>Design, develop, and maintain scalable data pipelines for batch and real-time processing.
</li>
<li>Build and optimize data lake, data warehouse, and data mart architectures.
</li>
<li>Develop and enhance ETL/ELT workflows using best practices.
</li>
<li>Work with large datasets, ensuring data quality, integrity, and governance.
</li>
<li>Collaborate with Data Scientists, Analysts, and Product teams to understand data needs.
</li>
<li>Implement data modeling (Star, Snowflake, 3NF) for analytics and reporting.
</li>
<li>Optimize database performance, query tuning, and data storage solutions.
</li>
<li>Ensure data security, access control, and compliance standards.
</li>
<li>Lead technical design discussions and mentor junior data engineers.
</li>
<li>Work with CI/CD pipelines, version control, and automation tools.</li> 
</ul>
<h3 class="jobSectionHeader">Requirements:</h3>
<ul>
<li>Strong experience with cloud platforms: AWS / Azure / GCP.
</li>
<li>Hands-on experience in SQL, Python, and distributed data processing.
</li>
<li>Expertise in tools like Spark, Databricks, Hadoop, Hive, Airflow, Kafka or similar.
</li>
<li>Experience with data warehouses such as Snowflake, Redshift, BigQuery, or Synapse.
</li>
<li>Strong understanding of ETL/ELT, data modeling, and data architecture principles.
</li>
<li>Experience building real-time streaming pipelines (Kafka, Kinesis, EventHub).
</li>
<li>Proficiency with CI/CD, Git, DevOps workflows.
</li>
<li>Knowledge of data governance, security, and metadata management.</li></ul></div>

Managing Consultant Data Engineering

<div>We are seeking a Data Engineer with 8+ years of hands-on experience and a strong background in real-time and batch data processing, containerization, and cloud-based data orchestration. This role is ideal for someone passionate about building robust, scalable, and efficient data pipelines, and who thrives in agile, collaborative environments.
Key Responsibilities
<ul><li>Design, build, and maintain real-time data pipelines using streaming frameworks such as Kafka, Apache Flink, and Spark Structured Streaming.</li>
<li>Develop batch processing workflows with Apache Spark (PySpark)</li>
<li>Orchestrate and schedule data workflows using orchestration frameworks such as Apache Airflow and Azure Data Factory</li>
<li>Containerize applications using Docker, manage deployments with Helm, and run them on Kubernetes</li>
<li>Implement modern storage solutions using open formats such as Parquet, Delta Lake, and Apache Iceberg</li>
<li>Build high-performance analytics engines using tools like Trino or Presto</li>
<li>Collaborate with DevOps to manage infrastructure with Terraform and integrate with CI/CD pipelines via Azure DevOps</li>
<li>Ensure data quality and consistency using tools like Great Expectations</li>
<li>Write modular, well-tested, and maintainable Python and SQL code</li>
<li>Develop an observability layer to monitor and optimize performance across data pipelines</li>
<li>Lead and mentor a team of data engineers</li>
<li>Participate in agile ceremonies and contribute to sprint planning and reviews</li>
</ul>
Required Skills &amp; Experience
<ul><li>Advanced Python programming with a strong focus on modular and testable code</li>
<li>Strong knowledge of SQL and experience working with large-scale datasets</li>
<li>Hands-on experience with at least one major cloud platform (Azure preferred)</li>
<li>Solid experience with real-time data processing (Kafka, Flink, or Spark Streaming)</li>
<li>Expertise in Apache Spark (PySpark) for batch processing</li>
<li>Experience implementing lakehouse architectures and working with columnar storage (e.g., ClickHouse)</li>
<li>Proficient in using Azure Data Factory or Apache Airflow for data orchestration</li>
<li>Experience in building APIs to expose large datasets</li>
<li>Solid experience with Docker, Kubernetes, and Helm</li>
<li>Familiarity with data lake open formats such as Parquet, Delta Lake, and Iceberg</li>
<li>Basic experience with Terraform for infrastructure provisioning</li>
<li>Practical experience with data quality frameworks (e.g., Great Expectations)</li>
<li>Comfortable working in agile development teams</li>
<li>Proven ability in debugging and performance tuning of streaming and batch data jobs</li>
<li>Experience with AI-driven tools (e.g., text-to-SQL) is a plus</li>
<li>Bachelor&rsquo;s degree in computer science or related discipline</li>
</ul>
We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Associate Data Architect - Data Lake/Data Streaming

Godai Tech

About GodaitecAt Godaitec, we are a full-service technology thought partner dedicated to empowering innovation. We blend creativity, strategy, and technology to transform ideas into scalable, future-ready solutions. From Software Development and Solution Consulting to cutting-edge Data Analytics and Emerging Technologies (AI, IoT, Blockchain), we help businesses turn their data into strategic value.We are looking for a passionate Data Engineer to join our growing team. If you are driven by building robust data ecosystems and have a knack for solving complex data challenges, we want to hear from you.Role OverviewWe are seeking a skilled Data Engineer with up to 2 years of hands-on experience to build, optimize, and maintain our data infrastructure. You will play a critical role in our Data Analytics vertical, working on high-impact projects involving ETL pipelines, cloud data warehousing, and data governance. You will bridge the gap between raw data and actionable insights, ensuring our clients can make smarter, faster decisions.Key Responsibilities<ul><li>Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines using industry-standard tools (Informatica, Talend, Spark) to ingest data from various sources.</li><li>Data Modeling &amp; Architecture: Develop robust data models and architectures that support business intelligence and advanced analytics requirements.</li><li>Cloud Data Engineering: Implement and manage data solutions on cloud platforms (Databricks, Azure, AWS, or GCP).</li><li>Data Quality &amp; Governance: Establish frameworks for data quality, reliability, and governance to ensure data accuracy and compliance across the organization.</li><li>Orchestration &amp; Automation: Schedule and monitor workflows using orchestration tools like Airflow or Prefect to ensure timely data delivery.</li><li>Containerization &amp; CI/CD: Utilize Docker and Kubernetes for containerization and Jenkins for CI/CD pipelines to streamline deployment processes.</li><li>Collaboration: Work closely with Data Scientists and Analysts to prepare data for visualization and machine learning models.</li></ul>Candidate Profile &amp; Requirements1. Education &amp; Experience<ul><li>Bachelor&rsquo;s degree in computer science, Mathematics and statistics, or a related field.</li><li>Up to 2 years of proven hands-on experience in data engineering.</li><li>I must have prior experience working on Data Modeling, Data Operations, ETL pipelines, Data Quality, and Governance related projects.</li></ul>2. Technical Proficiency<ul><li>ETL &amp; Big Data: Strong proficiency in tools like Informatica or Talend. Experience with Big Data technologies such as Hadoop, Spark, and Kafka.</li><li>Languages &amp; Frameworks: Expert-level SQL and Python skills. Experience with Linux environments and PySpark for large-scale data processing.</li><li>Cloud Platforms: Hands-on experience with Databricks and major cloud providers (Azure, AWS, or GCP).</li><li>Orchestration: Proficiency in workflow management using Apache Airflow or Prefect.</li><li>Containerization &amp; CI/CD: Familiarity with Jenkins, Docker, and Kubernetes for deploying and scaling applications.</li><li>Data Visualization: Ability to integrate data with visualization tools like Tableau or Power BI, and Python libraries like Matplotlib or Seaborn.</li></ul>3. &quot;Good to Have&quot; (Bonus Skills)<ul><li>Generative AI &amp; Agentic AI: Knowledge of or interest in GenAI applications and Agentic AI workflows is a significant plus as we expand into emerging technologies.</li><li>Certifications: Any active Data Engineering certification from GCP, AWS, or Azure will be considered a strong advantage.</li></ul>Why Join Godaitec?<ul><li>Work on Emerging Tech: Get the chance to work not just on traditional data stacks but also explore AI, IoT, and Blockchain integrations.</li><li>Growth &amp; Learning: We are a learning-centric organization that supports your professional development through exposure to diverse industries (Finance, Healthcare, Retail, etc.).</li><li>Impact: Your work will directly influence the strategic decisions of global clients.</li></ul>Job Type: Full-timeEducation:<ul><li>Bachelor's (Preferred)</li></ul>Location:<ul><li>Karachi (Preferred)</li></ul>Work Location: In person

Data Engineer (Mid-Level)

Explore 1116+ Data pipeline engineer in Karachi this December. Find roles across top companies and apply now to boost your career.

1116 data pipeline engineer jobs IN Karachi

Cloudera Data Engineer

Cloudera Data Engineer

Data Engineer

Principal Data Engineer – Data Quality & Governance

Managing Consultant Data Engineering

Associate Data Architect - Data Lake/Data Streaming

Data Engineer (Mid-Level)