data analyst

data scientist

data engineer

data infrastructure engineer Jobs in Karachi

data infrastructure engineer

data scientist ii

big data engineer

Confiz

<div>We are looking for an experienced data engineer responsible for expanding and optimizing our big data pipelines. You will be working alongside other engineers and developers working on different layers of the data warehouse. Therefore, a commitment to collaborative problem solving, sophisticated design, and the creation of quality products is essential.
Responsibilities:
<ul><li>Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.</li>
<li>Build &amp; maintain ETL/ELT Ingestion pipelines and analytics solutions using cloud 0technologies.</li>
<li>Ensure the design, code and procedural aspects of solution are production ready, in terms of operational, security and compliance standards.</li>
<li>Participate in day-to-day project and production delivery status meetings, and provide technical support for faster resolution of issues.</li>
<li>Work with Stakeholders and Understand their Requirements</li>
</ul>
Qualifications
<ul><li>Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field (or equivalent work experience).</li>
<li>3+ years of overall experience in data engineering &amp; data warehousing domain focusing on building large scale data pipelines</li>
<li>2+ years' experience in Big Data Distributed systems such as Microsoft Fabric, Databricks, AWS EMR</li>
<li>Strong experience in developing data pipelines using PySpark and ETL tools like ADF, Talend</li>
<li>Proficiency in Python &amp; SQL</li>
<li>Strong problems solving and critical thinking skills</li>
<li>Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.</li>
<li>Working knowledge of CI/CD pipelines for deployments.</li>
<li>Working knowledge of using code version control repositories like Git.</li>
<li>Knowledge of security best practices including encryption of sensitive data, encryption at rest and encryption in flight.</li>
</ul>
We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Senior Azure Data Engineer

Pivotize Consulting

Position Overview:We are seeking a experienced Data Engineer to join the team. As a Data Engineer, you will play a crucial role in designing, implementing, and maintaining the infrastructure necessary for our data initiatives, ensuring scalability, reliability, and efficiency. You will collaborate with cross-functional teams to develop robust data pipelines, optimize data workflows, and ensure the reliability and scalability of our data systems.Key Responsibilities:&middot; Develop and maintain ETL (Extract, Transform, Load) processes to integrate data from disparate sources into SQL Server databases.&middot; Write complex SQL queries, stored procedures, and optimize database performance for large-scale datasets.&middot; Develop and implement efficient data models to support analytical and reporting requirements.&middot; Design, implement, and optimize database structures, stored procedures, queries, and views for performance and scalability.&middot; Design and implement database schemas, tables, indexes, Triggers and constraints to ensure efficient storage and retrieval of data.&middot; Perform database debugging, troubleshooting, and bug fixing to identify and resolve issues related to data integrity, performance, and functionality.&middot; Troubleshoot and resolve database issues, including performance bottlenecks, data corruption, and system errors.&middot; Collaborate with cross-functional teams, including software developers, analysts, and business stakeholders, to gather requirements and implement database solutions.&middot; Implement security measures and access controls to protect sensitive data stored in SQL Server databases, in compliance with regulatory requirements and industry best practices.Requirements:&middot; Degree in Computer Science, Engineering, or a related field.&middot; Strong expertise in Microsoft SQL Server (T-SQL, indexing, query tuning, performance optimization).&middot; 5+ years of experience as a Data Engineer or similar role, with a strong background in data warehousing, ETL, and database technologies.&middot; Can demonstrate experience working with large and complex data sets as well as experience analyzing volumes of data.&middot; Experience with monitoring and optimizing database performance in production environments.&middot; Familiarity with cloud-based data architecture and pipelines.&middot; Strong problem-solving, analytical, and debugging skills.&middot; Ability to analyze complex data sets and design scalable data solutions.&middot; Excellent communication and collaboration skills.Job Type: Full-timeApplication Question(s):<ul><li>what is your current salary?</li><li>what is your expected salary?</li><li>What is your notice period?</li><li>Since it's an onsite role, would you be able to commute to Bahria Complex 1, M.T. Khan Road?</li></ul>Work Location: In person

Data Engineer

Datamatics Global Services Ltd

<div><h2 class="jobSectionHeader">Cloudera Data Engineer</h2>
Location: Remote (UAE Time Zone) 
Experience: 5+ Years 
Availability: Immediate joiners or candidates who can start within 30 days preferred<h3 class="jobSectionHeader">
Role Overview</h3>
We are seeking an experienced Cloudera Data Engineer to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.<h3 class="jobSectionHeader">
Key Responsibilities</h3><ul><li>
Design, build, and maintain ETL/ELT pipelines using Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Optimize data pipelines for performance, reliability, and cost efficiency across batch and streaming workloads.</li><li>
Develop automation scripts and reusable frameworks for data ingestion, transformation, and cleansing.</li><li>
Collaborate with Cloudera Administrators to tune performance and manage cluster resources efficiently.</li><li>
Troubleshoot and resolve data pipeline failures, performance bottlenecks, and data quality issues.</li><li>
Monitor cluster utilization, job performance, and data throughput using Cloudera Manager.</li><li>
Work with DevOps/Infrastructure teams to define and maintain CI/CD pipelines, automation for deployments, and monitoring of data jobs.</li><li>
Participate in incident management, root cause analysis, and preventive maintenance.</li><li>
Ensure pipelines meet SLAs, data quality, and latency requirements for analytics workloads.</li></ul><h3 class="jobSectionHeader">
Requirements</h3><ul><li>
5+ years of experience as a Data Engineer in a Cloudera-based environment.</li><li>
Strong hands-on experience with Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Solid understanding of ETL design, performance optimization, and data governance principles.</li><li>
Experience with Linux/Unix scripting and CI/CD integration.</li><li>
Excellent troubleshooting and communication skills.</li></ul>
BJoI7cAFtW</div>

Cloudera Data Engineer

Assurety Consulting & Solutions Pvt. ltd

Job Description: Join our growing team at Assurety Consulting &amp; Solutions as a Data Engineer! We&rsquo;re looking for a detail-oriented professional to design, build, and optimize data pipelines, integrate multiple data sources, and ensure high-quality, reliable data that powers business insights.Key Responsibilities:<ul><li>Design, develop, and maintain ETL workflows using SQL Server, SSIS, and Azure Data Factory (ADF)</li><li>Integrate structured and unstructured data across diverse systems</li><li>Ensure data accuracy, integrity, and performance optimization</li><li>Partner with cross-functional teams to deliver scalable, data-driven solutions</li></ul>Requirements:<ul><li>Bachelor&rsquo;s degree in Computer Science or a related field</li><li>3+ years of experience in data engineering</li><li>Proficiency in SQL Server, SSIS, and Azure Data Factory (ADF)</li><li>Strong analytical and problem-solving skills</li></ul>Location: Shahrah-e-Faisal, KarachiJob Type: Full-timeWork Location: In person

Data Engineer (Immediate Hiring – Karachi)

Horizon Tech

We are looking for a skilled Data Engineer to design, build, and optimise our modern data platform on the cloud. You'll be responsible for architecting scalable, secure, and automated data pipelines and enabling data-driven applications and AI/ML solutions across the organisation.This is an exciting opportunity to shape the data foundation of a growing business, leveraging a modern AWS and engineering ecosystem.Must Haves:<ul><li>3-5 years of experience in data engineering or a similar role working with large-scale, distributed data systems.</li><li>Strong proficiency in Python, SQL, and data modelling for analytical and transactional systems.</li><li>Deep understanding of modern AWS data and analytics services (e.g., S3, Glue, Lambda, Redshift, or equivalents).</li><li>Experience with Apache Spark / PySpark for batch and stream processing.</li><li>Solid knowledge of data governance, security, and compliance frameworks.</li><li>Familiarity with Infrastructure as Code (Terraform, CloudFormation) and CI/CD practices.</li><li>Understanding of AI/ML data workflows, including feature engineering and data preparation for LLMs or predictive models.</li><li>Strong problem-solving, collaboration, and communication skills.</li></ul>Preferred (Nice to Have)<ul><li>Knowledge of containerization (Docker, ECS, EKS) or serverless data architectures.</li><li>Understanding of AI/ML and NLP technologies, including integration of LLMs, automation frameworks, and RPA tools for intelligent data workflows.</li><li>Awareness of MLOps pipelines, model monitoring, and feature store management.</li></ul>Responsibilities:<ul><li>Design, build, and maintain large-scale, distributed, and event-driven data pipelines using modern AWS Data Engineering services (e.g., Glue, Lambda, Step Functions, etc.).</li><li>Develop ETL/ELT workflows integrating structured and unstructured data into Snowflake, Redshift, and S3-based data lakes.</li><li>Architect and optimise data ingestion, transformation, and storage layers for structured and unstructured data in a data lake or warehouse environment.</li><li>Implement and enforce data security, governance, and access controlincluding encryption, IAM, auditing, and compliance with best practices.</li><li>Develop and maintain metadata management, data lineage, and cataloguing frameworks to ensure traceability and consistency across datasets.</li><li>Automate infrastructure provisioning using Infrastructure-as-Code (IaC) tools such as Terraform or AWS CloudFormation.</li><li>Collaborate with analytics and AI/ML teams to deliver clean, high-quality, and feature-ready data for training and inference pipelines.</li><li>Integrate observability and monitoring into data workflows to ensure reliability, performance, and cost efficiency.</li><li>Contribute to CI/CD pipelines and Git-based version control for data workflows and infrastructure changes.</li><li>Mentor junior engineers and champion engineering best practices for data quality, reliability, and scalability.</li></ul>Other Details<ul><li>Work Days: Monday-Friday (3 pm - 12 am)</li><li>Office location: Off to Shahrah-e-Faisal, PECHS, Karachi</li></ul>Job Type: Full-timePay: Rs150,000.00 - Rs250,000.00 per monthApplication Question(s):<ul><li>What is your expected salary?</li></ul>Education:<ul><li>Bachelor's (Required)</li></ul>Experience:<ul><li>data engineering with large-scale data systems: 3 years (Required)</li><li>Python, SQL and modern AWS data and analytics services: 3 years (Required)</li><li>ETL/ELT workflows : 3 years (Required)</li></ul>Location:<ul><li>Karachi (Required)</li></ul>Work Location: In person

<div>We are seeking a Data Engineer with 3+ years of hands-on experience and a strong background in real-time and batch data processing, containerization, and cloud-based data orchestration. This role is ideal for someone passionate about building robust, scalable, and efficient data pipelines, and who thrives in agile, collaborative environments.
Responsibilities
<ul><li>Design, build, and maintain real-time data pipelines using streaming frameworks such as Kafka, Apache Flink, and Spark Structured Streaming.</li>
<li>Develop batch processing workflows with Apache Spark (PySpark)</li>
<li>Orchestrate and schedule data workflows using orchestration frameworks such as Apache Airflow and Azure Data Factory</li>
<li>Containerize applications using Docker, manage deployments with Helm, and run them on Kubernetes</li>
<li>Implement modern storage solutions using open formats such as Parquet, Delta Lake, and Apache Iceberg</li>
<li>Build high-performance analytics engines using tools like Trino or Presto</li>
<li>Collaborate with DevOps to manage infrastructure with Terraform and integrate with CI/CD pipelines via Azure DevOps</li>
<li>Ensure data quality and consistency using tools like Great Expectations</li>
<li>Write modular, well-tested, and maintainable Python and SQL code</li>
<li>Develop an observability layer to monitor and optimize performance across data pipelines</li>
<li>Participate in agile ceremonies and contribute to sprint planning and reviews</li>
</ul>
Required Skills
<ul><li>Advanced Python programming with a strong focus on modular and testable code</li>
<li>Strong knowledge of SQL and experience working with large-scale datasets</li>
<li>Hands-on experience with at least one major cloud platform (Azure preferred)</li>
<li>Solid experience with real-time data processing (Kafka, Flink, or Spark Streaming)</li>
<li>Expertise in Apache Spark (PySpark) for batch processing</li>
<li>Experience implementing lakehouse architectures and working with columnar storage (e.g., ClickHouse)</li>
<li>Proficient in using Azure Data Factory or Apache Airflow for data orchestration</li>
<li>Experience in building APIs to expose large datasets</li>
<li>Solid experience with Docker, Kubernetes, and Helm</li>
<li>Familiarity with data lake open formats such as Parquet, Delta Lake, and Iceberg</li>
<li>Basic experience with Terraform for infrastructure provisioning</li>
<li>Practical experience with data quality frameworks (e.g., Great Expectations)</li>
<li>Comfortable working in agile development teams</li>
<li>Proven ability in debugging and performance tuning of streaming and batch data jobs</li>
<li>Experience with AI-driven tools (e.g., text-to-SQL) is a plus</li>
<li>Bachelor&rsquo;s degree in computer science or related discipline</li>
</ul>
We have an amazing team of 700+ individuals working on highly innovative enterprise projects &amp; products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast-growth fintech, and multiple Silicon Valley startups.
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Senior/Principal Data Engineer

Enterprise64

<div>Overview 
Enterprise64 is a U.S.-based technology company that creates digital solutions for startups to Fortune 500 companies in the United States, Europe, and the United Arab Emirates. We are growing rapidly across all our global offices and have career opportunities for tech professionals who are looking to work in a fast-growing company, work in a fun environment, be challenged, grow, and fast-track their careers.
Job Overview 
We are hiring a Senior Python Backend Developer to join our dynamic team (Contract Role). The ideal candidate will have strong expertise in backend development, API design, and database optimization, ensuring high-performance, scalable, and secure applications. You will collaborate with cross-functional teams to build efficient, maintainable, and reliable backend solutions that drive our innovative products.
Key Responsibilities<ul><li>
Design, develop, and maintain ETL pipelines for large-scale data processing and integration.</li><li>
Work with AWS Glue, PySpark, and Apache Spark to process structured and unstructured data efficiently.</li><li>
Build and optimize data ingestion workflows from multiple sources into data lakes and warehouses.</li><li>
Ensure data quality, consistency, and reliability through validation and monitoring.</li><li>
Collaborate with cross-functional teams, including data scientists, analysts, and backend engineers, to deliver scalable data solutions.</li><li>
Implement and manage data transformation and schema evolution using AWS services (Glue, S3, Lambda, Redshift, etc.).</li><li>
Optimize Spark jobs for performance and cost efficiency.</li><li>
Develop and maintain metadata management and data cataloging in AWS Glue.</li><li>
Ensure adherence to data security, compliance, and governance best practices.</li><li>
Use Python for automation, data manipulation, and integration tasks.</li><li>
Implement CI/CD pipelines for data workflows and maintain infrastructure-as-code (IaC) using tools like Terraform or CloudFormation (a plus).</li></ul> 
Preferred Qualifications<ul><li>
Strong expertise in Python and PySpark for large-scale data processing.</li><li>
Hands-on experience with AWS Glue, AWS Lambda, S3, and Redshift.</li><li>
Solid understanding of ETL frameworks, data modeling, and data warehousing concepts.</li><li>
Experience with Apache Spark and distributed computing principles.</li><li>
Familiarity with workflow orchestration tools (e.g., Airflow, Step Functions).</li><li>
Knowledge of SQL and NoSQL databases for data querying and optimization.</li><li>
Strong problem-solving and debugging skills for data-intensive applications.</li><li>
Experience working in agile environments and collaborating with cross-functional teams.</li></ul></div>

Senior Python Developer - Contractor

Explore 1023+ Big data engineer in Karachi this November. Find roles across top companies and apply now to boost your career.

1023 big data engineer jobs IN Karachi

Senior Azure Data Engineer

Data Engineer

Cloudera Data Engineer

Data Engineer (Immediate Hiring – Karachi)

Data Engineer

Senior/Principal Data Engineer

Senior Python Developer - Contractor