big data engineer

Liquid Technologies

Liquid Technologies (LiqTeq) is an end-to-end Design and Development Agency. LiqTeq offers various services in DevOps, Data services, and AI to multiple businesses and organizations and those that are in the startup phase. We're a team of creators, thinkers, explorers, and judo trainees. We approach work and play with curiosity and experimentation, using what we learn to create meaningful digital products that connect with people, just like you.Job Description <ul><li>Assemble large, complex data sets and build data pipelines that meet functional, non-functional business and data monetization requirements.</li><li>Work with the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources and cloud-based data storage and analytics services such as AWS, Snowflake etc</li><li>Build analytics tools that utilize the data pipeline to provide actionable insights</li><li>Architect and implement large scale data intelligence solutions around various warehouse solutions by various technologies</li><li>Understand data transformation and translation requirements and which tools to leverage to get the job done</li><li>Developing ETL pipelines in and out of data warehouse using a combination of Python and Snowflakes Snow SQL</li><li>Provide production support for Data Warehouse issues such as data load problems, transformation, and translation problems</li><li>Translate requirements for BI and Reporting to database design</li><li>Understanding data pipelines and modern ways of automating data pipelines using cloud</li><li>Ability to build conceptual, physical and logical data models</li><li>Utilize database and web application technologies to design, develop, and evaluate innovative business intelligence tools and automate reports</li><li>Merge BI platforms with enterprise systems and applications and document models, solutions, and implementations.</li><li>Work with various internal and stakeholders including the product, data and design teams to assist with data-related technical issues.</li></ul>Qualification - Experience Requirements:<ul><li>Minimum 2 years of experience in the data engineering role is a must.</li><li>Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.</li><li>Experience building and optimizing &lsquo;big data&rsquo; data pipelines, architectures and data sets.</li><li>Data engineering experience with database-centric and pipeline-centric roles ○ Strong analytic skills related to working with unstructured datasets.</li><li>Working knowledge of message queuing, stream processing, and highly scalable &lsquo;big data&rsquo; data stores.</li><li>A solid experience and understanding of architecting, designing and operationalization of large scale data and analytics solutions on Snowflake Cloud Data Warehouse is a must.</li><li>Solid proficiency in data warehousing concepts (ETL, Stage/Core layer, Dimensional Modeling, Data Marts etc.) and SQL programming</li><li>Ability to capture customer requirements and translate into functional and technical solutions</li><li>Past experience in using Enterprise BI tools like PowerBI, Tableau, Oracle BI, Oracle Analytics Cloud, AWS, Azure And GCP etc.</li><li>Ability to develop technical design documentation, training and presentation material</li><li>Understanding and experience of working in an agile project environment</li><li>Highly proficient in DDL, DML, DCL, TCL, and DQL</li><li>Experience with big data tools: Apache Hadoop, Spark, Kafka, Storm, Airflow is a plus.</li><li>Experience with relational SQL and NoSQL databases.</li><li>Experience with AWS cloud services: EC2, EMR and RDS is a plus</li></ul>Job Type: Full-timeApplication Question(s):<ul><li>How many years of experience do you have working with relational databases and SQL?</li><li>Have you worked with big data tools like Spark, Hadoop, or Kafka?</li><li>Have you built ETL pipelines using Python and Snowflake in a production environment?</li><li>What is your Current Salary?</li><li>What is your Expected Salary?</li><li>Are you comfortable to manage your commute to Zamzama DHA Phase V?</li></ul>Work Location: In person

Data Engineer

Careem

<div><div>Careem is building the Everything App for the greater Middle East &mdash; making it easy to move around, order food and groceries, manage payments, and more. Our purpose is simple: to simplify and improve people&rsquo;s lives and build an awesome organisation that inspires. 
Since 2012, Careem has enabled earnings for over 2.5 million Captains, simplified the lives of more than 70 million customers, and built a platform where the region&rsquo;s best talent and entrepreneurs thrive. We operate in 70+ cities across 10 countries, from Morocco to Pakistan.

We&rsquo;re now entering our next chapter &mdash; one powered by AI. We&rsquo;re looking for AI talent: curious problem-solvers who know how to apply AI to build tools, automate workflows, and create real impact. Whether it&rsquo;s streamlining operations, enhancing customer experience, or reimagining internal systems &mdash; we want people who can make Careem work smarter and move faster.</div><div><div>
About the role: 
We are seeking a highly skilled and motivated Data Warehouse Engineer to join our team. As a Data Warehouse Engineer, you will play a critical role in designing, developing, and maintaining our data infrastructure, ensuring the availability, scalability, and reliability of our data systems. You will collaborate closely with cross-functional teams, including data scientists, analysts, and software engineers, to support data-driven decision-making and enable the efficient processing of large-scale data.</div>
<div></div>
<div>What you'll do:</div>
<ul><li>Design and develop robust, scalable, and high-performance data pipelines and ETL processes to extract, transform, and load data from various sources into our data warehouse or data lake.</li>
<li>Collaborate with stakeholders to understand their data requirements, and design and implement appropriate data models and database schemas to support their needs.</li>
<li>Optimize data pipelines and ETL processes for performance and efficiency, ensuring timely and accurate data delivery to end-users.</li>
<li>Monitor, troubleshoot, and resolve issues related to data quality, data consistency, and data integrity, ensuring the reliability and correctness of our data systems.</li>
<li>Implement and maintain data governance practices and policies, ensuring compliance with data privacy and security regulations.</li>
<li>Collaborate with data scientists and analysts to provide them with the necessary data infrastructure and tools for conducting advanced analytics and deriving insights.</li>
<li>Stay up-to-date with the latest trends and technologies in data engineering and recommend innovative solutions to improve data engineering processes and systems.</li>
<li>Document data engineering processes, data flows, and system architectures to ensure knowledge sharing and maintain an up-to-date repository of technical documentation.</li>
<li>Work closely with cross-functional teams, including software engineers and infrastructure teams, to optimize data infrastructure and ensure its seamless integration with other systems.</li>
</ul><div>What you'll need</div>
<ul><li>Bachelor's degree in Computer Science, Engineering, or a related field.</li>
<li>At least 6 years of experience as a Data Warehouse] Engineer or in a similar role, working with large-scale data processing and ETL pipelines.</li>
<li>Strong programming skills in languages such as Python, Java, or Scala, with experience in data manipulation and processing frameworks like Apache Spark.</li>
<li>Experience with SQL and database technologies (e.g., relational databases, SQL queries, data modeling).</li>
<li>Proficiency in working with big data technologies such as Hadoop, Hive and knowledge of distributed systems and cloud computing platforms (e.g., AWS, Azure, GCP).</li>
<li>Familiarity with data integration and workflow management tools such as Apache Airflow</li>
<li>Knowledge of data warehousing concepts and experience with data warehousing solutions is highly desirable.</li>
<li>Strong analytical and problem-solving skills, with the ability to analyze complex data-related issues and propose effective solutions.</li>
<li>Excellent communication and collaboration skills, with the ability to work effectively in a cross-functional team environment.</li>
<li>Attention to detail and a strong commitment to delivering high-quality work within established timelines.</li>
<li>Join our dynamic team and contribute to building scalable data infrastructure that powers data analytics and decision-making processes. Apply your expertise in data engineering to help us unlock the full potential of our data assets and drive meaningful insights for our organization.</li></ul></div></div>

Senior Data Warehouse Engineer I

Confiz

<div>We are seeking an experienced Scrum Master to facilitate Agile practices within our Big Data engineering teams. The ideal candidate will have a strong understanding of Agile principles, experience working with large-scale data platforms, and the ability to foster collaboration across cross-functional teams. 
 
Key Responsibilities
<ul><li>Guide the team in Scrum practices, ensuring adherence to Agile principles and continuous improvement.</li>
<li>Organize and facilitate daily stand-ups, sprint planning, backlog refinement, reviews, and retrospectives.</li>
<li>Act as a liaison between Product Owners, Data Engineers, Data Scientists, and other stakeholders.</li>
<li>Identify and resolve blockers to maintain team productivity.</li>
<li>Track and report key Agile metrics (velocity, burndown charts) to stakeholders.</li>
<li>Mentor team members and stakeholders on Agile best practices.</li>
<li>Understand data pipelines, ETL processes, and distributed systems to effectively support technical discussions.</li>
</ul>
Required Skills &amp; Qualifications
<ul><li>Certified Scrum Master (CSM) or equivalent.</li>
<li>Overall 8+ years of experience:</li>
<li>5+ years as Scrum Master in Agile environments.</li>
<li>Must have 3+ years of experience of managing Big Data engineering teams.</li>
<li>Familiarity with data engineering concepts, technologies (Data Lake, Apache Spark, Apache Kafka), cloud platforms (AWS, Azure, GCP), and CI/CD pipelines.</li>
<li>Excellent communication, facilitation, and conflict resolution skills.</li>
<li>Experience with Scaled Agile Framework (SAFe).</li>
<li>Knowledge of Data Quality, and Analytics.</li>
 <li>Experience working in Telecommunications sector.</li>
</ul>
What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), ISO 14001:2015 (EMS), ISO 45001:2018 (OHSMS) Certified. We have a vibrant culture of learning via collaboration and making workplace fun.
People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.
To know more about Confiz Limited, visit: https://www.linkedin.com/company/confiz-pakistan/</div>

Scrum Master - Big Data

Squarera

Are you a passionate pioneer in technology?If so, consider an opportunity with SQUARERA with a full career path within the firm. To succeed in this data engineering position, you should have strong analytical skills and the ability to combine data from different sources. Data engineer skills include familiarity with several programming languages.As a future Trainee Data Engineer, you should have the ability to share new ideas and collaborate within team. If you are detail-oriented, with excellent organizational skills and have some knowledge in this field, we&rsquo;d like to hear from you.Fresh graduates or done with their final semester candidates are encouraged to apply!Job Responsibilities:As a data engineer trainee, you will/may:<ul><li>Collect and process log events from multiple data sources to support the design, implementation, automation and deployment of our distributed system.</li><li>Design data schema and optimization of SQL/NoSQL database systems.</li><li>Work collaboratively in designing, development, and maintenance of ongoing metrics, reports, analyzes, and dashboards to drive key business decisions.</li><li>Monitor and troubleshoot operational or data issues in the data pipelines.</li><li>Develop visual automated data pipelines able to process millions of data points.</li><li>You will be responsible for the technical development of any new data integration solution to be plugged into the overall data architecture.</li><li>Undergraduates &amp; candidates outside of Karachi please do not apply.</li></ul>BASIC QUALIFICATIONS<ul><li>Currently enrolled in or will receive a Bachelor&rsquo;s in Computer Science, Computer Engineering, Information Management, Information Systems, or an equivalent technical discipline.</li><li>Strong knowledge of SQL.</li></ul>Key Competencies:<ul><li>Attention to detail</li><li>Data collection, management, and analysis</li><li>Problem analysis and problem-solving</li><li>Planning and organizing</li><li>Decision-making</li><li>Teamwork</li><li>Ability to think and work independently</li></ul>Perks &amp; Benefits (After selection as a full-time employee)<ul><li>Travel Abroad Opportunities</li></ul>Note: The selected trainees would have:<ul><li>To sign an undertaking for 27 months</li><li>Medical Insurance</li><li>Annual Bonus</li><li>Yearly Increment</li></ul>Job Type: Full-timeWork Location: In person

Trainee Data Engineer

Systems Limited

<div>We are looking for an experienced ETL PySpark Data Engineer to design, build, and optimize scalable data pipelines.
<h4 class="jobSectionHeader">
Responsibilities</h4>
<ul>
<li>Design, develop, and maintain ETL pipelines using PySpark
</li>
<li>Process large-scale structured and semi-structured data efficiently
</li>
<li>Build batch and (optional) streaming data pipelines
</li>
<li>Perform data ingestion, transformation, and loading from multiple sources
</li>
<li>Optimize Spark jobs for performance and scalability
</li>
<li>Work with cloud data platforms (Azure / AWS / GCP)
</li>
<li>Ensure data quality, reliability, and governance
</li>
<li>Collaborate with data analysts, architects, and stakeholders
</li>
</ul><h4 class="jobSectionHeader">Requirement:</h4>
<ul>
<li>Strong hands-on experience with PySpark
</li>
<li>Solid understanding of ETL concepts &amp; data warehousing
</li>
<li>Strong SQL skills
</li>
<li>Experience with Spark SQL, DataFrames, and RDDs
</li>
<li>Familiarity with HDFS, Hive, Delta Lake
</li>
<li>Experience with cloud services (Azure Data Factory, Databricks, AWS Glue, etc.)
</li>
<li>Knowledge of Linux &amp; Git</li></ul></div>

Senior Consultant Data Analytics

Arpatech (Pvt.) Ltd

Key Responsibilities:Data Pipeline Engineering<ul><li>Design and build batch and near-real-time pipelines using:</li><li>Azure Fabric Data Factory</li><li>Snowflake ingestion patterns (COPY, Snowpipe, Streams, Tasks)</li><li>Implement lakehouse-style data layers:</li><li>Raw &rarr; Curated &rarr; Consumption</li><li>Manage data structures using:</li><li>Azure Fabric OneLake</li><li>Snowflake databases, schemas, and secure objects</li></ul>Data Consolidation &amp; Modeling<ul><li>Consolidate data from multiple enterprise systems into governed datasets</li><li>Apply enterprise data modeling patterns:</li><li>Conformed dimensions</li><li>Canonical fact tables</li><li>Shared reference data</li><li>Manage schema design, evolution, and versioning for long-term stability</li></ul>Data Quality &amp; Trust<ul><li>Implement data validation for completeness, consistency, and freshness</li><li>Detect and surface data quality issues early</li><li>Partner with business teams to define clear data contracts</li></ul>Platform Optimization &amp; Operations<ul><li>Optimize performance and cost across Snowflake and Fabric through:</li><li>Partitioning, clustering, and pruning</li><li>Warehouse sizing and query tuning</li><li>Support monitoring, alerting, and incident response for data pipelines</li></ul>Required Skills &amp; Experience:<ul><li>4+ years of hands-on data engineering experience</li><li>Strong SQL skills with Snowflake</li><li>Experience building pipelines in Microsoft Azure Fabric</li><li>Solid understanding of data modeling (dimensional, canonical, wide tables)</li><li>Experience integrating data from SaaS, ERP, operational, and internal systems</li><li>Familiarity with CI/CD and source-controlled data pipelines</li><li>Ability to work independently in remote, client-embedded environments</li></ul>Nice to Have:<ul><li>Experience with Microsoft Purview, Fabric governance, or Snowflake Data Sharing</li><li>Exposure to AI/ML-ready or feature datasets</li><li>Knowledge of data quality frameworks or automated testing tools</li><li>Experience in regulated or finance-heavy environments</li></ul>Location: Onsite for KHI &amp; ISB | Remote for other cities.Job Type: Full-timeWork Location: In person

Enterprise64

<div>Overview 
Enterprise64 is a U.S.-based technology company that creates digital solutions for startups to Fortune 500 companies in the United States, Europe, and the United Arab Emirates. We are growing rapidly across all our global offices and have career opportunities for tech professionals who are looking to work in a fast-growing company, work in a fun environment, be challenged, grow, and fast-track their careers.
Job Overview 
We are hiring a Senior Python Backend Developer to join our dynamic team (Contract Role). The ideal candidate will have strong expertise in backend development, API design, and database optimization, ensuring high-performance, scalable, and secure applications. You will collaborate with cross-functional teams to build efficient, maintainable, and reliable backend solutions that drive our innovative products.
Key Responsibilities<ul><li>
Design, develop, and maintain ETL pipelines for large-scale data processing and integration.</li><li>
Work with AWS Glue, PySpark, and Apache Spark to process structured and unstructured data efficiently.</li><li>
Build and optimize data ingestion workflows from multiple sources into data lakes and warehouses.</li><li>
Ensure data quality, consistency, and reliability through validation and monitoring.</li><li>
Collaborate with cross-functional teams, including data scientists, analysts, and backend engineers, to deliver scalable data solutions.</li><li>
Implement and manage data transformation and schema evolution using AWS services (Glue, S3, Lambda, Redshift, etc.).</li><li>
Optimize Spark jobs for performance and cost efficiency.</li><li>
Develop and maintain metadata management and data cataloging in AWS Glue.</li><li>
Ensure adherence to data security, compliance, and governance best practices.</li><li>
Use Python for automation, data manipulation, and integration tasks.</li><li>
Implement CI/CD pipelines for data workflows and maintain infrastructure-as-code (IaC) using tools like Terraform or CloudFormation (a plus).</li></ul> 
Preferred Qualifications<ul><li>
Strong expertise in Python and PySpark for large-scale data processing.</li><li>
Hands-on experience with AWS Glue, AWS Lambda, S3, and Redshift.</li><li>
Solid understanding of ETL frameworks, data modeling, and data warehousing concepts.</li><li>
Experience with Apache Spark and distributed computing principles.</li><li>
Familiarity with workflow orchestration tools (e.g., Airflow, Step Functions).</li><li>
Knowledge of SQL and NoSQL databases for data querying and optimization.</li><li>
Strong problem-solving and debugging skills for data-intensive applications.</li><li>
Experience working in agile environments and collaborating with cross-functional teams.</li></ul></div>

Senior Python Developer - Contractor

Explore 734+ Big data engineer in Karachi Pakistan this January. Find roles across top companies and apply now to boost your career.

734 big data engineer jobs IN Karachi Pakistan

Data Engineer

Senior Data Warehouse Engineer I

Scrum Master - Big Data

Trainee Data Engineer

Senior Consultant Data Analytics

Data Engineer

Senior Python Developer - Contractor