data pipeline engineer

Devsinc

<div>
 We are seeking a highly skilled
 
 Senior Data Engineer
 
 to design, build, and optimize scalable data pipelines, storage layers, and analytics infrastructure. The ideal candidate has strong data engineering foundations, deep experience with distributed data systems, and the ability to turn complex business requirements into high-performance, production-grade data solutions. The role also requires leadership in driving complex data projects end-to-end.
 
 
 
 Key Responsibilities:
 
 
 
 
 Leadership &amp; Project Management
 
 
 
 <ul>
 <li>
 Lead end-to-end technical delivery of data engineering projects.
 </li>
 <li>
 Define technical scope, timelines, and engineering milestones.
 </li>
 <li>
 Manage and coordinate multiple stakeholders (data scientists, ML engineers, backend teams, and product teams).
 </li>
 <li>
 Conduct architectural reviews, ensure engineering best practices, and drive technical decision-making.
 </li>
 <li>
 Mentor junior and mid-level engineers, fostering a culture of learning and excellence.
 </li>
 <li>
 Oversee code quality, documentation, and adherence to team standards.
 </li>
 <li>
 Proactively identify risks, dependencies, and areas for improvement across projects.
 
 
 </li>
 </ul>
 
 Data Engineering &amp; Systems Development
 
 
 
 <ul>
 <li>
 Design and implement scalable ETL/ELT pipelines for both batch and streaming workloads, and build/maintain data warehouses, lakes, and lakehouse architectures.
 </li>
 <li>
 Develop optimized data models (relational, dimensional, NoSQL) while ensuring data quality, lineage, governance, and metadata management.
 </li>
 <li>
 Write clean, modular, and well-tested production code in Python, Java, or Scala; build APIs, microservices, and integrations for data workflows; and implement CI/CD automation and testing.
 </li>
 <li>
 Build secure, scalable cloud-based data infrastructure on AWS/GCP/Azure using containerized, serverless, and distributed technologies, and manage IaC using Terraform, CloudFormation, etc.
 </li>
 <li>
 Work with modern data and streaming frameworks (Spark, Flink, Beam, Kafka, Kinesis, Pub/Sub) and cloud data platforms (Snowflake, BigQuery, Redshift, Databricks) to deliver real-time and event-driven pipelines.
 </li>
 <li>
 Collaborate with cross-functional teams (Data Scientists, ML Engineers, Software Engineers) and lead design discussions, code reviews, and mentoring to maintain engineering best practices
 
 
 </li>
 </ul>
 
 Requirements
 
 
 
 <ul>
 <li>
 3+ years of experience in Data Engineering roles.
 </li>
 <li>
 Strong proficiency in Python, SQL, and at least one compiled language (Java/Scala/Go).
 </li>
 <li>
 Hands-on experience with data pipeline orchestration (Airflow, Dagster, Prefect, etc.).
 </li>
 <li>
 Deep understanding of distributed systems, data partitioning, fault tolerance, and performance tuning.
 </li>
 <li>
 Experience with containerized environments (Docker, Kubernetes).
 </li>
 <li>
 Strong problem-solving skills, debugging abilities, and systems-level thinking.
 
 
 </li>
 </ul>
 
 Preferred Qualifications:
 
 
 
 <ul>
 <li>
 Experience supporting ML Ops workflows and feature/data pipelines for machine learning models and AI applications.
 </li>
 <li>
 Knowledge of Delta Lake and Data Warehouse.
 </li>
 <li>
 Familiarity with security, compliance, access controls, and enterprise data governance.
 </li>
 <li>
 Exposure to data observability tools (Monte Carlo, Databand, Soda, etc.).
 </li>
 <li>
 Background in AI, analytics, or large-scale product engineering is a plu
 </li>
 </ul>
</div>

Senior Data Engineer.

Algo

<div>Job Title: Data Engineer 
Location: DHA Phase-8, Lahore 
 
Job Responsibilities: 
1. Design, build, and maintain efficient and scalable ETL/ELT data pipelines using tools like Azure Data Factory or Apache Airflow 
2. Develop, test, and deploy Python-based data processing solutions 
3. Work with Snowflake, Databricks and other data warehousing technologies to support data analytics and reporting needs 
4. Write complex and optimized SQL queries for data transformation and extraction 
5. Ensure data quality, accuracy, and compliance with internal and external standards 
6. Collaborate with data engineers, analysts, and stakeholders to define and implement data workflows 
7. Utilize version control (Git) and CI/CD pipelines for code deployment and process 
automation 
8. (Optional/Preferred) Support backend services using Django and task queues with 
Celery 

Qualifications Needed:
<ul>
<li>Python 
</li>
<li>SQL 
</li>
<li>Snowflake 
</li>
<li>Databricks 
</li>
<li>Azure (or AWS) 
</li>
<li>ETL/ELT 
</li>
<li>ADF / Apache Airflow 
</li>
<li>Data Warehousing 
</li>
<li>Git 
</li>
<li>CI/CD 
</li>
<li>Django (preferred) 
</li>
<li>Celery (preferred)</li></ul></div>

Data Engineer (4-5 Years)

Taraki

<div>
 
 About The Job
 
 
 
 <ul>
 <li>
 Build and maintain data pipelines using Python and Apache Spark.
 </li>
 <li>
 Orchestrate workflows using Airflow and Google Cloud Workflows (CloudFlow).
 </li>
 <li>
 Develop, deploy, and manage containerized data services using Docker and Cloud Run.
 </li>
 <li>
 Design, optimize, and monitor datasets and queries in BigQuery.
 </li>
 <li>
 Ingest, transform, and integrate external data through REST APIs.
 </li>
 <li>
 Manage data lifecycle and storage using Google Cloud Storage (GCS).
 </li>
 <li>
 Implement data quality, monitoring, and observability best practices.
 </li>
 <li>
 Collaborate with cross-functional engineering, product, and data science teams.
 
 
 
 </li>
 </ul>
 
 Requirements
 
 
 
 <ul>
 <li>
 2&ndash;4+ years of experience as a Data Engineer or similar role.
 </li>
 <li>
 Strong proficiency in Python, SQL, and Spark/PySpark.
 </li>
 <li>
 Hands-on experience with Airflow and cloud-native orchestration (e.g., Cloud Workflows).
 </li>
 <li>
 Experience with Docker, containers, and deploying services in Cloud Run.
 </li>
 <li>
 Skilled with BigQuery, GCS, and general GCP data tooling.
 </li>
 <li>
 Experience working with REST APIs and building ingestion integrations.
 </li>
 <li>
 Solid understanding of data modeling, ETL/ELT pipelines, and distributed systems.
 
 
 
 </li>
 </ul>
 
 Bonus:
 
 Experience with healthcare data standards (FHIR, HL7) or regulated environments.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 Competitive salary and performance-based incentives
 </li>
 <li>
 Flexible work arrangements
 </li>
 <li>
 Paid time off and public holidays
 </li>
 <li>
 Professional development opportunities (training, workshops, certifications)
 </li>
 <li>
 Supportive, collaborative, and mission-driven work culture
 
 </li>
 </ul>
</div>

Data Engineer - Marham

Soho Dragon

<div>SoHo Dragon represents a tech-driven, AI-native pharmaceutical company that&rsquo;s shaping the future of healthcare. We&rsquo;re looking for a mid-to-senior-level Data Engineer to play a key role in building and optimizing data solutions. 
You&rsquo;ll work with Snowflake and Dagster as your core tools, while also leveraging Terraform, GitHub Actions, and Databricks to deliver scalable, high-impact data pipelines. 
Start date: ASAP

Contract duration: 2-6 months</div>

Data Engineer

<div>Position: Data Engineer (Azure Platform)

Location: Lahore, Pakistan (on-site)<div> 
Job Description 
<ul><li>Design and develop scalable data warehouse solutions to support analytics and reporting.</li><li>Build and maintain efficient Azure Data Pipelines using Azure Data Factory for ETL processes.</li><li>Develop and optimize Spark jobs using Databricks and PySpark for large-scale data transformations.</li><li>Coordinate with onsite and offshore teams to ensure seamless project execution and business continuity.</li><li>Develop and maintain dashboards and reports using BI tools such as Power BI, Tableau, MicroStrategy, or Qlik.</li><li>Manage the presentation and access layers to ensure secure and efficient data access.</li><li>Work directly with clients and stakeholders to understand requirements and deliver actionable data solutions</li></ul></div> 
Qualifications
<ul><li>BS in Computer Science or a related field with 4-5 years of experience in BI and data management projects.</li><li>Strong expertise in database and data warehouse design, support, and maintenance.</li><li>Proficiency in writing complex SQL and SparkSQL queries.</li><li>Experience in Python and its data-related frameworks.</li><li>Hands-on experience with Databricks and PySpark for data engineering tasks.</li><li>Strong proficiency in Power BI; experience with other BI tools is a plus.</li><li>Experience with Azure Data Platform (Data Factory, Azure SQL, Synapse, Data Lake) is preferred.</li><li>Ability to generate and maintain comprehensive technical documentation.</li><li>Excellent communication and interpersonal skills.</li><li>Strong analytical and problem-solving abilities.</li><li>Willingness to learn and adapt to new technologies and project requirements.</li><li>Experience in data analysis is a plus.</li><li>Ability to work independently and handle critical situations confidently.</li><li>A collaborative, team-oriented mindset.</li></ul></div>

Data Engineer (Azure Platform)

Cloudelligent

<div>Position Title: Sr. AWS Data Engineer 
Job Timings: 8:00 AM &ndash; 5:00 PM CST (6:00 PM- 3:00 AM PKT) 
Location: Lahore 
 
About Cloudelligent 
 
Cloudelligent is Cloud-native consultancy and AWS Advanced consulting partner! We specialize in providing bespoke cloud solutions to Startups &amp; SMBs. Being a next-gen cloud service provider, Cloudelligent helps businesses to make the most out of their cloud investment. We have an international footprint with a diverse team of domain experts, and we are customer obsessed. 
 
Job Objective 
 
We are seeking an experienced Sr. Data Engineer with a strong background in data engineering, database optimization, and cloud-based data solutions, particularly on AWS. The ideal candidate will have hands-on experience with both SQL and NoSQL databases and be proficient in migrating, managing, and modernizing data platforms. You should be skilled in designing and maintaining scalable, secure, and high-performance data infrastructure in the cloud. 
 
Responsibilities<ul><li>
Design, implement, and manage cloud-based data infrastructure using Amazon RDS, Aurora, and EC2-hosted databases.</li><li>
Administer and optimize PostgreSQL, MySQL, and Oracle databases, ensuring high availability, performance, and cost-efficiency.</li><li>
Perform data migrations and transformations using AWS Database Migration Service (DMS) and other tools.</li><li>
Collaborate with data architects, engineers, and DevOps teams to deliver reliable and scalable data solutions.</li><li>
Support developers by analyzing and optimizing SQL queries and data workflows for performance and efficiency.</li><li>
Monitor and troubleshoot data issues, ensuring availability, security, and compliance.</li><li>
Implement data automation, infrastructure as code (IaC), and CI/CD pipelines for database and data pipeline deployments.</li><li>
Ensure data security, encryption, and compliance with industry best practices and internal policies.</li><li>
Maintain comprehensive documentation on data architecture, processes, and procedures.</li><li>
Stay up to date with emerging AWS features, data engineering tools, and industry best practices.</li></ul> 

Requirements<ul><li>
Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, Information Technology, or a related field.</li><li>
5+ years of experience in data engineering or database engineering, including hands-on experience with databases on AWS.</li><li>
Strong experience with AWS data services such as RDS, Aurora, and DMS.</li><li>
Proficient in performance tuning, indexing strategies, partitioning, and query optimization.</li><li>
Strong scripting and automation experience using Python, Bash, or similar tools.</li><li>
Experience with infrastructure as code tools like Terraform or CloudFormation.</li><li>
Solid understanding of backup strategies, monitoring tools, and security best practices.</li><li>
Familiarity with multi-account AWS environments and cross-region replication.</li><li>
Excellent communication and problem-solving skills.</li></ul> 

Highly Desirable<ul><li>
AWS Certification (e.g., AWS Certified Data Engineer &ndash; Associate, AWS Certified Solutions Architect &ndash; Associate).</li><li>
Experience with large-scale on-prem to AWS data migrations.</li><li>
Familiarity with data governance, compliance, and auditing tools.</li><li>
Exposure to data lake architecture and AWS services like Glue, Athena, or Kinesis is a plus.</li></ul>
tkOFySDYmB</div>

Sr. AWS Data Engineer

Sharp Brains Ltd

Role Overview: AI/ML Engineer (Animation Technology)As an AI/ML Engineer within our Animation Department your mission is to bridge the gap between complex machine learning research and the creative production pipeline.Key Responsibilities<ul><li>Generative AI Development: Design and deploy GANs, Diffusion models, and Transformers specifically tailored for character animation, background synthesis, and texture generation.</li><li>Motion Synthesis &amp; Rigging: Develop ML-driven solutions for automated rigging, muscle simulation, and motion retargeting to create more fluid, lifelike character movements.</li><li>Pipeline Integration: Collaborate with Technical Directors (TDs) to integrate AI tools directly into industry-standard software like Maya, Houdini, or Unreal Engine.</li><li>Neural Rendering: Explore and implement Neural Radiance Fields (NeRFs) and other AI-driven rendering techniques to optimize lighting and look-development workflows.</li><li>Dataset Curation: Build and manage high-quality datasets derived from our internal library of motion capture and hand-keyed animation to train proprietary models.</li></ul>Core Requirements<ul><li>Technical Proficiency: Strong mastery of Python, PyTorch, or TensorFlow. Experience with C++ is a significant plus for high-performance tool integration.</li><li>Domain Knowledge: Solid understanding of 3D mathematics (linear algebra, quaternions) and the standard animation pipeline (Modeling $\rightarrow$ Rigging $\rightarrow$ Animation $\rightarrow$ Rendering).</li><li>ML Expertise: Proven experience in Computer Vision (CV), Deep Learning, and specifically Generative Modeling.</li></ul>Preferred Qualifications<ul><li>Experience with OpenUSD (Universal Scene Description).</li><li>Prior work with Real-time engines (Unreal Engine 5 / Unity).</li><li>A portfolio or GitHub showing projects related to style transfer, automated lip-syncing, or pose estimation.</li></ul>Job Type: Full-timePay: Rs75,000.00 - Rs85,000.00 per monthWork Location: In person

AI/ML Engineer (Animation Technology)

Explore 910+ Data pipeline engineer in Lahore Pakistan this January. Find roles across top companies and apply now to boost your career.