data pipeline engineer

Bilight Solutions

<p><b>Job Title: Data Engineer (2-3 Years Experience)</b></p><p><b>Location: Chennai</b></p><p><b>Work Mode: Full-time, Work From Office (9:30 AM &ndash; 6:30 PM)</b></p><p><b>Availability: Immediate Joiners</b></p><p><b>Contact: 9384854315 (Reshma C - HR Executive)</b></p><p><b>About the Role:</b></p><p>We are looking for a highly skilled Data Engineer to join our delivery team. The role involves building robust, scalable, and secure data pipelines that power analytics and reporting. The ideal candidate will have strong experience in cloud platforms, ETL frameworks, and big data tools, and will work closely with analysts, business teams, and clients to ensure the timely delivery of clean, high-quality data for strategic insights and decision-making.</p><p><b>Key Responsibilities:</b></p><p>&#10003; Design and build scalable ETL/ELT pipelines for structured and unstructured data.</p><p>&#10003; Develop data infrastructure using cloud services (AWS Glue, S3, Redshift, etc.).</p><p>&#10003; Optimize data storage, processing, and query performance using PySpark or similar tools.</p><p>&#10003; Implement and maintain data security, governance, and access control mechanisms.</p><p>&#10003; Monitor and troubleshoot pipeline performance to ensure high availability and reliability.</p><p>&#10003; Collaborate with business and analytics teams to understand and meet data requirements.</p><p>&#10003; Ensure seamless integration of data from internal and external sources.</p><p>&#10003; Create clear technical documentation for pipelines, workflows, and architecture.</p><p>&#10003; Participate in client meetings, provide status updates, and highlight risks or blockers.</p><p>&#10003; Drive automation, standardization, and process improvements across data engineering functions.</p><p><b>Skills &amp; Qualifications:</b></p><ul><li><b>Programming</b>: Strong in SQL, Python, or Scala</li></ul><ul><li><b>Big Data</b>: Experience with PySpark, Spark, or Hadoop</li></ul><ul><li><b>Cloud Platforms</b>: Hands-on with AWS, Azure, or GCP (Glue, S3, Redshift, BigQuery, etc.)</li></ul><ul><li><b>ETL Tools</b>: Familiarity with Airflow, Azure Data Factory, or other orchestration tools</li></ul><ul><li><b>DevOps &amp; CI/CD</b>: Experience with Git, pipeline automation, and deployment tools</li></ul><ul><li><b>Security &amp; Governance</b>: Knowledge of data compliance and best practices</li></ul><ul><li><b>Monitoring Tools</b>: Experience with CloudWatch, Prometheus, or Datadog</li></ul><ul><li><b>Education</b>: Bachelor's degree in computer science, Engineering, or related field</li></ul><ul><li><b>Certifications</b>: AWS/Azure/GCP cloud certifications are a plus</li></ul><p><b>Experience:</b></p><p>&#10003; 2&ndash;3 years of experience in Data Engineering or similar roles</p><p>&#10003; Strong experience in building cloud-based data pipelines and real-time data processing</p><p>Job Types: Full-time, Permanent</p><p>Pay: &#8377;450,000.00 per year</p><p>Benefits:</p><ul><li>Health insurance</li><li>Provident Fund</li></ul><p>Work Location: In person</p>

Data Engineer

The Services Tree Enterprises 

<p><b>Job Description:</b><br/>SourceTeK is hiring a <b>Data Engineer</b> for one of our clients based in <b>Johar Town, Lahore</b>. We are looking for a motivated professional with <b>2&ndash;3+ years of experience</b> in data engineering, especially someone who has <b>hands-on experience with DBT tools</b> and a strong understanding of modern data pipelines.</p><p><b>Key Responsibilities:</b></p><ul><li>Build, maintain, and optimize data pipelines</li><li>Work with DBT tools for data modeling and transformations</li><li>Collaborate with cross-functional teams to ensure data accuracy and availability</li><li>Troubleshoot and optimize data workflows</li></ul><p><b>Requirements:</b></p><ul><li>2&ndash;3+ years of experience as a Data Engineer</li><li>Strong experience with <b>DBT tools</b></li><li>Proficiency in SQL and data pipeline development</li><li>Experience with cloud platforms is a plus</li><li>Strong analytical and problem-solving skills</li></ul><p><b>How to Apply:</b><br/>If you&rsquo;re interested, please <b>send your resume to talent@sourcetek.co</b> or <b>apply through Indeed</b>.</p><p>Job Type: Full-time</p><p>Pay: Rs120,000.00 - Rs200,000.00 per month</p><p>Experience:</p><ul><li>Data Engineer: 2 years (Preferred)</li></ul><p>Work Location: In person</p>

NRIT UPSKILLS SOFTWARE SOLUTIONS PVT LTD - Client NTT DATA

<p>Required Experience &amp; Technical Stack</p><p>&#61623; We are looking for an Engineer II to join our growing team of data experts. This role is</p><p>responsible for developing, maintaining and optimizing data pipelines, delivering data timely</p><p>to our cross-functional teams and support reporting and analytics. The Engineer will work</p><p>closely with product partner, data analyst, report developers, architects, data scientists and</p><p>other engineers on data initiatives and will be involved in designing our next generation of</p><p>data platform architecture to support increasing data demands.</p><p>Primary Responsibilities</p><p>&#61623; Design, build and maintain scalable and reliable data pipelines to process large volumes of</p><p>structures, semi-structured and un-structured data from diverse sources.</p><p>&#61623; Develop ETL processes to ingest, transform and load data into data warehouse, ensuring data</p><p>quality, integrity, and consistency.</p><p>&#61623; Collaborate with business stakeholders, report developers, and data scientists to understand</p><p>data requirements and translate then into technical solutions for various business purposes.</p><p>&#61623; Optimize performance and efficiency of data infrastructure and processes, including data</p><p>storage, processing, and querying.</p><p>&#61623; Implement data governance policies and best practice to ensure compliance, security, and</p><p>privacy of sensitive data.</p><p>&#61623; Troubleshoot data issues, identify root cause, and implement solutions in timely manner.</p><p>&#61623; Excellent communication and collaboration skills, with the ability to work effectively in a</p><p>cross-functional team environment.</p><p>Basic Qualifications</p><p>&#61623; We are looking for candidate with 4+ years of experience in a Data Engineer role and has</p><p>attained bachelor&rsquo;s degree in technology and has experienced with Agile/Scrum</p><p>development process and methodologies.</p><p>&#61623; Experience as a Data Engineer with strong track record of designing and implementing data</p><p>solutions.</p><p>&#61623; Experience in programming language as Python or Java, with experience in building data</p><p>pipelines and workflows.</p><p>&#61623; Experience with cloud data warehousing technologies, such as Snowflake and Redshift.</p><p>&#61623; Experience with distributed computing frameworks such as Cloudera, Apache Hadoop and</p><p>Spark.</p><p>&#61623; Experience with Cloud platforms such as AWS, Azure or Google Cloud Platform.</p><p>&#61623; Experience with AWS cloud services, such as S3, EC2, EMR, Glue, CloudWatch, Athena,</p><p>Lambda.</p><p>&#61623; Experience with containerization and orchestration technologies, such as Docker and</p><p>Kubernetes.</p><p>&#61623; Experience with building CI/CD pipeline using tools, such has GitLab and Bitbucket.</p><p>&#61623; Experience with data pipeline orchestration tools, such as Airflow and Jenkins.</p><p>&#61623; Knowledge of database concepts, data modelling, schemas and query languages, SQL and</p><p>Hive.</p><p>&#61623; Knowledge of data visualization and reporting tools, such as MicroStrategy, Tableau and</p><p>Power BI.</p><p>&#61623; Knowledge of data quality and monitoring techniques and tools, such as Great Expectations</p><p>or similar.</p><p>&#61623; Knowledge of data governance processes, lineage, cataloging, dictionaries using tools, such</p><p>as DataHub or similar.</p><p>&#61623; Knowledge of streaming data processing and real-time analytics technologies, such as Kafka.</p><p>&#61623; Retail experience is plus.</p><p>Candidate Should Be</p><p>&#61623; Bachelors/Masters in computer science required</p><p>Job Type: Contractual / Temporary<br/>Contract length: 12 months</p><p>Pay: &#8377;1,000,000.00 - &#8377;1,617,623.53 per year</p><p>Work Location: In person</p>

Buzzworks Business Services Pvt Ltd

<ul><li><b>Job Title :</b> Data Engineer</li><li>Job Type :<b>Contract To Hire (9 Months) After 9 months payroll from the ClientSide</b></li><li><b>Yrs of Experience : </b> 4 - 14 yrs</li><li><b>Notice Period :</b> Immediate - 30 Days</li><li><b>Interview Date :</b> 06/12/25 (Saturaday)</li><li><b>Primary Skill Set : GCP, Big Query, Data Pipeline Development, SQL, Python/ Java </b></li></ul><p><b>About the Role</b></p><p>We are seeking a highly skilled <b>Data Engineer</b> to design, develop, and optimize scalable data pipelines and solutions on <b>Google Cloud Platform (GCP)</b>. The ideal candidate will have strong expertise in <b>BigQuery, SQL, Python/Java</b>, and hands-on experience in building robust <b>data pipelines</b> to support advanced analytics and business intelligence.</p><p><b>Key Responsibilities</b></p><ul><li>Design, develop, and maintain scalable <b>data pipelines</b> and ETL/ELT processes.</li><li>Work extensively on <b>GCP services</b> including BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Composer.</li><li>Optimize data models and queries for performance and cost efficiency in <b>BigQuery</b>.</li><li>Collaborate with data analysts, data scientists, and business stakeholders to understand data requirements.</li><li>Ensure data quality, governance, and security across all data pipelines and storage systems.</li><li>Troubleshoot and resolve data pipeline issues in real time.</li><li>Contribute to architecture design discussions and provide best practices for data engineering.</li></ul><p><b>Required Skills</b></p><ul><li>Strong hands-on experience with <b>Google Cloud Platform (GCP)</b>, particularly <b>BigQuery</b>.</li><li>Proven expertise in building and maintaining <b>data pipelines</b>.</li><li>Strong <b>SQL</b> skills for query optimization and large-scale data manipulation.</li><li>Proficiency in <b>Python</b> or <b>Java</b> for developing scalable ETL/ELT solutions.</li><li>Good understanding of <b>data modeling, partitioning, and performance tuning</b>.</li><li>Experience with workflow orchestration tools (e.g., Airflow/Cloud Composer) is a plus.</li><li>Familiarity with CI/CD, version control (Git), and agile methodologies.</li></ul><p><b>Good to Have</b></p><ul><li>Exposure to <b>streaming data pipelines</b> (Pub/Sub, Kafka).</li><li>Knowledge of <b>machine learning pipelines</b> integration.</li><li>Experience with <b>other cloud platforms (AWS/Azure)</b>.</li></ul><p><b>Soft Skills</b></p><ul><li>Strong analytical and problem-solving skills.</li><li>Excellent communication and collaboration abilities.</li><li>Ability to work independently and as part of a team in a fast-paced environment.</li></ul><p>Job Type: Contractual / Temporary</p><p>Pay: &#8377;500,000.00 - &#8377;4,000,000.00 per year</p><p>Benefits:</p><ul><li>Food provided</li><li>Health insurance</li><li>Provident Fund</li></ul><p>Work Location: In person</p>

GCP Data Engineer/ Senior data engineer

Dentsu

<div>The purpose of this role is to maintain, improve, clean and manipulate data in the business&rsquo;s operational and analytics databases. The Data Engineer works with the business&rsquo;s software engineers, data analytics teams, data scientists and data warehouse engineers in order to understand and aid in the implementation of database requirements, analyse performance, and troubleshoot any existent issues.<div></div><div>
Job Description:</div>
Key responsibilities:
Creates and maintains optimal data pipeline architecture
Assembles large, complex data sets that meet functional / non-functional business requirements
Identifies, designs and implements internal process improvements: automating manual processes, optimising data delivery, re-designing infrastructure for greater scalability, etc
Builds analytics tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
Works with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs
Keeps our data separated and secure
Creates data tools for analytics and data scientist team members that assist them in building and optimising our product into an innovative industry leader
Works with data and analytics experts to strive for greater functionality in our data systems<div></div><div>
Location:</div>
India - Maharashtra- Pune - Baner - Amar Madhuban Tech Park<div></div><div>
Brand:</div>
Merkle<div></div><div>
Time Type:</div>
Full time<div></div><div>
Contract Type:</div>
Permanent</div>

Pragma Edge

<div></div><div><div><div><div><div><div><div><div><div><div><div><div><div><ul><div><div>Job Category - IT</div>
</div><div></div><div><div>Job Type - Full Time</div>
</div><div></div><div><div>No of openings: 01</div>
</div><div></div><div><div>Experience : 6 to 8 Years</div>
</div><div></div><div><div>Job Location - Hyderabad</div>
</div></ul></div></div></div></div></div></div></div></div></div></div><div></div><div><div><div><div><div><div><div>Senior IT Data Engineer</div>
</div></div><div><div><p>The Senior IT Data Engineers are experts in data streaming, building data pipelines that support real-time data refreshes and are cost-optimized for computing resources. They deeply understand data security, implementing row-level and column-level security measures.
</p><p>This role typically involves leading the implementation of complex projects, using advanced big data technologies, and ensuring robust data pipeline orchestration across multiple systems.</p>
</div></div><div><div><div>Job Responsibilities</div>
</div></div><div><div><ul><li>Lead the design and orchestration of complex data pipelines and ETL processes, focusing on optimizing data models.</li><li>
Architect scalable data solutions, including data warehousing and big data technologies, and ensure alignment with organizational goals.</li><li>
Implement advanced data governance frameworks and ensure robust data security measures.</li><li>
Optimize cloud costs by strategically using cloud resources and efficient data architectures.</li><li>
Lead the development of data visualization strategies, ensuring they contribute to data-driven decision-making at a strategic level.</li><li>
Design and deploy enterprise-scale data visualization solutions, optimizing performance within tools.</li><li>
Oversee complex data pipeline orchestration and ensure seamless integration across systems.</li><li>
Interact with 3rd party vendors to incorporate new functionality and evaluate tools.</li><li>
Resolve complex escalations and situations, driving effective resolutions.</li><li>
Ensure adherence to data privacy and security practices to protect sensitive information.</li><li>
Communicate effectively with stakeholders to understand and address their needs through data visualization.</li><li>
Perform other assigned job-related duties that align with our organization&rsquo;s vision, mission, and values and fall within your scope of practice.</li></ul>
</div></div><div><div><div>Qualifications</div>
</div></div><div><div><ul><li>Education: Bachelor&rsquo;s Degree or relevant experience.</li><li>
Preferred Certification(s): Any relevant IT Certification.</li><li>
Experience: 5+ years of relevant and practical experience.</li></ul><p><b>
Special Skills:</b></p><ul><li>
Advanced skills in data streaming and real-time data processing.</li><li>
Expertise in designing complex data architectures and scalable systems.</li><li>
Deep knowledge of big data technologies and extensive experience with data security measures.</li><li>
Ability to lead data architecture reviews and approvals.</li></ul><p><b>
Soft Skills:</b></p><ul><li>
Project Management: Learning and understanding complex data initiatives from conception to completion.</li><li>
Mentorship: Guiding junior team members in their professional development.</li><li>
Strategic Thinking: Aligning data solutions with broader organizational goals.</li><li>
Communication: Articulating data strategies to both technical and non-technical stakeholders.</li><li>
Problem-Solving: Resolving complex data challenges and escalations.</li><li>
Adaptability: Staying current with rapidly evolving data technologies and best practices.</li><li>
Creativity: Innovating new approaches to data pipeline design and optimization.</li></ul></div></div></div></div></div></div></div></div></div><p></p>

Prodapt Solutions

<div>Overview:
<div>
The Data Engineer designs, builds, and maintains robust data pipelines and ETL processes to support analytics, machine learning, and BI initiatives. This role ensures high data quality, integrity, and availability across platforms such as OCI, Snowflake, Azure Data Lake, and Tableau.</div>
Responsibilities:
<ul><li>Data Pipeline Development</li><li>Develop and optimize ETL pipelines for ingesting, transforming, and curating datasets.</li><li>Support data migration, remediation, and integration across enterprise systems.</li><li>Collaboration &amp; Integration</li><li>Partner with data scientists and architects to deliver feature stores and semantic layers.</li><li>Validate data products with business stakeholders.</li><li>Data Quality &amp; Governance</li><li>Ensure data lineage, metadata capture, and compliance with governance standards.</li><li>Monitor data integrity metrics and implement preventive strategies for recurring data issues.</li><li>Performance Optimization</li><li>Optimize data infrastructure for scalability and reliability.</li></ul>
Requirements:
<ul><li>Technical Expertise</li><li>5+ years in data engineering roles with strong experience in ETL and pipeline development.</li><li>Proficiency in SQL, Python, and cloud data platforms (OCI, Snowflake, Azure Data Lake).</li><li>Experience with data modeling and integration tools.</li><li>Collaboration &amp; Communication</li><li>Strong teamwork skills across engineering, analytics, and business teams.</li><li>Tools &amp; Technologies</li><li>Familiarity with data governance, metadata management, and data quality frameworks.</li></ul></div>

WIN_Data_Engineer

Search 10000+ Data pipeline engineer jobs near you this December. New roles added regularly across leading industries. Apply today.

10000 data pipeline engineer jobs NEAR_ME

Data Engineer

Data Engineer

Data Engineer

GCP Data Engineer/ Senior data engineer

Data Engineer

Data Engineer

WIN_Data_Engineer