data pipeline engineer

Succedere solutions opc private limited

Role: Data Science EngineerLocation: ChennaiExperience: 2 YearsCTC: Up to &#8377;6 LPANotice Period: Immediate preferred / Max 30 DaysMandatory Skills: Python, SQL, BI Tools, Data Pipeline Job Summary:We are looking for an experienced and motivated Data Science Engineer with strong expertise in data engineering and analytics. The ideal candidate must be skilled in building scalable data pipelines, transforming large datasets, and supporting data-driven decision-making. The role includes working closely with data scientists, analysts, and backend teams to ensure high-quality, reliable, and well-orchestrated data systems.Key Responsibilities:Design, build, and maintain scalable data pipelines for structured and unstructured data.Develop ETL workflows to ingest data from multiple sources ensuring data quality and consistency.Collaborate with data scientists and analysts to prepare data for analytics and ML use cases.Manage and optimize data warehouses, data lakes, and real-time streaming environments.Strong Python skills for pipeline automation and data processing.Experience with FastAPI is a plus for building scalable APIs.Exposure to LLM technologies is an added advantage.Implement data governance, cataloging, and lineage monitoring.Monitor pipeline health and troubleshoot issues.Work closely with cross-functional teams to meet data requirements.Support data migration, transformation, and integrations.Stay updated on emerging technologies in big data, analytics, and cloud platforms.Skills &amp; Qualifications:Bachelor&rsquo;s degree in Computer Science, Engineering, or a related field (Master&rsquo;s preferred).Minimum 2+ years of experience in data engineering or analytics engineering.Proficiency in SQL and experience with PostgreSQL/MySQL.Experience with NoSQL databases such as MongoDB or Terra.Strong knowledge of big data tools (Kafka, Spark, Airflow).Expertise in orchestration tools like Apache Airflow.Experience in data modeling, warehousing, and performance tuning.Familiarity with BI tools such as Metabase or Power BI.Strong programming skills in Python or Scala.Understanding of data security and privacy standards (e.g., GDPR).Good problem-solving, analytical, and communication skills.Experience supporting ML pipelines is a plus.Job Types: Full-time, PermanentPay: &#8377;500,000.00 - &#8377;600,000.00 per yearBenefits:<ul><li>Health insurance</li><li>Provident Fund</li></ul>Work Location: In person

Data Science Engineer

Nuarsh TECHSERVE

Job Title: Data EngineerExperience: 2-4 yearsLocation: Indore, IndiaAbout the RoleThis is a full-time on-site role for a Data Engineer located in Indore. The ideal candidate will have hands-on experience working with modern cloud data platforms such as Azure, AWS, Snowflake, Microsoft Fabric, and Databricks. You will be responsible for building and optimizing data pipelines, ensuring data quality, and supporting data-driven initiatives across the organization.Key Responsibilities<ul><li>Design, build, and maintain scalable and reliable data pipelines and ETL/ELT processes.</li><li>Work with cloud technologies such as Azure, AWS, Snowflake, Microsoft Fabric, and Databricks to manage and optimize data workflows.</li><li>Collaborate with Data Leads, Analysts, and cross-functional teams to understand data needs and deliver efficient solutions.</li><li>Implement and maintain data models that support business intelligence, analytics, and reporting needs.</li><li>Ensure the accuracy, consistency, and reliability of data across all systems.</li><li>Troubleshoot data pipeline issues and optimize performance.</li><li>Participate in data architecture discussions and contribute to continuous improvement of data systems.</li><li>Maintain proper documentation of data workflows, models, and processes.</li></ul>Qualifications &amp; Skills<ul><li>2&ndash;4 years of hands-on experience in Data Engineering or related fields.</li><li>Strong experience in Data Modeling, Data Management, and ETL/ELT pipeline development.</li><li>Solid understanding of Data Analytics and Data Analysis concepts.</li><li>Proficiency with one or more cloud platforms: Azure, AWS, Snowflake, Microsoft Fabric, Databricks.</li><li>Experience with SQL and at least one programming language such as Python or Scala.</li><li>Strong analytical, problem-solving, and troubleshooting skills.</li><li>Ability to work independently as well as collaboratively in a team environment.</li><li>Excellent communication skills and attention to detail.</li><li>Bachelor&rsquo;s degree in Computer Science, Information Technology, or a related field.</li><li>Experience in a consulting environment is a plus.</li></ul>Job Type: Full-timePay: &#8377;800,000.00 - &#8377;1,000,000.00 per yearWork Location: In person

Data Engineer

FiveRivers Technologies

<div><div><h3 class="jobSectionHeader">Job Information</h3>
<ul><div>Date Opened
<div>11/13/2025
</div></div><div>Job Type
<div>Full time
</div></div><div>Work Experience
<div>3-6 years
</div></div><div>Industry
<div>IT Services
</div></div><div>City
<div>Lahore
</div></div><div>State/Province
<div>Punjab
</div></div><div>Country
<div>Pakistan
</div></div><div>Zip/Postal Code
<div>54000
</div></div></ul></div><div><h3 class="jobSectionHeader">Job Description</h3>
<div><div>As a Backend Engineer, you will play a key role in advancing our analytics platform to the next level. Our data pipeline architecture focuses on scalability, flexibility, and high performance using cutting-edge AWS technologies. 
You&rsquo;ll be part of a small, innovative team that continually enhances our pipeline by adopting the latest tools and approaches. This role involves leading initiatives to transition components of our analytics process to a scalable, serverless architecture&mdash;supporting advanced analytics with improved computational capabilities and performance.<div> 
 
</div>We are looking for someone who thrives on innovation, enjoys solving complex data challenges, and can mentor others while driving major backend projects forward.<div></div><h3 class="jobSectionHeader"> 
Responsibilities</h3><ul><li> 
Lead major initiatives within the data pipeline team.</li><li> 
Design, develop, and maintain scalable ETL/ELT pipelines.</li><li> 
Participate in architectural decisions and evaluate new technologies for pipeline optimization.</li><li> 
Collaborate cross-functionally with teams such as Data Integration, Support, and TAM to ensure pipeline reliability and visibility.</li><li> 
Drive the migration of components toward serverless and scalable architectures.</li><li> 
Mentor and guide junior engineers by providing technical feedback and support.</li><li> 
Ensure code quality, performance, and adherence to best practices in backend development.</li></ul></div><h3 class="jobSectionHeader"> 
Requirements</h3><ul><li>
Bachelor&rsquo;s degree in Computer Science, Software Engineering, or a related field.</li><li> 
2+ years of experience in backend development.</li><li> 
Proven experience developing ETL/ELT pipelines from the ground up.</li><li> 
Strong proficiency in Java, Python, and SQL.</li><li> 
Hands-on experience with AWS services such as S3, Glue, Athena, Lambda, and Step Functions (strong plus).</li><li> 
Familiarity with Infrastructure as Code (IaC) tools such as AWS CDK, CloudFormation, or Terraform.</li><li> 
Strong problem-solving skills and a passion for building scalable, data-driven systems.</li><li> 
Excellent communication and collaboration abilities.</li></ul></div></div></div>

Software Engineer (Python)_01

Datamatics Global Services Ltd

<div><h2 class="jobSectionHeader">Cloudera Data Engineer</h2>
Location: Remote (UAE Time Zone) 
Experience: 5+ Years 
Availability: Immediate joiners or candidates who can start within 30 days preferred<h3 class="jobSectionHeader">
Role Overview</h3>
We are seeking an experienced Cloudera Data Engineer to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.<h3 class="jobSectionHeader">
Key Responsibilities</h3><ul><li>
Design, build, and maintain ETL/ELT pipelines using Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Optimize data pipelines for performance, reliability, and cost efficiency across batch and streaming workloads.</li><li>
Develop automation scripts and reusable frameworks for data ingestion, transformation, and cleansing.</li><li>
Collaborate with Cloudera Administrators to tune performance and manage cluster resources efficiently.</li><li>
Troubleshoot and resolve data pipeline failures, performance bottlenecks, and data quality issues.</li><li>
Monitor cluster utilization, job performance, and data throughput using Cloudera Manager.</li><li>
Work with DevOps/Infrastructure teams to define and maintain CI/CD pipelines, automation for deployments, and monitoring of data jobs.</li><li>
Participate in incident management, root cause analysis, and preventive maintenance.</li><li>
Ensure pipelines meet SLAs, data quality, and latency requirements for analytics workloads.</li></ul><h3 class="jobSectionHeader">
Requirements</h3><ul><li>
5+ years of experience as a Data Engineer in a Cloudera-based environment.</li><li>
Strong hands-on experience with Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Solid understanding of ETL design, performance optimization, and data governance principles.</li><li>
Experience with Linux/Unix scripting and CI/CD integration.</li><li>
Excellent troubleshooting and communication skills.</li></ul>
IppsLo883Y</div>

Cloudera Data Engineer

Datamatics Technologies

<div>
 
 Cloudera Data Engineer
 
 
 
 Location:
 
 Remote (UAE Time Zone)
 
 
 Experience:
 
 5+ Years
 
 
 Availability:
 
 Immediate joiners or candidates who can start within 30 days preferred
 
 
 Role Overview
 
 
 We are seeking an experienced
 
 Cloudera Data Engineer
 
 to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.
 
 
 Key Responsibilities
 
 <ul>
 <li>
 
 Design, build, and maintain
 
 ETL/ELT pipelines
 
 using Spark, Hive, Impala, NiFi, and Oozie.
 
 </li>
 <li>
 
 Optimize data pipelines for
 
 performance, reliability, and cost efficiency
 
 across batch and streaming workloads.
 
 </li>
 <li>
 
 Develop
 
 automation scripts and reusable frameworks
 
 for data ingestion, transformation, and cleansing.
 
 </li>
 <li>
 
 Collaborate with Cloudera Administrators to
 
 tune performance
 
 and manage cluster resources efficiently.
 
 </li>
 <li>
 
 Troubleshoot and resolve
 
 data pipeline failures, performance bottlenecks, and data quality issues
 
 .
 
 </li>
 <li>
 
 Monitor
 
 cluster utilization, job performance, and data throughput
 
 using Cloudera Manager.
 
 </li>
 <li>
 
 Work with DevOps/Infrastructure teams to define and maintain
 
 CI/CD pipelines, automation for deployments
 
 , and monitoring of data jobs.
 
 </li>
 <li>
 
 Participate in
 
 incident management, root cause analysis, and preventive maintenance
 
 .
 
 </li>
 <li>
 
 Ensure pipelines meet
 
 SLAs, data quality, and latency requirements
 
 for analytics workloads.
 
 </li>
 </ul>
 
 Requirements
 
 <ul>
 <li>
 
 5+ years of experience as a
 
 Data Engineer
 
 in a Cloudera-based environment.
 
 </li>
 <li>
 
 Strong hands-on experience with
 
 Spark, Hive, Impala, NiFi, and Oozie
 
 .
 
 </li>
 <li>
 
 Solid understanding of
 
 ETL design, performance optimization
 
 , and
 
 data governance
 
 principles.
 
 </li>
 <li>
 
 Experience with
 
 Linux/Unix scripting
 
 and
 
 CI/CD integration
 
 .
 
 </li>
 <li>
 
 Excellent troubleshooting and communication skills.
 
 </li>
 </ul>
</div>

<div><h2 class="jobSectionHeader">Cloudera Data Engineer</h2>
Location: Remote (UAE Time Zone) 
Experience: 5+ Years 
Availability: Immediate joiners or candidates who can start within 30 days preferred<h3 class="jobSectionHeader">
Role Overview</h3>
We are seeking an experienced Cloudera Data Engineer to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.<h3 class="jobSectionHeader">
Key Responsibilities</h3><ul><li>
Design, build, and maintain ETL/ELT pipelines using Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Optimize data pipelines for performance, reliability, and cost efficiency across batch and streaming workloads.</li><li>
Develop automation scripts and reusable frameworks for data ingestion, transformation, and cleansing.</li><li>
Collaborate with Cloudera Administrators to tune performance and manage cluster resources efficiently.</li><li>
Troubleshoot and resolve data pipeline failures, performance bottlenecks, and data quality issues.</li><li>
Monitor cluster utilization, job performance, and data throughput using Cloudera Manager.</li><li>
Work with DevOps/Infrastructure teams to define and maintain CI/CD pipelines, automation for deployments, and monitoring of data jobs.</li><li>
Participate in incident management, root cause analysis, and preventive maintenance.</li><li>
Ensure pipelines meet SLAs, data quality, and latency requirements for analytics workloads.</li></ul><h3 class="jobSectionHeader">
Requirements</h3><ul><li>
5+ years of experience as a Data Engineer in a Cloudera-based environment.</li><li>
Strong hands-on experience with Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Solid understanding of ETL design, performance optimization, and data governance principles.</li><li>
Experience with Linux/Unix scripting and CI/CD integration.</li><li>
Excellent troubleshooting and communication skills.</li></ul>
l17IPmQvjU</div>

<div><h2 class="jobSectionHeader">Cloudera Data Engineer</h2>
Location: Remote (UAE Time Zone) 
Experience: 5+ Years 
Availability: Immediate joiners or candidates who can start within 30 days preferred<h3 class="jobSectionHeader">
Role Overview</h3>
We are seeking an experienced Cloudera Data Engineer to design, build, and maintain data pipelines and processing frameworks across Cloudera&rsquo;s ecosystem. The ideal candidate will have strong expertise in Spark, Hive, Impala, NiFi, and Oozie, and proven experience optimizing large-scale data pipelines for performance and reliability.<h3 class="jobSectionHeader">
Key Responsibilities</h3><ul><li>
Design, build, and maintain ETL/ELT pipelines using Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Optimize data pipelines for performance, reliability, and cost efficiency across batch and streaming workloads.</li><li>
Develop automation scripts and reusable frameworks for data ingestion, transformation, and cleansing.</li><li>
Collaborate with Cloudera Administrators to tune performance and manage cluster resources efficiently.</li><li>
Troubleshoot and resolve data pipeline failures, performance bottlenecks, and data quality issues.</li><li>
Monitor cluster utilization, job performance, and data throughput using Cloudera Manager.</li><li>
Work with DevOps/Infrastructure teams to define and maintain CI/CD pipelines, automation for deployments, and monitoring of data jobs.</li><li>
Participate in incident management, root cause analysis, and preventive maintenance.</li><li>
Ensure pipelines meet SLAs, data quality, and latency requirements for analytics workloads.</li></ul><h3 class="jobSectionHeader">
Requirements</h3><ul><li>
5+ years of experience as a Data Engineer in a Cloudera-based environment.</li><li>
Strong hands-on experience with Spark, Hive, Impala, NiFi, and Oozie.</li><li>
Solid understanding of ETL design, performance optimization, and data governance principles.</li><li>
Experience with Linux/Unix scripting and CI/CD integration.</li><li>
Excellent troubleshooting and communication skills.</li></ul>
BJoI7cAFtW</div>

Search 10000+ Data pipeline engineer jobs near you this November. New roles added regularly across leading industries. Apply today.

10000 data pipeline engineer jobs NEAR_ME

Data Science Engineer

Data Engineer

Software Engineer (Python)_01

Cloudera Data Engineer

Cloudera Data Engineer

Cloudera Data Engineer

Cloudera Data Engineer