data engineer

Tata Consultancy Services (TCS)

<div></div><div><div><div><div><div><div>TCS Inviting applicants for
</div><div>Role: Azure Data Engineer
</div><div>Experience: 5 to 10 years
</div><div>Location: Chennai
</div><div>Job Description:
</div><div>1. Expertise in PySpark, Python</div><br>
<div><br>
</div><div>2. Knowledge and Skill in Databricks</div><br>
<div><br>
</div><div>3. Strong proficiency in Azure services: Data Factory, Synapse, Data Lake, Databricks.</div><br>
<div><br>
</div><div>4. Hands-on experience with ETL tools and data pipeline orchestration.</div><br>
<div><br>
</div><div>5. Proficiency in Python or Scala for data processing.</div><br>
<div><br>
</div><div>6. Knowledge of SQL and NoSQL databases.</div><br>
<div><br>
</div><div>7. Familiarity with data modeling and data warehousing concepts.</div><br>
<div><br>
</div><div>8. Understanding of security best practices for data in AWS.</div><br>
<div><br>
</div><div>9. Good hands on experience on Python, Numpy , pandas.</div><br>
<div><br>
</div><div>10. Experience in building ETL/ Data Warehouse transformation process.</div><br>
<div><br>
</div><div>11. Experience working with structured and unstructured data.</div><br>
<div><br>
</div><div>12. Developing Big Data and non-Big Data cloud-based enterprise solutions in PySpark and SparkSQL and related frameworks/libraries,</div><br>
<div><br>
</div><div>13. Developing scalable and re-usable, self-service frameworks for data ingestion and processing,</div><br>
<div><br>
</div><div>14. Integrating end to end data pipelines to take data from data source to target data repositories ensuring the quality and consistency of data,</div><br>
<div><br>
</div><div>Knowledge of big data frameworks (Spark, Hadoop).
</div></div></div></div></div><div><div><div><b>Location</b></div>
<div>Chennai</div>
</div><div><div><b>Job Function</b></div>
<div>IT INFRASTRUCTURE SERVICES</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>381897</div>
</div><div><div><b>Desired Skills</b></div>
<div>Azure Data Factory | Azure</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF ENGINEERING</div></div></div><div></div>

Data Engineer

Client of Clastech Solutions & Services Pvt. Ltd.

<p><b>Please provide responses to the screening questionnaire. </b></p><p><b>Job Code: ITMP1411-01</b></p><p><b>Designation: Software Engineer (Data Engineer) </b><br/><b>Location: Ghansoli, Mumbai</b></p><p><b>Nature: Work from office only </b><br/><b>Budget: 17 LPA CTC per year (Including 7% Variable) </b><br/><b>Experience: 3 to 5 Years</b></p><p><b>Job Description</b></p><p>We are seeking an experienced Data Engineer with 3+ years of hands-on expertise in building robust data pipelines, working with big data technologies, and managing cloud-based data infrastructures. The ideal candidate will play a key role in designing, developing, and optimizing data workflows to ensure reliable and scalable data operations across the organization.</p><p><b>Key Responsibilities</b></p><p>&middot; Build and maintain scalable ETL/ELT data pipelines to support analytics, reporting, and machine learning workloads.</p><p>&middot; Write clean, efficient, and maintainable Python or Scala code for data ingestion, transformation, and processing tasks.</p><p>&middot; Use SQL extensively to query, manipulate, and optimize large datasets.</p><p>&middot; Work with big data technologies such as Apache Spark, Airflow, Kafka, or similar tools for distributed processing and workflow orchestration.</p><p>&middot; Design, implement, and manage data infrastructure on cloud platforms including AWS, Google Cloud Platform (GCP), or Microsoft Azure.</p><p>&middot; Collaborate with data scientists, analysts, and cross-functional teams to ensure smooth data availability and quality.</p><p>&middot; Monitor pipeline performance, troubleshoot issues, and optimize workflows for efficiency and reliability.</p><p>&middot; Ensure best practices in data security, quality, and governance across all data engineering processes.</p><p><b>Required Qualifications</b></p><p>&middot; 3+ years of experience as a Data Engineer or in a similar data-focused engineering role.</p><p>&middot; Strong programming skills in Python or Scala.</p><p>&middot; Excellent proficiency in SQL with experience working on large-scale datasets.</p><p>&middot; Hands-on experience with Spark, Airflow, Kafka, or equivalent big data and orchestration tools.</p><p>&middot; Practical exposure to cloud services such as AWS, GCP, or Azure.</p><p>&middot; Solid understanding of ETL/ELT principles, data warehousing concepts, and distributed computing.</p><p>&middot; Strong problem-solving abilities and attention to detail.</p><p>&middot; Effective communication and collaboration skills to work in a team-oriented environment.</p><p>Job Type: Full-time</p><p>Pay: Up to &#8377;1,700,000.00 per year</p><p>Application Question(s):</p><ul><li>Please provide your CV in word format. You may upload it here or send it to Services@clastechsolutions.com </li></ul><p>Please provide responses to the screening questions. </p><p>How many years experience do you have in the following skills:<br/>- Programming skills in Python or Scala:<br/>- SQL with experience working on large-scale datasets: <br/>- Hands-on experience with Spark, Airflow, Kafka, or equivalent big data and orchestration tools: <br/>- Practical exposure to cloud services such as AWS, GCP, or Azure: <br/>- ETL/ELT principles, data warehousing concepts, and distributed computing:</p><p>Current Location:</p><p>Are you willing to work from Ghansoli office- purely WFO?:</p><p>Notice Period:</p><p>Total Experience in Years:</p><p>Nationality:</p><p>Mobile No.:</p><p>Email ID:</p><p>Upon offer issuance, how soon can you join?:</p><p>Do you have a Passport (mandatory)?:</p><p>Current CTC/monthly pay:</p><p>Expected CTC/monthly pay:</p><p>Work Location: In person</p>

CAI (Computer Aid, Inc.)

<b>Req number:</b> R6655
<br><br>
<b>Employment type:</b> Full time
<br><br>
<b>Worksite flexibility:</b> Remote Who we are
<br>CAI is a global technology services firm with over 8,500 associates worldwide and a yearly revenue of $1 billion+. We have over 40 years of excellence in uniting talent and technology to power the possible for our clients, colleagues, and communities. As a privately held company, we have the freedom and focus to do what is right—whatever it takes. Our tailor-made solutions create lasting results across the public and commercial sectors, and we are trailblazers in bringing neurodiversity to the enterprise.
<br><br>
Job Summary We are looking for a motivated Data Engineer ready to take us to the next level! If you have strong experience in building cloud-based data lake and analytics architectures using AWS and Databricks, and is proficient in Python programming for data processing and automation and are looking for your next career move, apply now.
<br><br>
Job Description We are looking for a Data Engineer that has experience in building data products using Databricks and related technologies. This position will be Full-time and Remote position. What You’ll Do
<br>Design, develop, and maintain data lakes and data pipelines on AWS using ETL frameworks and Databricks.
<br>Integrate and transform large-scale data from multiple heterogeneous sources into a centralized data lake environment.
<br>Implement and manage Delta Lake architecture using Databricks Delta or Apache Hudi.
<br>Develop end-to-end data workflows using PySpark, Databricks Notebooks, and Python scripts for ingestion, transformation, and enrichment.
<br>Design and develop data warehouses and data marts for analytical workloads using Snowflake, Redshift, or similar systems.
<br>Design and evaluate data models (Star, Snowflake, Flattened) for analytical and transactional systems.
<br>Optimize data storage, query performance, and cost across the AWS and Databricks ecosystem.
<br>Build and maintain CI/CD pipelines for Databricks notebooks, jobs, and Python-based data processing scripts.
<br>Collaborate with data scientists, analysts, and stakeholders to deliver high-performance, reusable data assets.
<br>Maintain and manage code repositories (Git) and promote best practices in version control, testing, and deployment.
<br>Participate in making major technical and architectural decisions for data engineering initiatives.
<br>Monitor and troubleshoot Databricks clusters, Spark jobs, and ETL processes for performance and reliability.
<br>Coordinate with business and technical teams through all phases of the software development life cycle. What You'll Need Required
<br>5+ years of experience building and managing Data Lake Architecture on AWS Cloud
<br>3+ years of experience with AWS Data services such as S3, Glue, Lake Formation, EMR, Kinesis, RDS, DMS, and Redshift.
<br>3+ years of experience building Data Warehouses on Snowflake, Redshift, HANA, Teradata, or Exasol.
<br>3+ years of hands-on experience working with Apache Spark or PySpark, on Databricks.
<br>3+ years of experience implementing Delta Lakes using Databricks Delta or Apache Hudi.
<br>3+ years of experience in ETL development using Databricks, AWS Glue, or other modern frameworks.
<br>Proficiency in Python for data engineering, automation, and API integrations.
<br>Experience in Databricks Jobs, Workflows, and Cluster Management.
<br>Experience with CI/CD pipelines and Infrastructure as Code (IaC) tools like Terraform or CloudFormation is a plus.
<br>Bachelor’s degree in computer science, Information Technology, Data Science, or related field.
<br>Experience working on Agile projects and methodology in general.
<br>Preferred
<br>Strong SQL, RDBMS, and data modeling skills.
<br>Experience with Databricks Unity Catalog, Delta Live Tables (DLT), and MLflow for data governance and model lifecycle.
<br>AWS or Databricks Cloud Certifications (e.g., AWS Data Analytics Specialty, Databricks Certified Data Engineer Professional) are a big plus.
<br>Understanding data security, access control, and compliance in cloud environments.
<br>Strong analytical, problem-solving, and communication skills. Physical Demands
<br>This role involves mostly sedentary work, with occasional movement around the office to attend meetings, etc.
<br>Ability to perform repetitive tasks on a computer, using a mouse, keyboard, and monitor.
<br><br>
Reasonable accommodation statement
<br>If you require a reasonable accommodation in completing this application, interviewing, completing any pre-employment testing, or otherwise participating in the employment selection process, please direct your inquiries to application.accommodations@cai.io or (888) 824 – 8111.

PayU

<b>Job Title:</b> Data Engineer
<br><b>Location:</b> Bangalore
<br><b>Experience Level:</b> 3+ Years
<br><b>Employment Type:</b> Full-Time
<br><br>
Job Summary
<br><br>
We are seeking an experienced and innovative Lead Data Engineer to drive our data systems and pipelines to the next level. You will play a pivotal role in designing, implementing, and optimizing end-to-end data solutions to support scalable and efficient data-driven decision-making processes. This position is ideal for someone passionate about large-scale data systems, cutting-edge cloud technology, and mentoring a high-performing data engineering team.
<br><br>
Key Responsibilities
<br><br>
Design, develop, and maintain scalable and performant ETL/ELT pipelines and workflows for structured and unstructured data.
<br>Lead the architecture and implementation of data engineering solutions using Python, Airflow, PySpark, and SQL.
<br>Build and manage data pipelines integrating with AWS services such as Athena, S3, EC2, and other cloud-native tools.
<br>Work closely with data scientists, analysts, and cross-functional teams to ensure data quality, availability, and accessibility.
<br>Optimize existing data architectures and query performance on massive data sets for analytical use cases.
<br>Oversee the data governance process with documentation, monitoring, and automation of data workflow processes.
<br>Mentor and guide junior data engineers in best practices, performance optimization, and cutting-edge tools.
<br>Ensure compliance with data security, privacy regulations, and organizational policies.
<br><br>
Key Qualifications
<br><br>
<b>Education:</b> Bachelor’s or Master’s degree in Computer Science, Software Engineering, Data Science, or a related field
<br><b>Experience:</b> 5+ years of experience in data engineering with at least 1 year of experience as Lead.
<br><br>
Required Technical Skills
<br><br>
Proficiency in Python for data pipeline and systems development.
<br>Strong experience with data orchestration tools, particularly Apache Airflow.
<br>Advanced SQL expertise for data transformation, analysis, and query optimization.
<br>Hands-on experience with PySpark for processing large-scale data.
<br>Expertise in cloud technologies, specifically AWS, including Athena, EC2, S3, and other serverless computing services.
<br><br>
Additional Skills & Tools
<br><br>
Familiarity with big data tools like Hadoop, EMR, or equivalent cloud-supported ecosystems.
<br>Knowledge of modern data warehousing tools and architectures (e.g., Redshift, Snowflake, BigQuery).
<br>Proficiency in CI/CD pipelines and DevOps practices for data engineering.
<br>Experience with monitoring and debugging solutions for data workflows.
<br>Strong problem-solving skills with an ability to work under tight deadlines.
<br><br>
Preferred Attributes
<br><br>
Previous experience in e-commerce, fintech, or related industries.
<br>Leadership or mentoring experience, with the ability to interact and collaborate cross-functional teams.
<br>Strong communication skills with a track record of translating business needs into technical solutions.
<br>Understanding of data security and privacy best practices.
<br><br>
What We Offer
<br>A dynamic and inclusive work environment.
<br>Competitive compensation and benefits package.
<br>Opportunities for continuous learning and professional development.
<br>The chance to work on exciting, large-scale projects leveraging state-of-the-art technologies.
<br><br>
If you’re passionate about building cutting-edge data solutions and enjoy working in a collaborative, fast-paced environment, we’d love to hear from you!
<br><br>
Apply today and become an integral part of our data journey.

Weekday AI (YC W21)

<div>
 <strong>
  This role is for one of the Weekday's clients
  <br/>
  <br/>
 </strong>
 <strong>
  Salary range: Rs 3000000 - Rs 4000000 (ie INR 30-40 LPA)
  <br/>
  <br/>
 </strong>
 Min Experience: 5 years
 <br/>
 <br/>
 Location: Bengaluru
 <br/>
 <br/>
 JobType: full-timeWe are seeking an experienced
 <strong>
  Data Engineer
 </strong>
 to design, build, and maintain robust data pipelines and architectures that power analytics, interoperability, and data-driven decision-making across our healthcare ecosystem. The ideal candidate will have a strong background in
 <strong>
  Azure data services
 </strong>
 ,
 <strong>
  ETL development
 </strong>
 ,
 <strong>
  Databricks
 </strong>
 , and hands-on experience with
 <strong>
  FHIR (Fast Healthcare Interoperability Resources)
 </strong>
 standards. This role requires a mix of technical proficiency, problem-solving ability, and a deep understanding of healthcare data integration.
 <br/>
 <br/>
 <strong>
  Requirements
  <br/>
  <br/>
 </strong>
 <strong>
  <strong>
   Key Responsibilities
   <br/>
   <br/>
  </strong>
 </strong>
 <ul>
  <li>
   Data Pipeline Development: Design, develop, and optimize scalable ETL/ELT workflows to extract, transform, and load structured and unstructured healthcare data into Azure-based data platforms (e.g., Azure Data Lake, Azure Synapse, Databricks).
  </li>
  <li>
   FHIR Data Integration: Implement and manage FHIR-based data models and APIs for healthcare data interoperability. Ensure compliance with healthcare data standards and regulatory frameworks.
  </li>
  <li>
   Data Architecture &amp; Modeling: Develop logical and physical data models to support data warehousing, reporting, and analytics needs. Ensure data accuracy, integrity, and security across all stages of the data lifecycle.
  </li>
  <li>
   Databricks Development: Build and manage scalable data processing solutions using Azure Databricks. Optimize performance of large-scale data transformations using PySpark, SQL, or Scala.
  </li>
  <li>
   Data Quality &amp; Governance: Establish and maintain data quality frameworks, validation processes, and governance standards. Ensure consistent and compliant data handling across systems.
  </li>
  <li>
   Collaboration &amp; Stakeholder Management: Partner with data scientists, analysts, architects, and business teams to deliver reliable and accessible data assets. Translate business requirements into technical solutions.
  </li>
  <li>
   Performance Optimization: Monitor, troubleshoot, and enhance data pipelines for performance, scalability, and cost efficiency in the Azure environment.
  </li>
  <li>
   Documentation &amp; Best Practices: Maintain detailed documentation of architecture, data flows, and processes. Contribute to continuous improvement by implementing data engineering best practices and automation.
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Required Skills &amp; Qualifications
  <br/>
  <br/>
 </strong>
 <li>
  Education: Bachelor's or Master's degree in Computer Science, Information Technology, Data Engineering, or a related field.
 </li>
 <li>
  Technical Skills:
 </li>
 <ul>
  <li>
   Strong expertise in Azure Data Services - Azure Data Factory, Azure Synapse, Azure Data Lake, and Azure SQL.
  </li>
  <li>
   Hands-on experience with Databricks for data processing, transformation, and analytics.
  </li>
  <li>
   Proven ability to design and maintain ETL/ELT pipelines using modern data engineering tools and frameworks.
  </li>
  <li>
   Proficiency in SQL, Python, or Scala for data manipulation and automation.
  </li>
  <li>
   Solid understanding of FHIR standards, data schemas, and API integration for healthcare data exchange.
  </li>
  <li>
   Experience with data modeling, metadata management, and data governance frameworks.
  </li>
  <li>
   Familiarity with CI/CD, Git, and DevOps practices in data engineering environments.
  </li>
  <li>
   Experience: 5-10 years of professional experience in data engineering, with at least 2 years of exposure to healthcare or interoperability projects leveraging FHIR.
  </li>
  <li>
   Soft Skills:
  </li>
  <ul>
   <li>
    Strong analytical and problem-solving abilities.
   </li>
   <li>
    Excellent communication and collaboration skills.
   </li>
   <li>
    Ability to work in a fast-paced, agile environment with cross-functional teams.
    <br/>
    <br/>
   </li>
  </ul>
 </ul>
 <strong>
  Preferred Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Experience with HL7, CDA, or other healthcare interoperability standards.
  </li>
  <li>
   Exposure to machine learning data pipelines or advanced analytics environments.
  </li>
  <li>
   Azure certifications (e.g., Azure Data Engineer Associate) are a plus
  </li>
 </ul>
</div>


Nuarsh TECHSERVE

<p><b>Job Title: Data Engineer</b></p><p><b>Experience: 2-4 years</b></p><p><b>Location: Indore, India</b></p><p><b>About the Role</b></p><p>This is a full-time on-site role for a Data Engineer located in Indore. The ideal candidate will have hands-on experience working with modern cloud data platforms such as Azure, AWS, Snowflake, Microsoft Fabric, and Databricks. You will be responsible for building and optimizing data pipelines, ensuring data quality, and supporting data-driven initiatives across the organization.</p><p><b>Key Responsibilities</b></p><ul><li>Design, build, and maintain scalable and reliable data pipelines and ETL/ELT processes.</li><li>Work with cloud technologies such as <b>Azure, AWS, Snowflake, Microsoft Fabric, and Databricks</b> to manage and optimize data workflows.</li><li>Collaborate with Data Leads, Analysts, and cross-functional teams to understand data needs and deliver efficient solutions.</li><li>Implement and maintain data models that support business intelligence, analytics, and reporting needs.</li><li>Ensure the accuracy, consistency, and reliability of data across all systems.</li><li>Troubleshoot data pipeline issues and optimize performance.</li><li>Participate in data architecture discussions and contribute to continuous improvement of data systems.</li><li>Maintain proper documentation of data workflows, models, and processes.</li></ul><p><b>Qualifications &amp; Skills</b></p><ul><li><b>2&ndash;4 years of hands-on experience</b> in Data Engineering or related fields.</li><li>Strong experience in <b>Data Modeling, Data Management, and ETL/ELT pipeline development</b>.</li><li>Solid understanding of <b>Data Analytics and Data Analysis</b> concepts.</li><li>Proficiency with one or more cloud platforms: <b>Azure, AWS, Snowflake, Microsoft Fabric, Databricks</b>.</li><li>Experience with SQL and at least one programming language such as Python or Scala.</li><li>Strong analytical, problem-solving, and troubleshooting skills.</li><li>Ability to work independently as well as collaboratively in a team environment.</li><li>Excellent communication skills and attention to detail.</li><li>Bachelor&rsquo;s degree in Computer Science, Information Technology, or a related field.</li><li>Experience in a consulting environment is a plus.</li></ul><p>Job Type: Full-time</p><p>Pay: &#8377;800,000.00 - &#8377;1,000,000.00 per year</p><p>Work Location: In person</p>

Technology Next

<p><b>Urgently hiring for Data Engineer (Python, SQL)</b></p><p><b>Experience Required: 9+ Years</b><br/><b>Work Mode: Hybrid</b><br/><b>Location: Bangalore</b></p><p><b>Job Description</b></p><p>We are seeking a highly skilled Lead Consultant &ndash; Data Engineer / Data Analyst with strong expertise in Python, SQL, and end-to-end data pipeline development. This role is ideal for professionals who excel in fast-paced, innovation-driven environments and enjoy solving complex data challenges using modern, scalable solutions.</p><p>You will design, build, and optimize robust data workflows, collaborate with cross-functional teams, and contribute to next-generation analytics and AI-driven initiatives.</p><p><b>Key Responsibilities</b></p><p>* Design, build, and maintain robust and scalable data pipelines using Python and SQL.<br/>* Write complex SQL queries and optimize existing database interactions.<br/>* Develop reusable scripts for extracting data from structured and semi-structured sources (CSV, JSON, Excel).<br/>* Integrate data from diverse sources including REST APIs and multiple relational databases.<br/>* Clean, transform, and prepare high-quality datasets for analytics, reporting, and modeling.<br/>* Collaborate closely with analysts, data scientists, and business stakeholders.<br/>* Document workflows, maintain best practices in coding standards, and use Git for version control.<br/>* Good to have experience with Power BI, Paginated Reports, and BI migration/enhancement projects.</p><p><b>Minimum Qualifications</b></p><p>* Expert-level Python for ETL, automation, and data transformation.<br/>* Strong proficiency in complex SQL, stored procedures, optimization, and working with relational databases (PostgreSQL, MySQL, SQL Server).<br/>* Experience with file-based data sources (CSV, JSON, Excel) and parsing/transformation techniques.<br/>* Ability to consume and integrate data from REST APIs.<br/>* Working knowledge of Git, basic shell scripting, and data validation techniques.<br/>* Preferred: Power BI development, migration, performance enhancement skills.</p><p>Job Type: Contractual / Temporary<br/>Contract length: 6-12 months</p><p>Pay: &#8377;70,000.00 - &#8377;90,000.00 per month</p><p>Experience:</p><ul><li>Python: 5 years (Required)</li><li>SQL: 5 years (Required)</li></ul><p>Work Location: In person</p>

Search 10000+ Data engineer jobs near you this November. New roles added regularly across leading industries. Apply today.

10000 data engineer jobs NEAR_ME

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer

Data Engineer