big data engineer

MetLife

<div><div><div><div><h3 class="jobSectionHeader"><b>General Information
</b></h3></div></div><div><div><div><div><div><b>Location</b></div><div>
Hyderabad, Telangana</div>
</div></div></div></div></div><div><div><div><div><div>Working Schedule
</div><div>Full-Time
</div></div><div><div>Work Arrangement
</div><div>Hybrid
</div></div><div><div>Relocation Assistance Available
</div><div>No
</div></div><div><div>Posted Date
</div><div>30-Jun-2026
</div></div><div><div>Job ID
</div><div>18747
</div></div></div></div></div><div><div><div><h3 class="jobSectionHeader"><b>Description and Requirements
</b></h3></div></div><div></div><div><div><div><div><div><div><div><div>Role Value Proposition</div><div> Data Engineer plays a critical role in data and analytics life cycle and significantly contributes to production grade data and analytics solutions. The role requires one to demonstrate Big Data, Engineering and Cloud expertise. This role mentors Data engineers. It is an individual contributor role, expected to solve wide ranging business problems,</div></div><br>
<div><div>
Experience</div><div> 8-10+ years of relevant experience</div></div><br>
<div><div>
Education</div><div> Bachelors degree in computer science, information technology or equivalent educational qualification</div></div><br>
<div><div>
Responsibilities</div><div><li>Design, build, and maintain robust ETL/ELT pipelines on cloud(Azure) or on-prem to collect, ingest and store large volumes of structured and unstructured data for batch/real time processing</li><br>
<li>Monitor, optimize, and troubleshoot data pipelines to ensure reliability, scalability, and performance</li><br>
<li>Ensure data processing, quality, security, and compliance guidelines, policies and standards are followed</li><br>
<li>Collaborate with multiple partners from Business, Technology, Operations and D&amp;A capabilities (Data Governance, Data Quality, Data Modeling, Data Architecture, Data science, DevOps, BI &amp; insights)
</li><br>
<li>Mentor Data Engineers</li><br>
<li>Independently lead design, solutioning &amp; estimations</li></div></div><br>
<div><div>
Technical Skills</div><div><li>SQL, Python/Scala</li><br>
<li>NoSql and distributed databases (Hbase, Cosmos DB)</li><br>
<li>ETL pipleine design and development; Solutioning and estimation</li><br>
<li>Big Data Frameworks : Apache Spark, Hadoop, Hive</li><br>
<li>Cloud platforms: Azure data factory, Eventhub, Azure functions, Synapse, Databricks</li><br>
<li>Datawarehouses, data marts, data lakes</li><br>
<li>Medallion architecture</li><br>
<li>Performance tuning, optimization, and data quality validation</li><br>
<li>Real-time and batch data processing , streaming pieplines with Spark</li><br>
<li>Communication skills, analytical skills, structured problem-solving skills, mentorship skills</li><br>
<li>Mentorship experience</li><br>
<li>Storytelling skills , Partner &amp; Stakeholder engagement experience</li></div></div><br>
<div><div>
Good To have</div><div><li>DevOps practices: Git, AzureDevops, CI/CD pipelines</li><br>
<li>Unix shell scripting, Kafka,MongoDB, Nifi</li><br>
<li>Exposure to Gen AI technology and tools</li><br>
<li>Banking Financial Services and Insurance domain knowledge</li></div></div><br>
</div></div>
</div></div></div></div></div><div><div><div><div><div><b>About MetLife</b><br>
<div></div>
<p>Recognized on Fortune magazine's list of the &quot;World's Most Admired Companies&quot; and Fortune World&rsquo;s 25 Best Workplaces&trade;, MetLife, through its subsidiaries and affiliates, is one of the world&rsquo;s leading financial services companies; providing insurance, annuities, employee benefits and asset management to individual and institutional customers. With operations in more than 40 markets, we hold leading positions in the United States, Latin America, Asia, Europe, and the Middle East.
</p><p></p><p>Our purpose is simple - to help our colleagues, customers, communities, and the world at large create a more confident future. United by purpose and guided by our core values - Win Together, Do the Right Thing, Deliver Impact Over Activity, and Think Ahead - we&rsquo;re inspired to transform the next century in financial services. At MetLife, it&rsquo;s #AllTogetherPossible. Join us!</p></div></div></div></div></div></div>

Senior Big Data Engineer - I

Maaloomatiia Academy

<div>
 <strong>
  Job Title
  <br/>
  <br/>
 </strong>
 <strong>
  Cloudera Data Engineer
  <br/>
  <br/>
 </strong>
 <strong>
  Job Scope
  <br/>
  <br/>
 </strong>
 We are seeking an experienced
 <strong>
  Cloudera Data Engineer
 </strong>
 with strong hands-on expertise in
 <strong>
  Cloudera CDP, Hadoop, Spark, Hive, and HDFS
 </strong>
 . The ideal candidate will be responsible for building and maintaining big data pipelines, optimizing data workflows, and supporting production environments to ensure reliability and performance.
 <br/>
 <br/>
 <strong>
  Main Duties And Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Design, build, and maintain big data pipelines using Cloudera CDP components.
  </li>
  <li>
   Develop and optimize ETL/ELT workflows using Spark, Hive, and Hadoop ecosystems.
  </li>
  <li>
   Manage and maintain HDFS storage, ensuring high availability and performance.
  </li>
  <li>
   Perform production support activities, including issue resolution, monitoring, and performance tuning.
  </li>
  <li>
   Work with data scientists, analysts, and other engineering teams to support data-driven initiatives.
  </li>
  <li>
   Ensure data governance, security, and compliance across big data environments.
  </li>
  <li>
   Troubleshoot cluster-level issues related to resource utilization, scheduling, and configuration.
  </li>
  <li>
   Participate in solution design, architecture discussions, and technical documentation.
   <br/>
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Position Requirements
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Strong hands-on experience with Cloudera CDP.
  </li>
  <li>
   Proficiency in Hadoop ecosystem: HDFS, YARN, MapReduce, Hive, Impala.
  </li>
  <li>
   Strong experience working with Apache Spark (PySpark or Scala preferred).
  </li>
  <li>
   Solid understanding of distributed systems and big data architecture.
  </li>
  <li>
   Experience with production support and troubleshooting in big data environments.
  </li>
  <li>
   Strong SQL skills and experience optimizing large-scale queries.
  </li>
  <li>
   Familiarity with Linux, shell scripting, and automation tools.
  </li>
  <li>
   Experience with version control (Git) and CI/CD workflows is a plus.
   <br/>
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Education
  <br/>
  <br/>
 </strong>
 Bachelor's degree in Computer Science or any other related field
 <br/>
 <br/>
 <strong>
  Experience
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Experience with cloud platforms (AWS, Azure, GCP) for big data workloads.
  </li>
  <li>
   Exposure to Kafka, NiFi, Oozie, or similar data orchestration tools.
  </li>
  <li>
   Knowledge of Docker, Kubernetes, or containerized environments.
  </li>
  <li>
   Understanding of security, encryption, and data governance concepts.
  </li>
 </ul>
</div>


Cloudera Data Engineer

Nous Infosystems

<div>
 <br/>
 <p>
  <strong>
   Role: Data Engineer
  </strong>
 </p>
 <p>
  <strong>
   Location: London, UK (2 days/week onsite)
  </strong>
 </p>
 <p>
  <br/>
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   <br/>
   <br/>
   Role Overvie
  </strong>
 </p>
 <p>
  w:Looking for a Data Engineer to build and maintain scalable data pipelines and support large-scale analytics systems. You&rsquo;ll work with high-volume data using modern orchestration frameworks and big data technologie
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   s.
   <br/>
   Key Responsibiliti
  </strong>
 </p>
 <ul>
  <li>
   es:Build and maintain data pipelines using Pyt
  </li>
  <li>
   honDevelop and optimize ETL workfl
  </li>
  <li>
   owsWrite complex SQL queries (Presto / Spark S
  </li>
  <li>
   QL)Work with Spark for large-scale data process
  </li>
  <li>
   ingEnsure data quality, monitoring, and reliabil
  </li>
  <li>
   ityCollaborate with cross-functional teams (Data Science, Product, Analyti
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   cs)
   <br/>
   Tech S
  </strong>
 </p>
 <ul>
  <li>
   tackLanguages: Python,
  </li>
  <li>
   SQLTools: Dataswarm (Airflow-like), Spark, Presto,
  </li>
  <li>
   HiveOthers: Data catalog, dashboarding, lineage &amp; monitoring t
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   ools
   <br/>
   Required S
  </strong>
 </p>
 <ul>
  <li>
   killsStrong Python + SQL expe
  </li>
  <li>
   rtiseExperience with data pipeline/orchestration tools (Airflow or sim
  </li>
  <li>
   ilar)Hands-on with Spark / big data proce
  </li>
  <li>
   ssingGood understanding of data warehousing con
  </li>
  <li>
   ceptsExperience working with large dat
  </li>
  <li>
   asetsExperience in FAANG-scale environ
  </li>
  <li>
   mentsExposure to data lineage / metrics frame
  </li>
  <li>
   worksFamiliarity with real-time monitoring
  </li>
 </ul>
 <p>
  <br/>
 </p>
 tools
</div>


Data Engineer

Information Tech Consultants

<div>
 <p>
  <strong>
   We&rsquo;re Hiring: Big Data Developer | Start Date: 14th July 2026
  </strong>
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Job Title:
  </strong>
  Big Data Developer (Python)
 </p>
 <p>
  <strong>
   Location:
  </strong>
  Camden Town, London
 </p>
 <p>
  <strong>
   Job Type:
  </strong>
  Full-time
 </p>
 <p>
  <strong>
   Salary:
  </strong>
  &pound;35,000 &ndash; &pound;45,000 per year
 </p>
 <p>
  <strong>
   Experience:
  </strong>
  2&ndash;5 years
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Job Description
  </strong>
 </p>
 <p>
  We are urgently hiring a Big Data Developer / Big Data Engineer with strong experience in Python. You will be responsible for building scalable data pipelines, ETL processes, and distributed data systems in a fast-paced environment.
 </p>
 <p>
  <br/>
 </p>
 <p>
  This is an excellent opportunity for candidates with a background in software development and data engineering, especially within large-scale data environments.
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Key Responsibilities
  </strong>
 </p>
 <ul>
  <li>
   Develop and maintain Big Data applications
  </li>
  <li>
   Build, optimize, and manage ETL pipelines and data workflows
  </li>
  <li>
   Work with large-scale structured and unstructured datasets
  </li>
  <li>
   Perform code reviews, debugging, and performance optimization
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Required Skills
  </strong>
 </p>
 <ul>
  <li>
   2&ndash;5 years of experience as a Big Data Developer / Python Developer / Software Engineer
  </li>
  <li>
   Strong programming skills in Python
  </li>
  <li>
   Basic knowledge of Scala, Java, Hadoop
  </li>
  <li>
   Good knowledge of SQL and database concepts
  </li>
  <li>
   Experience working in Linux/Unix environments
  </li>
  <li>
   Understanding of data structures, algorithms, and system design
  </li>
  <li>
   Strong analytical and problem-solving skills
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Education:
  </strong>
  Bachelor&rsquo;s in Computer Science, IT, Software Engineering, or a related field
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Additional Information
  </strong>
 </p>
 <ul>
  <li>
   Must have valid UK work authorization
  </li>
  <li>
   Willingness to travel within the UK is required
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Why Join Us?
  </strong>
 </p>
 <p>
  Work on innovative and real-world technology projects
 </p>
 <p>
  Gain exposure to modern tools and development practices
 </p>
 <p>
  Be part of a supportive and collaborative team
 </p>
 <p>
  Excellent opportunities for learning, growth, and career progression
 </p>
 <p>
  <br/>
 </p>
 <p>
  Apply today if you're available to start on
  <strong>
   14th July
  </strong>
  and are looking for your next exciting opportunity.
 </p>
</div>


Big Data Developer

Contango

<div>
 <strong>
  Contract duration:
 </strong>
 6 months (with a potential extension)
 <br/>
 <br/>
 <strong>
  Engagement Type:
 </strong>
 Full-time
 <br/>
 <br/>
 <strong>
  Start date:
 </strong>
 July 2026
 <br/>
 <br/>
 <strong>
  Location:
 </strong>
 Abu Dhabi (on-site)
 <br/>
 <br/>
 <strong>
  Role Overview
  <br/>
  <br/>
 </strong>
 We are seeking an experienced Data Engineer to design, build, and operate scalable, secure, and reliable data pipelines and platforms that power analytics and AI solutions for companies in the UAE. The role is hands-on and client-facing, owning data solutions end-to-end &mdash; from architecture and development to production and hypercare &ndash; on cloud and on-premise.
 <br/>
 <br/>
 <strong>
  Key Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Design, build, and maintain scalable batch and streaming data pipelines (ETL/ELT) that move data reliably from source systems to analytics and AI workloads.
  </li>
  <li>
   Build and optimize data models, databases, data warehouses, and lakehouse architectures (e.g., Databricks, Microsoft SQL Server) for performance and cost.
  </li>
  <li>
   Develop and tune distributed processing jobs using Spark/PySpark, including cluster management and optimization.
  </li>
  <li>
   Ensure data quality and data governance across the full data lifecycle (collection, access, storage, transformation, retention).
  </li>
  <li>
   Productionize pipelines with CI/CD, orchestration (e.g., Airflow, Azure Data Factory), monitoring, and alerting.
  </li>
  <li>
   Build the data foundation for analytics and AI use cases, collaborating closely with Data Scientists and AI Engineers (e.g., feature engineering, RAG data processing).
  </li>
  <li>
   Translate client business requirements into data architecture decisions and communicate trade-offs to technical and non-technical stakeholders.
  </li>
  <li>
   Maintain technical documentation, data lineage, and delivery artifacts across Design, Build, UAT, Production &amp; Hypercare.
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Candidate Requirements
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Demonstrated experience delivering AI solutions in an enterprise or client-facing/consulting environment.
  </li>
  <li>
   Prior experience at management consulting firms and/or Big Tech is an advantage.
  </li>
  <li>
   Strong programming in Python and SQL with solid software engineering fundamentals.
  </li>
  <li>
   Hands-on experience with Spark/PySpark and big-data processing at scale.
  </li>
  <li>
   Proven experience building production ETL/ELT pipelines.
  </li>
  <li>
   Practical knowledge of a major cloud data stack &mdash; Azure, AWS, or GCP.
  </li>
  <li>
   Experience with a modern data platform (Databricks, Snowflake, or equivalent).
  </li>
  <li>
   Familiarity with orchestration, CI/CD, and DevOps (e.g., Airflow, Git, Docker).
  </li>
  <li>
   Understanding of data warehousing concepts, data modeling, data governance, and security best practices.
  </li>
  <li>
   Experience with dashboard or frontend engineering is a plus.
  </li>
  <li>
   Strong collaboration and communication skills, with the ability to work directly with clients.
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Bachelor's or Master&rsquo;s degree in Computer Science, Engineering, Information Technology, or a related field.
  </li>
  <li>
   3&ndash;6 years of experience in data engineering, building and operating data pipelines at scale.
  </li>
  <li>
   Preferred certifications: cloud data certifications such as Azure Data Engineer, AWS Data Engineer, or Databricks Data Engineer.
  </li>
 </ul>
</div>


Data Engineer (6 months contract)

<div><p><b>Contract duration: </b>6 months (with a potential extension)<br>
<b>Engagement Type:</b> Full-time<br>
<b>Start date: </b>July 2026<br>
<b>Location:</b> Abu Dhabi (on-site)</p><p></p><p><b>
Role Overview</b></p><p>
We are seeking an experienced Data Engineer to design, build, and operate scalable, secure, and reliable data pipelines and platforms that power analytics and AI solutions for companies in the UAE. The role is hands-on and client-facing, owning data solutions end-to-end &mdash; from architecture and development to production and hypercare &ndash; on cloud and on-premise.<br>
<br>
<b>Key Responsibilities</b></p><ul><li><p>
Design, build, and maintain scalable batch and streaming data pipelines (ETL/ELT) that move data reliably from source systems to analytics and AI workloads.</p></li><li><p>
Build and optimize data models, databases, data warehouses, and lakehouse architectures (e.g., Databricks, Microsoft SQL Server) for performance and cost.</p></li><li><p>
Develop and tune distributed processing jobs using Spark/PySpark, including cluster management and optimization.</p></li><li><p>
Ensure data quality and data governance across the full data lifecycle (collection, access, storage, transformation, retention).</p></li><li><p>
Productionize pipelines with CI/CD, orchestration (e.g., Airflow, Azure Data Factory), monitoring, and alerting.</p></li><li><p>
Build the data foundation for analytics and AI use cases, collaborating closely with Data Scientists and AI Engineers (e.g., feature engineering, RAG data processing).</p></li><li><p>
Translate client business requirements into data architecture decisions and communicate trade-offs to technical and non-technical stakeholders.</p></li><li><p>
Maintain technical documentation, data lineage, and delivery artifacts across Design, Build, UAT, Production &amp; Hypercare.</p></li></ul><p><b><br>
Candidate Requirements</b></p><ul><li><p>
Demonstrated experience delivering AI solutions in an enterprise or client-facing/consulting environment.</p></li><li><p>
Prior experience at management consulting firms and/or Big Tech is an advantage.</p></li><li><p>
Strong programming in Python and SQL with solid software engineering fundamentals.</p></li><li><p>
Hands-on experience with Spark/PySpark and big-data processing at scale.</p></li><li><p>
Proven experience building production ETL/ELT pipelines.</p></li><li><p>
Practical knowledge of a major cloud data stack &mdash; Azure, AWS, or GCP.</p></li><li><p>
Experience with a modern data platform (Databricks, Snowflake, or equivalent).</p></li><li><p>
Familiarity with orchestration, CI/CD, and DevOps (e.g., Airflow, Git, Docker).</p></li><li><p>
Understanding of data warehousing concepts, data modeling, data governance, and security best practices.</p></li><li><p>
Experience with dashboard or frontend engineering is a plus.</p></li><li><p>
Strong collaboration and communication skills, with the ability to work directly with clients.</p></li></ul><p><b><br>
Qualifications</b></p><ul><li><p>
Bachelor's or Master&rsquo;s degree in Computer Science, Engineering, Information Technology, or a related field.</p></li><li><p>
3&ndash;6 years of experience in data engineering, building and operating data pipelines at scale.</p></li><li><p>
Preferred certifications: cloud data certifications such as Azure Data Engineer, AWS Data Engineer, or Databricks Data Engineer.</p></li></ul></div>

APLOMB Technologies

<p><b>Job Summary</b>  <br/>We are seeking a highly skilled and motivated Senior Data Engineer to lead the design, development, and maintenance of our enterprise data infrastructure. In this dynamic role, you will harness the power of big data technologies and cloud platforms to enable insightful analytics and data-driven decision-making. Your expertise will drive the creation of scalable data pipelines, optimize data storage solutions, and support advanced analytics initiatives. Join us to empower our organization with innovative data solutions that fuel growth and strategic insights.</p><p><b>Responsibilities</b></p><ul><li>Design, develop, and optimize robust ETL (Extract, Transform, Load) processes to integrate data from diverse sources such as AWS, Azure Data Lake, Hadoop, and on-premises systems.  </li><li>Build scalable data pipelines utilizing Apache Spark, Hadoop ecosystem components (e.g., Hive), and Python scripting to process large volumes of structured and unstructured data efficiently.  </li><li>Develop and maintain data warehouses leveraging SQL Server, Oracle, and other relational databases; ensure high performance and security standards.  </li><li>Collaborate with cross-functional teams to implement data models, database designs, and linked data architectures that support analytics and reporting needs.  </li><li>Utilize tools like Talend, Informatica, and Shell Scripting to automate workflows and streamline data processing tasks.  </li><li>Integrate RESTful APIs for seamless data exchange between systems; support model training activities for machine learning initiatives.  </li><li>Work within Agile methodologies to deliver continuous improvements in data infrastructure while maintaining documentation for compliance and knowledge sharing.</li></ul><p><b>Requirements</b></p><ul><li>Proven experience as a Data Engineer or similar role with a focus on big data technologies such as Hadoop, Spark, Hive, and Azure Data Lake.  </li><li>Strong proficiency in SQL (including Microsoft SQL Server and Oracle), Python, Java, Bash (Unix shell), Shell Scripting, and VBA for automation and analysis tasks.  </li><li>Hands-on experience designing scalable data warehouses and implementing ETL pipelines using tools like Talend or Informatica.  </li><li>Familiarity with cloud platforms such as AWS and Azure for deploying large-scale data solutions.  </li><li>Knowledge of database design principles, linked data concepts, RESTful API integration, and model training techniques for machine learning applications.  </li><li>Excellent analysis skills with the ability to interpret complex datasets; experience with Looker or similar BI tools is a plus.  </li><li>Strong understanding of Agile development practices to facilitate iterative project delivery in fast-paced environments. Join us as a Senior Data Engineer to shape the future of our organization&rsquo;s data landscape! Bring your passion for innovation, technical expertise in big data ecosystems, cloud platforms, and analytics to help us unlock valuable insights that drive strategic success. We&rsquo;re committed to fostering an inclusive environment where your skills can thrive&mdash;empowering you every step of the way!</li></ul><p>Pay: $50.00 - $55.00 per hour</p><p>Work Location: In person</p>

Senior Data Engineer

Search 10000+ Big data engineer jobs near you this July. New roles added regularly across leading industries. Apply today.

10000 big data engineer jobs Near Me

Senior Big Data Engineer - I

Cloudera Data Engineer

Data Engineer

Big Data Developer

Data Engineer (6 months contract)

Data Engineer (6 months contract)

Senior Data Engineer