hadoop engineer

TECHSTRA SOLUTIONS

<div><div><div><p><b>Role Overview</b><br>
We are seeking an experienced Senior Data Engineer with strong Scala development expertise to build and support large-scale data processing solutions. This role will focus on developing high-performance data pipelines and applications using Spark and the Hadoop ecosystem, supporting critical enterprise data platforms. The ideal candidate brings strong problem-solving skills, a deep understanding of distributed data processing, and an engineering mindset that considers the full lifecycle of code&mdash;from design and development through deployment, performance optimization, and long-term production stability.</p></div><div><h3 class="jobSectionHeader"><b>
Key Responsibilities</b></h3><ul><li><p>
Design, develop, and maintain scalable data pipelines and distributed data processing applications using Scala and Spark.</p></li><li><p>
Build and optimize large-scale batch and data processing frameworks within the Hadoop ecosystem.</p></li><li><p>
Write efficient and maintainable Scala-based applications for enterprise data platforms.</p></li><li><p>
Develop and maintain SQL-based queries and transformations for data processing and validation.</p></li><li><p>
Collaborate with data architects, engineers, and business stakeholders to translate business requirements into scalable data solutions.</p></li><li><p>
Ensure data pipeline performance, reliability, and stability in production environments.</p></li><li><p>
Troubleshoot and resolve performance issues across distributed systems.</p></li><li><p>
Contribute to best practices around data engineering, code quality, testing, and operational support.</p></li><li><p>
Support production environments and continuously improve monitoring, resiliency, and stability of data systems.</p></li></ul><h3 class="jobSectionHeader"><b>
Required Qualifications/Skills</b></h3><ul><li>
Bachelor&rsquo;s degree in Computer Science, Information Technology, or related field.</li><li><p>
8&ndash;10 years of experience in data engineering or software development roles.</p></li><li><p>
Strong Scala development experience (primary programming language).</p></li><li><p>
Hands-on experience with Apache Spark for distributed data processing.</p></li><li><p>
Experience working within the Hadoop ecosystem (HDFS, Hive, YARN, or related technologies).</p></li><li><p>
Basic to intermediate SQL skills for querying and manipulating data.</p></li><li><p>
Strong problem-solving and analytical skills.</p></li><li><p>
Demonstrated ability to design solutions while considering the entire software lifecycle, including performance, monitoring, and production stability.</p></li></ul><h3 class="jobSectionHeader"><b>
Preferred Qualifications</b></h3><ul><li><p>
Experience working in Corporate &amp; Investment Banking (C&amp;IB) environments.</p></li><li><p>
Familiarity with large-scale enterprise data platforms handling high data volumes.</p></li><li><p>
Experience with data pipeline optimization and performance tuning.</p></li><li><p>
Exposure to modern data platforms, streaming frameworks, or cloud-based data services.</p></li><li><p>
Experience working in Agile development environments.</p></li></ul></div><div><div><b>
Location: </b><b>Pittsburgh/Cleveland/Dallas</b><br>
<br>
<b>About Techstra Solutions</b><br>
Techstra Solutions is a certified woman-owned (WBENC) management consulting firm specializing in strategy, technology, and implementation services for large organizations undergoing digital and talent transformation. Our experienced team partners with clients to co-create innovative solutions in applications, data, AI, and automation that accelerate measurable, sustainable change. From advisory consulting through technical execution, we are dedicated to driving world-class business solutions that fit your strategic requirements and deliver results. For more information: www.techstrasolutions.com</div></div></div><p><b>
Equal Employment Opportunity Statement</b></p>
<p>Techstra Solutions is an equal opportunity employer. The Company makes its decisions on merit, and its policy of equal opportunity prohibits discrimination in all phases of the employment process, including, but not limited to, recruitment, hiring, promotion, selection, transfer, demotion, layoff, termination, compensation, benefits, and other terms and conditions of employment. The policy of equal opportunity applies without regard to race, color, creed, religion, gender, sexual orientation, gender identification, pregnancy, marital status, national origin, ancestry, age, disability that can reasonably be accommodated without undue hardship, military status, veteran status, genetic predisposition or carrier status, alienage or citizenship, domestic partnership status, arrest or conviction record, status as a victim of domestic violence, or any other protected categories under federal, state, or local law. The Company also prohibits discrimination or harassment based upon the perception that a person has, or is associated with a person who has, any of these characteristics.</p><p>
</p><p>obSkI7eu1W</p></div>

Senior Data Engineer (Scala Development)

Systems Limited

<div>
 <p>
  <strong>
   Job Description &ndash; Kafka Data Engineer
  </strong>
 </p>
 <p>
  <br/>
 </p>
 <p>
  Data Engineer to build and manage data pipelines that support batch and streaming data solutions. The role requires expertise in creating seamless data flows across platforms like Data Lake/Lakehouse in Cloudera, Azure Databricks, Kafka for both batch and stream data pipelines etc.
 </p>
 <p>
  <strong>
   <u>
    Responsibilities
   </u>
  </strong>
 </p>
 <ul>
  <li>
   Strong experience in develop, test, and maintain data pipelines (batch &amp; stream) using Cloudera, Spark, Kafka and Azure services like ADF, Cosmos DB, Databricks, NoSQL DB/ Mongo DB etc.
  </li>
  <li>
   Strong programming skills in spark, python or scala &amp; SQL.
  </li>
  <li>
   Optimize data pipelines to improve speed, performance, and reliability, ensuring that data is available for data consumers as required.
  </li>
  <li>
   Create ETL pipelines for downstream consumers by transform data as per business logic.
  </li>
  <li>
   Work closely with Data Architects and Data Analysts to align data solutions with business needs and ensure the accuracy and accessibility of data.
  </li>
  <li>
   Implement data validation checks and error handling processes to maintain high data quality and consistency across data pipelines.
  </li>
  <li>
   Strong analytical and problem solving skills, with a focus on optimizing data flows and addressing impacts in the data pipeline.
  </li>
  <li>
   Document data pipeline processes, ensuring they are clear and can be maintained or adapted by other team members.
  </li>
  <li>
   Strong communication skills to collaborate across teams.
  </li>
 </ul>
 <p>
  <strong>
   <u>
    Qualifications
   </u>
  </strong>
 </p>
 <ul>
  <li>
   8+ years of IT experience with at least 5+ years in data engineering and cloud-based data platforms.
  </li>
  <li>
   Strong experience with Cloudera/any Data Lake, Confluent/Apache Kafka, and Azure Data Services (ADF, Databricks, Cosmos DB).
  </li>
  <li>
   Deep knowledge of NoSQL databases (Cosmos DB, MongoDB) and data modeling for performance and scalability.
  </li>
  <li>
   Proven expertise in designing and implementing batch and streaming data pipelines using Databricks, Spark, or Kafka.
  </li>
  <li>
   Experience in creating scalable, reliable, and high-performance data solutions with robust data governance policies.
  </li>
  <li>
   Strong collaboration skills to work with stakeholders, mentor junior Data Engineers, and translate business needs into actionable solutions.
  </li>
  <li>
   Bachelor&rsquo;s or master&rsquo;s degree in computer science, IT, or a related field.
  </li>
 </ul>
 <p>
 </p>
</div>


Kafka Data Engineer

Tata Consultancy Services (TCS)

<div></div><div><div><div><div><div>Key Responsibilities:
</div><ul><li>Design, build, and maintain data pipelines across on-prem Hadoop and AWS
</li></ul><ul><li>Develop and maintain Java applications, utilities, and data processing libraries
</li></ul><ul><li>Manage and enhance internal Java libraries used for ingestion, validation, and transformation
</li></ul><ul><li>Migrate and sync data from on-prem HDFS to AWS S3
</li></ul><ul><li>Develop and maintain Airflow DAGs for orchestration and scheduling
</li></ul><ul><li>Work with Kafka-based streaming pipelines for real-time/near-real-time ingestion
</li></ul><ul><li>Build and optimize Spark / PySpark jobs for large-scale data processing
</li></ul><ul><li>Use Hive, Presto/Trino, and Athena for querying and validation
</li></ul><ul><li>Implement data quality checks, monitoring, and alerting
</li></ul><ul><li>Support Iceberg tables and AWS external tables
</li></ul><ul><li>Troubleshoot production issues and ensure SLA compliance
</li></ul><ul><li>Collaborate with platform, analytics, and observability teams</li></ul><br>
<div><br>
</div><div>Technical Skills Required:
</div><div>Java (Development, maintenance, build tools like Gradle)
</div><div>AWS (S3, Glue, EMR, Athena, EKS basics)
</div><div>Hadoop/HDFS, Hive
</div><div>Apache Kafka (producers/consumers, topics, streaming ingestion)
</div><div>Apache Spark / PySpark (batch + streaming processing)
</div><div>Apache Airflow (DAG development and maintenance)
</div><div>Python
</div><div>Git and CI/CD workflows
</div><div>Observability tools (Prometheus/Grafana)
</div><div>SQL</div><br>
<div><br>
</div><div>Location: Austin, TX/ Sunnyvale, CA</div><br>
<div><br>
</div><div>Salary Range:$70,000-$135,000 a year
</div><div>#LI-AS3
</div></div></div></div><div><div><div><b>Location</b></div>
<div>Austin, TX</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>401474</div>
</div><div><div><b>Desired Skills</b></div>
<div>AWS | Hadoop | Java | Kafka</div>
</div><div><div><b>Salary Range</b></div>
<div>$70,000-$135,000 a year</div></div></div></div><div></div>

Data Engineer

Müller's Solutions

<div>
 M&uuml;ller's Solutions is looking for an experienced Big Data Engineer to join our innovative team. In this role, you will be responsible for designing and developing scalable big data solutions that enable advanced analytics and insights across the organization. You will work with large data sets and utilize cutting-edge technologies to ensure data processing is efficient and reliable.
 <br/>
 <br/>
 <strong>
  Key Responsibilities:
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Design, build, and maintain scalable data pipelines using big data technologies such as Apache Spark, Hadoop, and Kafka
  </li>
  <li>
   Collaborate with data scientists, analysts, and IT teams to identify data requirements and deliver solutions that meet business needs
  </li>
  <li>
   Ensure high data quality and integrity by implementing robust data validation and testing processes
  </li>
  <li>
   Optimize data storage solutions for performance and cost-efficiency across cloud and on-premises environments
  </li>
  <li>
   Monitor and troubleshoot data processing workflows to ensure timely data delivery
  </li>
  <li>
   Document data architectures, processes, and workflows for knowledge sharing and compliance
  </li>
  <li>
   Stay ahead of industry trends and best practices in big data technologies and methodologies
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Requirements
  <br/>
  <br/>
 </strong>
 <strong>
  Requirements:
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Bachelor's degree in Computer Science, Information Technology, or a related field
  </li>
  <li>
   3+ years of experience as a Big Data Engineer or in a similar data engineering role
  </li>
  <li>
   Proficiency in big data technologies such as Apache Spark, Hadoop, and Apache Kafka
  </li>
  <li>
   Strong coding skills in programming languages like Java, Scala, or Python
  </li>
  <li>
   Familiarity with data processing frameworks and ETL tools
  </li>
  <li>
   Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and data storage solutions (e.g., HDFS, S3, BigQuery)
  </li>
  <li>
   Analytical mindset and excellent problem-solving abilities
  </li>
  <li>
   Strong communication skills to collaborate effectively with technical and non-technical stakeholders
  </li>
  <li>
   A self-motivated and proactive approach to work, with the ability to manage multiple tasks and deadlines
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Preferred Qualifications:
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Experience with machine learning frameworks and algorithms is a plus
  </li>
  <li>
   Knowledge of data governance and security best practices
  </li>
  <li>
   Familiarity with containerization technologies and orchestration tools like Docker and Kubernetes
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Benefits
  <br/>
  <br/>
 </strong>
 1- Attractive Package.
 <br/>
 <br/>
 2- Family Benefits.
 <br/>
 <br/>
 3- Visa.
 <br/>
 <br/>
 4-Air Tickets.
</div>


Big Data Engineer

Muller's Solutions

<div><p>M&uuml;ller&rsquo;s Solutions is looking for an experienced Big Data Engineer to join our innovative team. In this role, you will be responsible for designing and developing scalable big data solutions that enable advanced analytics and insights across the organization. You will work with large data sets and utilize cutting-edge technologies to ensure data processing is efficient and reliable.</p>
Key Responsibilities:<ul>
<li>Design, build, and maintain scalable data pipelines using big data technologies such as Apache Spark, Hadoop, and Kafka.</li>
<li>Collaborate with data scientists, analysts, and IT teams to identify data requirements and deliver solutions that meet business needs.</li>
<li>Ensure high data quality and integrity by implementing robust data validation and testing processes.</li>
<li>Optimize data storage solutions for performance and cost-efficiency across cloud and on-premises environments.</li>
<li>Monitor and troubleshoot data processing workflows to ensure timely data delivery.</li>
<li>Document data architectures, processes, and workflows for knowledge sharing and compliance.</li>
<li>Stay ahead of industry trends and best practices in big data technologies and methodologies.</li><br>
</ul><br>
<br>
<b>Requirements</b><p><b><br>
Requirements:</b></p><ul>
<li>Bachelor's degree in Computer Science, Information Technology, or a related field.</li>
<li>3+ years of experience as a Big Data Engineer or in a similar data engineering role.</li>
<li>Proficiency in big data technologies such as Apache Spark, Hadoop, and Apache Kafka.</li>
<li>Strong coding skills in programming languages like Java, Scala, or Python.</li>
<li>Familiarity with data processing frameworks and ETL tools.</li>
<li>Experience with cloud platforms (e.g., AWS, Azure, Google Cloud) and data storage solutions (e.g., HDFS, S3, BigQuery).</li>
<li>Analytical mindset and excellent problem-solving abilities.</li>
<li>Strong communication skills to collaborate effectively with technical and non-technical stakeholders.</li>
<li>A self-motivated and proactive approach to work, with the ability to manage multiple tasks and deadlines.</li>
</ul><p><b>Preferred Qualifications:</b></p><ul>
<li>Experience with machine learning frameworks and algorithms is a plus.</li>
<li>Knowledge of data governance and security best practices.</li>
<li>Familiarity with containerization technologies and orchestration tools like Docker and Kubernetes.</li><br>
</ul><br>
<br>
<b>Benefits</b><p><br>
1- Attractive Package.
</p><p>2- Family Benefits.
</p><p>3- Visa.</p><p>
4-Air Tickets.</p></div><p></p>

Search 5992+ Hadoop engineer jobs near you this March. New roles added regularly across leading industries. Apply today.

5992 hadoop engineer jobs Near Me

Senior Data Engineer (Scala Development)

Kafka Data Engineer

Data Engineer

Big Data Engineer

Big Data Engineer

Big Data Engineer

Big Data Engineer