hadoop developer

Recruitment Hub 365

<p><b>Job Title: Big Data Hadoop Developer</b><br/><b>Location: Hyderabad</b><br/><b>Work mode: 5 Days - (WFO)</b><br/><b>Experience Required: 4+ years</b><br/><b>Key Skills: Big Data, Hadoop, Spark, Hive</b></p><p><b>Job Overview</b><br/>We are looking for an experienced Big Data Hadoop Developer to design, build, and optimize large-scale data processing systems. The ideal candidate should have strong expertise in the Hadoop ecosystem, hands-on experience with Spark and Hive, and the ability to develop efficient ETL pipelines.</p><p><b>Key Responsibilities</b></p><ul><li>Design, develop, and maintain scalable big data systems and data pipelines.</li><li>Implement data processing frameworks using <b>Hadoop, Spark, and Hive</b>.</li><li>Develop and manage ETL workflows to ensure data accuracy, consistency, and availability.</li><li>Collaborate with data architects, analysts, and business teams to translate requirements into robust big data solutions.</li><li>Perform data validation, quality checks, and issue resolution for data pipelines.</li><li>Optimize data storage, query performance, and cluster utilization in the Hadoop ecosystem.</li><li>Ensure compliance with security, governance, and data management standards.</li></ul><p><b>Required Skills &amp; Qualifications</b></p><ul><li>4+ years of hands-on experience in <b>Big Data technologies</b>.</li><li>Strong understanding of the <b>Hadoop ecosystem</b> (HDFS, MapReduce, YARN).</li><li>Proficiency in <b>Apache Spark</b> (batch and streaming) and <b>Hive</b>.</li><li>Experience in building and maintaining data pipelines and ETL processes.</li><li>Strong knowledge of data optimization, partitioning, and performance tuning.</li><li>Familiarity with NoSQL databases such as <b>HBase, Cassandra, or MongoDB</b> is an advantage.</li><li>Experience with programming/scripting languages: <b>Java, Scala, Python, or Shell</b>.</li><li>Strong analytical and problem-solving skills.</li></ul><p>Job Type: Full-time</p><p>Pay: &#8377;1,500,000.00 - &#8377;2,400,000.00 per year</p><p>Application Question(s):</p><ul><li>How many years of experience you have with Big Data, Hadoop, Spark, Hive?</li><li>Mention your Notice Period, Current CTC and ECTC.</li><li>How many years of experience you have with Big Data / Hadoop Developer?</li></ul><p>Work Location: In person</p>

Big Data/Hadoop Developer

CRUTZ LEELA ENTERPRISES

<p>Sr. Spark Big Data Developer</p><p>PFB the detailed JD:</p><p>&#65533;&#65533; Experience: 5 to 7+ years (Hands-on)</p><p>&#65533;&#65533; Notice Period: 0-30 Days</p><p>Must Have:</p><p>&#61623; Proficiency in at least one of the following programming languages:</p><p>Java/Scala/Python</p><p>&#61623; Good understanding of SQL</p><p>&#61623; Experience of development and deployment of at least one end-to-end data</p><p>storage/processing pipeline</p><p>&#61623; Strong Experience in Spark development with batch and streaming</p><p>&#61623; Intermediate level expertise in HDFS and Hive</p><p>&#61623; Experience with Pyspark and Data Engineering</p><p>&#61623; ETL implementation and migration to spark</p><p>&#61623; Experience of working with Hadoop cluster</p><p>&#61623; Python, PySpark, Data Bricks developer with knowledge of cloud</p><p>&#61623; Experience with Kafka and Spark streaming (Dstream and Structured Streaming)</p><p>&#61623; Experience with using Jupyter notebooks or any other developer tool</p><p>&#61623; Experience with Airflow or other workflow engines</p><p>&#61623; Good communication skills and logical skills</p><p>Good to Have Skills:</p><p>&#61623; Prior experience of writing Spark jobs using Java is highly appreciated</p><p>&#61623; Prior experience of working with Cloudera Data Platform (CDP)</p><p>&#61623; Hands-on experience with NoSQL databases like HBase, Cassandra,</p><p>Elasticsearch, etc.</p><p>&#61623; Experience of using maven and git</p><p>&#61623; Agile scrum methodologies</p><p>&#61623; Flink and Kudu streaming</p><p>&#61623; Automation of workflows CI/CD</p><p>&#61623; Nifi streaming and transformation</p><p>Job Types: Full-time, Permanent</p><p>Pay: &#8377;500,000.00 - &#8377;1,500,000.00 per year</p><p>Benefits:</p><ul><li>Provident Fund</li></ul><p>Work Location: In person</p>

Sr. Spark Big Data Developer

KMCCorp India

<div><div><h3 class="jobSectionHeader"><b>Job Information</b></h3>
<ul><div><p>Date Opened</p>
<div><b>11/19/2025</b><b>
</b></div></div><div><p>Industry</p>
<div><b>Information Technology</b><b>
</b></div></div><div><p>Job Type</p>
<div><b>Full time</b><b>
</b></div></div><div><p>Work Experience</p>
<div><b>5+ years</b><b>
</b></div></div><div><p>City</p>
<div><b>Hyderabad</b><b>
</b></div></div><div><p>State/Province</p>
<div><b>Telangana</b><b>
</b></div></div><div><p>Country</p>
<div><b>India</b><b>
</b></div></div><div><p>Zip/Postal Code</p>
<div><b>500003</b><b>
</b></div></div></ul></div><div><h3 class="jobSectionHeader"><b>Job Description</b></h3>
<div><div>Job Description<br>
<br>
The successful candidate will work on batch and real-time data pipelines, leveraging technologies such as Scala, Java, and Apache Spark. They will manage distributed systems and cloud resources, ensure security compliance using HashiCorp Vault, orchestrate workflows with Apache Airflow, and investigate production issues by analyzing logs and monitoring metrics.<br>
<br>
Responsibilities<br>
<br>
<ul><li>Develop and maintain batch and streaming data pipelines using Scala and Java.</li><li>Write and execute shell scripts and Yarn commands for Spark job management.</li><li>Manage and optimize big data environments using Apache Spark, EMR, Hadoop, and YARN.</li><li>Utilize cloud storage and container platforms such as Amazon S3 and EKS.</li><li>Implement and manage security features, including HashiCorp Vault and tokenization/encryption protocols.</li><li>Schedule and orchestrate batch workflows using Apache Airflow.</li><li>Conduct root cause analysis and handle production incidents.</li><li>Analyze application, Spark executor, and Dynatrace logs.</li><li>Monitor jobs and EMR clusters using Dynatrace metrics.</li><li>Validate data and set up production alerts.</li></ul></div><h3 class="jobSectionHeader"><b><br>
Requirements</b></h3><div>
Experience with batch and streaming processing.<br>
<ul><li>Proficiency in distributed systems management.</li><li>Knowledge of cloud platforms (AWS S3, EKS).</li><li>Security expertise (Vault, tokenization, encryption).</li><li>Workflow orchestration (Airflow).</li><li>Strong analytical and troubleshooting skills for production investigation</li></ul></div></div></div></div><p></p>

Big Data/ Scala Developer

Tata Consultancy Services (TCS)

<div></div><div><div><div><div><p>a. Design, develop, and deploy end-to-end data engineering solutions on the Databricks platform.</p>
<p>b. Build scalable, reliable, and optimized data pipelines that support large-scale analytics, machine learning, and real-time data processing.</p>
<p>c. Utilize Databricks to integrate, transform, and analyze big data from various sources.</p>
<p>2. Team Leadership and Mentorship:</p>
<p>a. Lead and mentor a team of data engineers and developers in the creation of high-performing data systems.</p>
<p>b. Provide technical direction, guidance, and feedback to junior developers and data engineers.</p>
<p>c. Foster a collaborative environment within the team to encourage learning and knowledge sharing</p>
<p>3. Architecture &amp; Solution Design:</p>
<p>a. Design data architectures that support the business's data needs, ensuring the scalability and flexibility of data systems
</p>
<p>b. Work closely with data scientists, data analysts, and business analysts to understand requirements and create solutions that align with organizational goals
</p><p>c. Develop data models, workflows, and frameworks for seamless data integration
</p><p>4. Cloud Infrastructure Management:</p>
<p>a. Manage the Databricks environment in a cloud platform (AWS, Azure, or GCP), ensuring efficient provisioning, performance optimization, and cost management.
</p><p>b. Work with cloud teams to manage cloud resources, including storage, compute, and networking
</p><p>5. Collaboration with Stakeholders:</p>
<p>a. Collaborate with business leaders, product teams, and other stakeholders to define and prioritize data engineering initiatives
</p><p>b. Translate business requirements into technical solutions, ensuring that data needs are met effectively and efficiently
</p><p>6.</p>
</div></div></div><div><div><div><b>Location</b></div>
<div>Hyderabad</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Developer</div>
</div><div><div><b>Job Id</b></div>
<div>371241</div>
</div><div><div><b>Desired Skills</b></div>
<div>Data Analyst</div>
</div><div><div><b>Additional Information</b></div>
<div>Location Pan India</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF ENGINEERING</div></div></div><div></div>

Databricks lead developer

Geetha Technology Solutions

<p><b>Position:</b> Data Scientist (ML Developer)<br/><b>Experience:</b> 6 &ndash; 9 Years<br/><b>Employment Type:</b> Contract to Hire (1 Year)<br/><b>Working Mode:</b> Hybrid<br/><b>Location:</b> Bangalore, Hyderabad, Pune, Indore &amp; Ahmedabad</p><p><b>About the Role</b></p><p>We are seeking a highly skilled <b>Data Scientist (ML Developer)</b> with strong expertise in <b>AWS Cloud, OCR, and NLP</b> to design and implement intelligent document processing solutions. The ideal candidate will have hands-on experience in leveraging <b>Large Language Models (LLMs)</b> and <b>AWS Bedrock</b> to extract, interpret, and automate data insights from unstructured documents.</p><p><b>Key Responsibilities</b></p><ul><li>Design and develop <b>OCR-based ML workflows</b> using <b>AWS Bedrock, Textract, Lambda, API Gateway, Comprehend</b>, and other AWS services.</li><li>Implement intelligent <b>document processing pipelines</b> utilizing LLMs (Claude, Amazon Titan, Meta Llama).</li><li>Apply <b>prompt engineering</b> and <b>few-shot learning</b> techniques to optimize model performance for custom use cases.</li><li>Develop and integrate RESTful APIs to connect ML components with business systems.</li><li>Process and extract structured data (JSON, XML) from complex document layouts such as invoices and forms.</li><li>Collaborate with data engineers and business analysts to improve automation and scalability.</li><li>Ensure adherence to <b>data security, IAM, and cloud best practices</b> in all deployed solutions.</li></ul><p><b>Required Skills &amp; Expertise</b></p><ul><li>Proficiency in <b>AWS Cloud</b> (Bedrock, Textract, Lambda, API Gateway, Comprehend, IAM).</li><li>Strong understanding of <b>OCR technologies</b> and <b>LLMs</b> for document automation.</li><li>Solid knowledge of <b>Machine Learning</b> and <b>NLP concepts</b>.</li><li>Excellent programming skills in <b>Python</b>, with experience using <b>Boto3</b> for AWS integration.</li><li>Experience in handling <b>structured data</b> (JSON, XML) and integrating APIs.</li><li>Good problem-solving, analytical, and collaboration skills.</li></ul><p>Job Type: Contractual / Temporary</p><p>Pay: &#8377;1,600,000.00 - &#8377;1,700,000.00 per year</p><p>Work Location: In person</p>

Hiring ML Developer

Tata Consultancy Services

<div>
 <p>
  <strong>
   Job Title
  </strong>
  &ndash; Kafka Developer
 </p>
 <p>
  <strong>
   Company
  </strong>
  &ndash; TCS (MEA)
 </p>
 <p>
  <strong>
   Location
  </strong>
  &ndash; Riyadh, Saudi Arabia
 </p>
 <p>
  <strong>
   Job type
  </strong>
  &ndash; Full time
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   About Us :
  </strong>
 </p>
 <p>
  Tata Consultancy Services (TCS) is an IT services, consulting and business solutions organization that has been partnering with many of the world&rsquo;s largest businesses in their transformation journeys for over 50 years. TCS offers a consulting-led, cognitive powered, integrated portfolio of business, technology and engineering services and solutions. This is delivered through its unique Location Independent Agile&trade; delivery model, recognized as a benchmark of excellence in software development.
 </p>
 <p>
  A part of the Tata group, India's largest multinational business group, TCS has over 616,171 of the world&rsquo;s best-trained consultants with 157 nationalities in 53 countries. For more information, visit www.tcs.com and follow TCS news at @TCS_News.
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   <u>
    Job Description :
   </u>
  </strong>
 </p>
 <ul>
  <li>
   In-depth exp in Kafka architecture, including brokers, topics, partitions, producers, consumers, and replication.
  </li>
  <li>
   Sound knowledge of Kafka integration pattern and best practices for integration with various data source
  </li>
  <li>
   Experience with streaming data processing frameworks like Kafka Streams or Apache Flink
  </li>
  <li>
   Experience in design, deploy, and manage Kafka clusters in a production environment, ensuring high availability, scalability, and reliability.
  </li>
  <li>
   Knowledge of Kafka distributed nature, fault tolerance, scalability and high availability
  </li>
  <li>
   Collaborate with engineering teams to understand data processing requirements and design Kafka solutions that meet their needs.
  </li>
  <li>
   Optimize Kafka configurations and performance to ensure efficient utilization of resources and low latency data processing.
  </li>
  <li>
   Provide guidance and best practices to engineering teams on Kafka usage, data modelling, and integration patterns.
  </li>
  <li>
   Work closely with operations teams to monitor Kafka clusters, troubleshoot issues, and perform capacity planning.
  </li>
  <li>
   Stay updated with the latest Kafka features, enhancements, and best practices, and evaluate their applicability to our environment.
  </li>
  <li>
   Good exposure in setting up DR platform &ndash; Active &amp; Passive.
  </li>
 </ul>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Application Deadline :
  </strong>
  31-Nov-2025
 </p>
 <p>
  <br/>
 </p>
 <p>
  <strong>
   Privacy Note:
  </strong>
 </p>
 <p>
  https://www.tcs.com/connect-with-tcs/privacy-policy
 </p>
</div>


Kafka Developer

Srijan Technologies Pvt Ltd

<div><div><b>Material</b> is a global strategy, insights, design, and technology partner to companies striving for true customer centricity and ongoing relevance in a digital-first, customer-led world. By leveraging proprietary, science-based tools that enable human understanding, we inform and create customer-centric business models and experiences + deploy measurement systems &ndash; to build transformational relationships between businesses and the people they serve.</div>
<div><div>https://www.materialplus.io/</div></div><br>
<p></p>
<p><b>
Senior Developer/ Lead - Data Science</b></p>
<p>
We are looking for <b>Senior Developer/ Lead - Data Scientist</b> with strong Generative AI experience, skilled in Python, TensorFlow/PyTorch/Scikit&#8209;learn, Azure OpenAI GPT, and multi-agent frameworks like LangChain or AutoGen, with strong data preprocessing, feature engineering, and model evaluation expertise.</p>
<p><b>
Bonus</b>: familiarity with Big Data tools (Spark, Hadoop, Databricks, SQL/NoSQL) and ReactJS</p><br>
<p></p>
<p><b>
Experience: 5+ year</b></p>
<p><b>
Preferred Location: -</b> Gurgaon/ Bangalore</p>
<p><b>
Job Description: -</b></p>
<p><b>
Generative AI: </b>GenAI models (Eg.: Azure OpenAI GPT), Multi-Agent System Architecture:</p>
<ul>
<li>Proficiency in <b>Python</b> and AI/ML libraries (e.g., TensorFlow, PyTorch, Scikit-learn).</li>
<li>Experience with <b>LangChain</b>, <b>AutoGen</b>, or similar frameworks for multi-agent systems.</li>
<li>Strong knowledge of <b>Data Science</b> techniques, including data preprocessing, feature engineering, and model evaluation.</li></ul><br>
<p></p>
<p><b>
Nice to have:</b></p>
<ul>
<li>Familiarity with <b>Big Data</b> tools (e.g., Spark, Hadoop) / Databricks and databases (e.g., SQL, NoSQL).</li>
<li>Expertise in <b>ReactJS</b> for building responsive and interactive user interfaces.</li></ul><br>
<p></p>
<p><b>
What We Offer</b></p>
<ul>
<li>Professional Development and Mentorship.</li>
<li>Hybrid work mode with remote friendly workplace. (6 times in a row Great Place To Work Certified).</li>
<li>Health and Family Insurance.</li>
<li>40+ Leaves per year along with maternity &amp; paternity leaves.</li>
<li>Wellness, meditation and Counselling sessions</li>
</ul>
<p>If you are looking forward to it, kindly apply or share your resume with me at Vineet.kumar@materialplus.io</p></div>

Senior Developer/ Technical Lead - Data Scientist (Agentic AI)

Search 3359+ Hadoop developer jobs near you this November. New roles added regularly across leading industries. Apply today.