hadoop software engineer

INFLUX SERVICE

Job Title: Hadoop Maintenance EngineerLocation: Mohali Experience: 2&ndash;5 YearsJob Overview:We are seeking a skilled Hadoop Maintenance Engineer to manage, monitor, and optimize our Hadoop ecosystem. The ideal candidate will have hands-on experience in maintaining HDFS, Hive, and Spark clusters, ensuring smooth data flow and system reliability across distributed environments.Key Responsibilities:<ul><li>Perform daily monitoring, maintenance, and troubleshooting of Hadoop clusters (HDFS, Hive, Spark).</li><li>Manage cluster upgrades, patching, and performance tuning.</li><li>Ensure data reliability, availability, and security within the Hadoop ecosystem.</li><li>Diagnose and resolve cluster issues, node failures, and performance bottlenecks.</li><li>Implement data replication, backup, and recovery strategies.</li><li>Work closely with data engineering teams to optimize Hive queries and Spark jobs.</li><li>Monitor and maintain system health using tools like Ambari, Cloudera Manager, or similar.</li><li>Develop scripts for automation and system improvements (using Shell, Python, etc.).</li><li>Document processes, configurations, and incident resolutions.</li></ul>Required Skills &amp; Experience:<ul><li>2&ndash;5 years of experience in Hadoop ecosystem management.</li><li>Strong knowledge of HDFS architecture and file management.</li><li>Expertise in Hive (data warehousing, query optimization) and Spark (batch/stream processing).</li><li>Proficiency in Linux/Unix systems and shell scripting.</li><li>Familiarity with Hadoop ecosystem tools like YARN, Oozie, Sqoop, or Kafka (preferred).</li><li>Understanding of cluster resource management and tuning.</li><li>Experience with monitoring tools and log analysis.</li></ul>Preferred Qualifications:<ul><li>Experience with Cloudera or Hortonworks distributions.</li><li>Knowledge of cloud-based Hadoop deployments (AWS EMR, Azure HDInsight, GCP Dataproc).</li><li>Strong analytical and troubleshooting skills.</li><li>Excellent communication and documentation abilities.</li></ul>Education:<ul><li>Bachelor&rsquo;s degree in Computer Science, Information Technology, or related field.</li></ul>Job Types: Full-time, PermanentWork Location: In person

Hadoop Maintenance Engineer

IQVIA

<div><h3 class="jobSectionHeader">About the Role
</h3>We are seeking a highly skilled Cloudera Engineer to join our Big Data team. In this role, you will design, build, and manage Hadoop clusters using Cloudera technologies, ensuring optimal performance, scalability, and security. You will collaborate with cross-functional teams to deliver robust big data solutions that power critical business insights.
<h3 class="jobSectionHeader">Key Responsibilities
</h3><ul><li>Build and configure new clusters and projects involving Cloudera (CDH) technologies.
</li><li>Manage storage, performance tuning, and volume management for Hadoop clusters.
</li><li>Troubleshoot and resolve application and cluster issues, escalating when necessary.
</li><li>Handle cluster design, capacity planning, performance optimization, monitoring, and scaling.
</li><li>Perform standard system administration tasks, including user management, permissions, and upgrades.
</li><li>Monitor and resolve day-to-day cluster issues, ensuring SLA compliance.
</li><li>Support users with administrative needs and optimize job performance.
</li><li>Perform SQL queries and performance tuning across all levels.
</li><li>Collaborate with cross-functional teams to set up production environments and enable business solutions.
</li><li>Develop scripts to automate reporting and maintenance activities.
</li><li>Implement backup and recovery strategies using Cloudera Replication Manager.
</li><li>Address live performance and stability issues and prevent recurrence.
</li><li>Support data engineering efforts, ETL pipelines, and data science teams using Python and Scala.
</li></ul><h3 class="jobSectionHeader">What We&rsquo;re Looking For
</h3><ul><li>Proven experience in Hadoop administration and Cloudera (CDH) technologies.
</li><li>Strong knowledge of cluster design, performance tuning, and troubleshooting.
</li><li>Proficiency in SQL and scripting for automation.
</li><li>Familiarity with backup/recovery strategies and operational best practices.
</li><li>Ability to support data engineering and data science teams.
</li><li>Strong problem-solving skills and ability to work under pressure.
</li></ul><h3 class="jobSectionHeader">Why Join Us?
</h3><ul><li>Work on cutting-edge big data technologies in a collaborative environment.
</li><li>Opportunity to influence architecture and performance strategies.
</li><li>Be part of a team that values innovation, efficiency, and continuous improvement.
</li></ul>IQVIA is a leading global provider of clinical research services, commercial insights and healthcare intelligence to the life sciences and healthcare industries. We create intelligent connections to accelerate the development and commercialization of innovative medical treatments to help improve patient outcomes and population health worldwide . Learn more at https://jobs.iqvia.com</div>

Cloudera Engineer

Webner Solutions pvt ltd

<div></div><div><div><div>We are looking for an experienced Hadoop Administrator to design, build, and maintain our Hadoop ecosystem based on the Open Data Platform (ODP) framework. The ideal candidate will have hands-on experience in the installation, configuration, tuning, and administration of key Hadoop components, including HDFS, YARN, Hive, Spark, Ranger, and Oozie.

This role involves end-to-end platform setup, ongoing maintenance, performance optimization, and support for enterprise big data workloads.

Key Responsibilities
<ul>
<li>Install, configure, and manage Hadoop clusters and ecosystem components (HDFS, YARN, Hive, Spark, Zookeeper, Oozie, etc.) on ODP-compliant distributions.</li>
<li>Build and deploy Hadoop stacks from scratch, including hardware sizing, capacity planning, and architecture design.</li>
<li>Implement cluster high availability (HA), backup/recovery, and disaster recovery strategies.</li>
<li>Manage user access, security policies, and Kerberos/Ranger configurations.</li>
<li>Perform cluster performance tuning, troubleshooting, and log analysis to ensure system stability.</li>
<li>Monitor system health and optimize resource utilization using Ambari, Cloudera Manager, or other monitoring tools.</li>
<li>Automate cluster operations using shell scripts or Python for deployment, maintenance, and patching.</li>
<li>Collaborate with data engineering and infrastructure teams for upgrades, migrations, and platform integrations.</li>
<li>Maintain detailed documentation for architecture, configurations, and operational runbooks.</li>
</ul>
Required Skills &amp; Experience
<ul>
<li>5&ndash;10 years of experience in Hadoop ecosystem administration.</li>
<li>Proven experience building Hadoop clusters from scratch using ODP distributions (Hortonworks, Cloudera, or similar).</li>
<li>Strong expertise in:
<ul><li>HDFS, YARN, Hive, Spark, Zookeeper, Oozie</li>
<li>Ambari or Cloudera Manager (installation, service management, and monitoring)</li>
<li>Kerberos, Ranger, or Sentry for security and authorization.</li>
<li>Proficiency in Linux system administration, shell scripting, and configuration management.</li>
<li>Experience with performance tuning, capacity planning, and troubleshooting in production environments.</li>
<li>Familiarity with HA configurations, NameNode failover, and cluster scaling.</li>
</ul>
</li></ul>Education
<ul>
<li>Regular MCA or Bachelor&rsquo;s degree in Computer Science, Information Technology, Engineering, or equivalent experience.
</li>
</ul>Nice to Have
<ul>
<li>Experience with cloud-based Hadoop environments (AWS EMR, Azure HDInsight, GCP Dataproc).</li>
<li>Exposure to containerized big data platforms (Kubernetes, Docker).</li>
<li>Knowledge of automation tools (Ansible, Terraform, Puppet).</li>
<li>Experience with Kafka, Airflow, or NiFi for data pipeline integration.\</li>
<li>Understanding of data governance, auditing, and monitoring best practices.</li></ul></div></div></div>
 To apply for a job, send an email to companyhr@webners.com. Do not forget to attach your detailed resume.

Hadoop Administrator / Big Data Platform Engineer

We are looking for skilled Hadoop Ecosystem Support Engineers to provide operational support and ensure the stability, performance, and availability of big data platforms. The ideal candidate will have hands-on experience managing and troubleshooting the Hadoop ecosystem &mdash; including HDFS, Hive, Spark, YARN, and other related components.This role focuses on providing support, maintenance, and resolving issues.Key Responsibilities<ul><li>Provide L3 production support for Hadoop ecosystem components (HDFS, Hive, Spark, YARN, Oozie, etc.).</li><li>Monitor cluster health, performance, and resource utilization using tools such as Ambari, Cloudera Manager, or Grafana.</li><li>Troubleshoot and resolve HDFS, Hive, and Spark job failures and performance issues.</li><li>Perform root cause analysis (RCA) for recurring incidents and work with engineering teams to implement fixes.</li><li>Manage user access, quotas, and security policies in Hadoop clusters.</li><li>Conduct routine maintenance tasks such as service restarts, cluster upgrades, and patch management.</li><li>Collaborate with data engineers and platform teams to ensure optimal cluster performance and reliability.</li><li>Document support procedures, incident reports, and configuration changes.</li></ul>Required Skills &amp; Experience<ul><li>3&ndash;8 years of experience supporting or administering Hadoop ecosystems in production.</li><li>Strong hands-on knowledge of:</li><li>HDFS (file system management, data balancing, recovery)</li><li>Hive (query execution, metastore management, troubleshooting)</li><li>Spark (job monitoring, debugging, performance tuning)</li><li>YARN, Oozie, and Zookeeper</li><li>Experience with cluster management tools like Ambari, Cloudera Manager, or similar.</li><li>Proficiency in Linux/Unix system administration and shell scripting.</li><li>Strong analytical and problem-solving skills with a focus on incident management and RCA.</li><li>Familiarity with Kerberos, Ranger, or other security frameworks within Hadoop.</li></ul>Nice to Have<ul><li>Exposure to cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).</li><li>Basic understanding of Python or Scala for log analysis and automation.</li><li>Experience with Kafka, Airflow, or other data orchestration tools.</li><li>Knowledge of ticketing systems (ServiceNow, JIRA) and ITIL processes.</li></ul>Job Types: Full-time, PermanentWork Location: In person

Hadoop Ecosystem Support Engineers

T-Mobile

<div>At T-Mobile, we invest in YOU! Our Total Rewards Package ensures that employees get the same big love we give our customers. All team members receive a competitive base salary and compensation package - this is Total Rewards. Employees enjoy multiple wealth-building opportunities through our annual stock grant, employee stock purchase plan, 401(k), and access to free, year-round money coaches. That&rsquo;s how we&rsquo;re UNSTOPPABLE for our employees!
Are you ready to join the Un-carrier movement? 
 
The Sr Software Engineer works on building and supporting a platform the manages customer risk and bad debt, working with a team of other software engineers, network and systems engineers to design, implement, and deploy software which meet customer's requirements, scales easily, removes the limitations of traditional networking solutions, and supports deployment in highly available environments. This Engineer actively participates in architecture and design of various software components focused on crafting applications that leverage technologies such as virtualization, micro services, SDN, NFV, and Big Data platforms and technologies. They work on technical assignments that are moderately difficult, requiring judgment in resolving issues or in making recommendations on industry leading products for risk and bad debt management. The assignments require considerable judgment and initiative, understand implications of work and make recommendations for solutions. They normally receive little instruction on daily work, and only require general instructions on newly introduced assignments. 
 
We are a team that encourages innovation and advocates an agile and open approach, truly working and playing in the Un-carrier way!
Job Responsibilities:
Technical Engineering Services<ul><li>
Drives engineering projects by developing software solutions; conducting tests and inspections; preparing reports and calculations. Encouraged to guide base and associate level engineers as needed.</li></ul>
Technical Leadership<ul><li>
Collaborates with technical teams and uses system expertise to deliver technical solutions.</li><li>
Continuously learns and teaches others existing and new technologies</li><li>
Contributes to the development of others through mentoring or in house workshops and learning sessions.</li></ul>
Technology Strategy<ul><li>
Contributes to new and existing technology options that support business goals.</li><li>
Understands current technology that supports business goals. Understands system protocols, how systems operate and data flows. Aware of current technology benefits. Encouraged to independently develop a full software stack. Understands the building blocks, interactions, dependencies, and tools required to complete software and automation work. Independent study of current technology is expected. Interact with system engineers to define system requirement and/or vital requirements for automation.</li></ul>
Innovation<ul><li>
Contributes to designs to implement new ideas which utilize new frameworks to improve an existing or new system/process/service.</li><li>
Review existing designs and processes to highlight more efficient ways to complete existing workload more optimally through industry perspectives.</li><li>
Understands the creation of company IPR.</li></ul>
Technical Writing<ul><li>
Writes basic documentation on how technology works. Builds clear documentation for new code and systems used.</li><li>
Documenting systems designs, presentations, and business requirements for consumption and consideration at the Manager and Sr.Manager level.</li></ul> 
Education and Work Experience:<ul><li>
Bachelor's Degree Computer Science or Engineering or a related field (Required)</li></ul>
Work Experience:<ul><li>
4-7 years Technical engineering experience. Required.</li><li>
3+ years experience in developing cloud solutions using Azure platform with Azure resources ADF, Event Hub, App Services, Log Analytics and VMs. Required.</li><li>
3+ years experience in Azure Data Bricks with programming in Python, Scala and Spark SQL. Required.</li><li>
Experience in software development, with a focus on creating scalable and high-availability solutions.</li><li>
Experience with Agile/Scrum methodologies and DevOps practices.</li><li>
Proficiency in Cloud Architecture and advances patterns.</li><li>
Proficiency in Snowflake platform. Preferred.</li></ul>
Knowledge, Skills and Abilities:<ul><li>
Expertise in cloud technologies, microservices architecture, backend system development, and alignment to security standards.</li><li>
Expertise with high-resilience and high-performing system design.</li><li>
Experience with Gitlab/Github CI/CD pipeline, SQL DB, SaaS, PaaS, Private &amp; Public Clouds, distributed system design, microservices, Kubernetes, containerized applications, Splunk, Java, SQL, Python.</li><li>
Strong communication, interpersonal and technical writing skills and the ability to convey technical concepts to a wide range of audiences.</li><li>
Excellent problem-solving skills and the ability to handle complex, high-stakes projects.</li><li>
AI/ML experience is a plus.</li></ul><ul><li>
At least 18 years of age</li><li>
Legally authorized to work in the United States</li></ul> 

Travel: 
Travel Required (Yes/No): Yes 
 
DOT Regulated: 
DOT Regulated Position (Yes/No): No 
Safety Sensitive Position (Yes/No): No 

Base Pay Range: $113,600 - $205,000
Corporate Bonus Target: 15%
The pay range above is the general base pay range for a successful candidate in the role. The successful candidate&rsquo;s actual pay will be based on various factors, such as work location, qualifications, and experience, so the actual starting pay will vary within this range.
At T-Mobile, employees in regular, non-temporary roles are eligible for an annual bonus or periodic sales incentive or bonus, based on their role. Most Corporate employees are eligible for a year-end bonus based on company and/or individual performance and which is set at a percentage of the employee&rsquo;s eligible earnings in the prior year. Certain positions in Customer Care are eligible for monthly bonuses based on individual and/or team performance. To find the pay range for this role based on hiring location, click here.
At T-Mobile, our benefits exemplify the spirit of One Team, Together! A big part of how we care for one another is working to ensure our benefits evolve to meet the needs of our team members. Full and part-time employees have access to the same benefits when eligible. We cover all of the bases, offering medical, dental and vision insurance, a flexible spending account, 401(k), employee stock grants, employee stock purchase plan, paid time off and up to 12 paid holidays - which total about 4 weeks for new full-time employees and about 2.5 weeks for new part-time employees annually - paid parental and family leave, family building benefits, back-up care, enhanced family support, childcare subsidy, tuition assistance, college coaching, short- and long-term disability, voluntary AD&amp;D coverage, voluntary accident coverage, voluntary life insurance, voluntary disability insurance, and voluntary long-term care insurance. We don't stop there - eligible employees can also receive mobile service &amp; home internet discounts, pet insurance, and access to commuter and transit programs! To learn about T-Mobile&rsquo;s amazing benefits, check out www.t-mobilebenefits.com.
Never stop growing! 
As part of the T-Mobile team, you know the Un-carrier doesn&rsquo;t have a corporate ladder&ndash;it&rsquo;s more like a jungle gym of possibilities! We love helping our employees grow in their careers, because it&rsquo;s that shared drive to aim high that drives our business and our culture forward. By applying for this career opportunity, you&rsquo;re living our values while investing in your career growth&ndash;and we applaud it. You&rsquo;re unstoppable! 
 
T-Mobile USA, Inc. is an Equal Opportunity Employer. All decisions concerning the employment relationship will be made without regard to age, race, ethnicity, color, religion, creed, sex, sexual orientation, gender identity or expression, national origin, religious affiliation, marital status, citizenship status, veteran status, the presence of any physical or mental disability, or any other status or characteristic protected by federal, state, or local law. Discrimination, retaliation or harassment based upon any of these factors is wholly inconsistent with how we do business and will not be tolerated. 
 
Talent comes in all forms at the Un-carrier. If you are an individual with a disability and need reasonable accommodation at any point in the application or interview process, please let us know by emailing ApplicantAccommodation@t-mobile.com or calling 1-844-873-9500. Please note, this contact channel is not a means to apply for or inquire about a position and we are unable to respond to non-accommodation related requests.</div>

Sr Software Engineer - Databricks

Version 1

<div>Company Description 
Version 1 has celebrated over 28 years in Technology Services and continues to be trusted by global brands to deliver solutions that drive customer success. Version 1 has several strategic technology partners including Microsoft, AWS, Oracle, Red Hat, OutSystems and Snowflake. We&rsquo;re also an award-winning employer reflecting how employees are at the heart of Version 1.
We&rsquo;ve been awarded: Innovation Partner of the Year Winner 2023 Oracle EMEA Partner Awards, Global Microsoft Modernising Applications Partner of the Year Award 2023, AWS Collaboration Partner of the Year - EMEA 2023 and Best Workplaces for Women by Great Place To Work in UK and Ireland 2023.
As a consultancy and service provider, Version 1 is a digital-first environment and we do things differently. We&rsquo;re focused on our core values; using these we&rsquo;ve seen significant growth across our practices and our Digital, Data and Cloud team is preparing for the next phase of expansion. This creates new opportunities for driven and skilled individuals to join one of the fastest-growing consultancies globally. 
Job Description 
We are seeking a talented and experienced Databricks Engineer to join our innovative team in Bengaluru, India. In this role, you will be responsible for designing, implementing, and optimising data pipelines and analytics solutions using the Databricks platform. You will collaborate closely with cross-functional teams to deliver high-quality, scalable data solutions that drive business insights and decision-making.<ul><li>
Develop and optimise ETL processes using Databricks and Apache Spark</li><li>
Design and implement efficient data models and schemas for optimal data storage and retrieval</li><li>
Collaborate with data scientists and analysts to build and deploy machine learning models</li><li>
Ensure data quality, consistency, and reliability across all data pipelines</li><li>
Optimise query performance and troubleshoot data processing issues</li><li>
Implement robust data security and access control measures</li><li>
Stay abreast of the latest Databricks features and best practices</li><li>
Participate in code reviews and contribute to the team's knowledge base</li><li>
Mentor junior team members and share expertise in big data technologies</li><li>
Develop and maintain documentation for data processes and workflows</li><li>
Collaborate with stakeholders to understand business requirements and translate them into technical solutions</li></ul> 

Qualifications 
<ul><li>
Bachelor's degree in Computer Science, Data Science, or a related field</li><li>
6+ years of experience in data engineering or a related field</li><li>
Strong proficiency in Databricks platform and Apache Spark</li><li>
Excellent SQL skills and experience with Python or Scala</li><li>
Proven experience in designing and implementing ETL processes and data pipelines</li><li>
Solid understanding of big data technologies and distributed computing concepts</li><li>
Experience working with cloud platforms such as AWS, Azure, or GCP</li><li>
Familiarity with version control systems, preferably Git</li><li>
Strong analytical and problem-solving skills with keen attention to detail</li><li>
Excellent communication and collaboration skills</li><li>
Experience with data warehousing and analytics concepts</li><li>
Knowledge of data governance and compliance standards</li><li>
Experience with real-time data processing and streaming technologies</li><li>
Ability to work in a fast-paced, dynamic environment and manage multiple priorities</li><li>
Databricks certifications are desirable</li></ul> 

Additional Information 
At Version 1, we believe in providing our employees with a comprehensive benefits package that prioritises their well-being, professional growth, and financial stability.
One of our standout advantages is the ability to work with a hybrid schedule along with business travel, allowing our employees to strike a balance between work and life. We also offer a range of tech-related benefits, including an innovative Tech Scheme to help keep our team members up-to-date with the latest technology.
We prioritise the health and safety of our employees, providing private medical and life insurance coverage, as well as free eye tests and contributions towards glasses. Our team members can also stay ahead of the curve with incentivized certifications and accreditations, including AWS, Microsoft, Oracle, and Red Hat.
Our employee-designed Profit Share scheme divides a portion of our company's profits each quarter amongst employees. We are dedicated to helping our employees reach their full potential, offering Pathways Career Development Quarterly, a programme designed to support professional growth.</div>

Databricks Engineer

Cognizant Technology Solutions

<div>Skills- AWS +Pyspark
Experience: 4 to 9 years
Location: AIA-Pune
As an AWS Data Engineer, you will design, build, and manage robust data pipelines on the AWS cloud platform. You will leverage your expertise in AWS data services and programming languages to ingest, process, transform, and store data efficiently, enabling data-driven insights and decision-making.
Key Responsibilities :
<ul><li>Data Pipeline Development: Design, implement, and maintain scalable data pipelines using AWS services such as S3, Glue, Lambda, EMR, and Kinesis.</li><li>Data Processing and Transformation: Utilize PySpark, Spark, and SQL to perform complex data transformations and aggregations on large datasets within the AWS ecosystem.</li><li>Data Storage and Management: Design and implement data storage solutions using Amazon S3, RDS, DynamoDB, and Redshift, ensuring data quality, integrity, and accessibility.</li><li>Data Modeling and Warehousing: Develop and maintain data models to support analytics and reporting, leveraging Redshift as the data warehousing solution.</li><li>Infrastructure and Cloud Technologies: Provision and manage scalable data infrastructure using EC2, VPC, IAM, CloudFormation, and other relevant AWS services.</li><li>Performance Optimization and Monitoring: Continuously monitor and optimize data pipelines and systems using CloudWatch, identifying and resolving performance bottlenecks.</li><li>Collaboration and Knowledge Sharing: Work closely with data scientists, analysts, and other stakeholders to understand data requirements and provide technical guidance.</li></ul>
Qualifications :
<ul><li>Proficiency in Python, SQL, PySpark, and Spark.</li><li>Strong expertise in AWS data services (S3, Glue, Lambda, EMR, Redshift, DynamoDB, RDS).</li><li>Experience with data warehousing, ETL processes, and data modeling.</li><li>Excellent problem-solving, analytical, and communication skills.</li></ul></div>

AWS + Pyspark

Search 10000+ Hadoop software engineer jobs near you this October. New roles added regularly across leading industries. Apply today.