spark developer

Kandor people

<p>​<b>Job Title: Scala Developer (4&ndash;10 Years Exp)</b></p><p>​<b>Company:</b> Tata Consultancy Services (TCS)</p><p><b>Location:</b> Bangalore, Chennai, Hyderabad, Pune, Kolkata</p><p><b>Job Type:</b> Full-Time</p><p>​<b>Job Summary</b></p><p>​We are looking for a highly skilled and experienced <b>Scala Developer</b> to join our engineering team. You will be responsible for designing and developing high-performance, distributed applications and big data pipelines. The ideal candidate has a deep understanding of functional programming and experience working within the Hadoop/Spark ecosystem to deliver scalable enterprise solutions.</p><p>​<b>Key Responsibilities</b></p><ul><li>​<b>Design &amp; Development:</b> Build and maintain scalable, high-performance backend services and data pipelines using <b>Scala</b> and <b>Apache Spark</b>.</li><li>​<b>Architecture:</b> Participate in the design of distributed systems, ensuring high availability, reliability, and low latency.</li><li>​<b>Code Quality:</b> Write clean, testable, and maintainable code following functional programming principles. Perform rigorous code reviews and unit testing (using ScalaTest or MUnit).</li><li>​<b>Optimization:</b> Debug and tune Spark jobs and SQL queries to optimize performance for large-scale data processing.</li><li>​<b>Collaboration:</b> Work closely with Data Scientists, Architects, and cross-functional teams in an <b>Agile/Scrum</b> environment to translate business requirements into technical specs.</li></ul><p>​<b>Technical Skills Required</b></p><ul><li>​<b>Core Language:</b> Strong proficiency in <b>Scala</b> (Object-Oriented &amp; Functional Programming).</li><li>​<b>Big Data Stack:</b> Extensive experience with <b>Apache Spark (Spark SQL, Streaming)</b>, Hadoop, Hive, and HDFS.</li><li>​<b>Frameworks:</b> Knowledge of <b>Akka, Play Framework</b>, or effect systems like <b>Cats/ZIO</b> is a plus.</li><li>​<b>Data Stores:</b> Proficiency in SQL (PostgreSQL/Oracle) and NoSQL (Cassandra/HBase/MongoDB).</li><li>​<b>Cloud &amp; DevOps:</b> Experience with <b>AWS/Azure/GCP</b> services. Familiarity with Docker, Kubernetes, and CI/CD pipelines (Jenkins/GitLab).</li><li>​<b>Streaming:</b> Experience with messaging systems like <b>Apache Kafka</b>.</li></ul><p>​<b>Desired Candidate Profile</b></p><ul><li>​<b>Experience:</b> 4 to 10 years of professional software development experience, with at least 3+ years dedicated to Scala.</li><li>​<b>Education:</b> B.E. / B.Tech / M.E. / M.Tech / MCA from a recognized university.</li><li>​<b>Soft Skills:</b> Strong analytical and problem-solving skills; excellent communication and ability to work in a global team environment.</li></ul><p>Job Type: Full-time</p><p>Work Location: In person</p>

Scala Developer

Tata Consultancy Services (TCS)

<div></div><div><div><div><p><div><div><div><div><p><b>Desired Competencies (Technical/Behavioral Competency</b><b>)</b></p><br>
</div></div><br>
<div><div><p><b>Must-Have**</b></p><br>
</div><div><ul><li> Strong hands-on experience of Hadoop, HIVE, Spark, PySpark, Big Data Eco System Tools.</li></ul>
<ul><li> Strong SQL experience.</li></ul>
<ul><li> Should be able to develop, tweak queries and work on performance enhancement.</li></ul>
<ul><li> Solid understanding of object-oriented programming and HDFS concepts.<br>
</li></ul></div></div><br>
<div><div><p><b>Good-to-Have</b></p><br>
</div><div><ul><li> Preferable to have good DWH/ Data Lake knowledge.</li></ul>
<ul><li> The candidate should have good DevOps and Agile Development Framework knowledge.</li></ul>
</div></div><br>
</div></div></p></div></div><div><div><div><b>Location</b></div>
<div>Bengaluru</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Developer</div>
</div><div><div><b>Job Id</b></div>
<div>397932</div>
</div><div><div><b>Desired Skills</b></div>
<div>Hadoop | Apache Spark | Hive</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF COMPUTER SCIENCE</div></div></div><div></div>

Big Data Developer

<div></div><div><div><div><div><p>1. Good understanding of Hadoop concepts including file system and Map Reduce.</p>
<p>2. Hands on experience on Spark framework, Unix scripting, Hive queries, wriring UDF in Hive. Theortical knowledge and POC alone will not suffice.</p>
<p><b>3.</b> Good knowledge in Software Development Life Cycle and Project Development Lifecycle.</p>
<b>4. </b>Associate should be able to work independently and should have strong debugging skill in both Hive and Spark. Associate should have experience developing large scale systems, experience debugging and performance tuning, excellent software design skills, communication skills and ability to work with client partners. Hands on in Apache Spark with Python or any other language (Preferred is Python -PySpark)<b>1. </b>Experience in Banking and Finance Domain.
<p><b>
2. </b>Experience in Agile Methodology</p>
<p><b>3. </b>Knowledge in job scheduling tools like Autosys.</p>
4. Knowledge in Kafka.
</div></div></div><div><div><div><b>Location</b></div>
<div>Bengaluru</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Developer</div>
</div><div><div><b>Job Id</b></div>
<div>397945</div>
</div><div><div><b>Desired Skills</b></div>
<div>BigData | Hadoop</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF ENGINEERING</div></div></div><div></div>

Bigdata Developer

<div></div><div><div><div><div><div>Must Have Technical/Functional Skills
</div><div>8+ years of skilled Databricks Developer to design, develop, and optimize data pipelines and analytics solutions using Databricks on cloud platforms (Azure/AWS/GCP). The ideal candidate will have strong experience in Spark, Python/Scala, and SQL, along with a solid understanding of big data processing and data engineering best practices
</div><div>Hands-on experience with Databricks and Apache Spark.
</div><div>Strong programming skills in Python and/or Scala.
</div><div>Proficiency in SQL for data manipulation and analysis.
</div><div>Experience with Delta Lake and Lakehouse architecture.
</div><div>Familiarity with cloud platforms (Azure Data Lake, AWS S3, or GCP).
</div><div>Knowledge of CI/CD pipelines and DevOps practices.
</div><div>Understanding of data modeling, data governance, and security.
</div><div>Experience with performance tuning for big data jobs.</div><div><br>
</div>
<div></div><br>
<div>Salary Range $110000-$135,000 year
</div>
<div></div><div>TCS Employee Benefits Summary:
</div><div>Discretionary Annual Incentive.
</div><div>Comprehensive Medical Coverage: Medical &amp; Health, Dental &amp; Vision, Disability Planning &amp; Insurance, Pet Insurance Plans.
</div><div>Family Support: Maternal &amp; Parental Leaves.
</div><div>Insurance Options: Auto &amp; Home Insurance, Identity Theft Protection.
</div><div>Convenience &amp; Professional Growth: Commuter Benefits &amp; Certification &amp; amp; Training Reimbursement.
</div><div>Time Off: Vacation, Time Off, Sick Leave &amp; Holidays.
</div><div>Legal &amp; Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
</div>
<div></div><div>#LI-SP1
</div></div></div></div><div><div><div><b>Location</b></div>
<div>Pleasanton, CA</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Engineer</div>
</div><div><div><b>Job Id</b></div>
<div>398064</div>
</div><div><div><b>Desired Skills</b></div>
<div>Python</div>
</div><div><div><b>Salary Range</b></div>
<div>$110,000-$135,000 a year</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF COMPUTER SCIENCE</div></div></div><div></div>

Databricks engineer

HoonarTek

<p></p><div><div><div><div><div><div><div>Pune</div></div></div></div></div><div><p>
</p></div></div>
<div><div><div><div><div><p><b>About Us</b></p><div><div><div>
We empower enterprises globally through intelligent, creative, and insightful services for data integration, data analytics and data visualization.</div><div>
Hoonartek is a leader in enterprise transformation, data engineering and an acknowledged world-class Ab Initio delivery partner.</div><div>
Using centuries of cumulative experience, research and leadership, we help our clients eliminate the complexities &amp; risk of legacy modernization and safely deliver big data hubs, operational data integration, business intelligence, risk &amp; compliance solutions and traditional data warehouses &amp; marts.</div><div>
At Hoonartek, we work to ensure that our customers, partners and employees all benefit from our unstinting commitment to delivery, quality and value. Hoonartek is increasingly the choice for customers seeking a trusted partner of vision, value and integrity</div><div></div><div><b><br>
How We Work?</b></div><div></div><div><br>
Define, Design and Deliver (D3) is our in-house delivery philosophy. It&rsquo;s culled from agile and rapid methodologies and focused on &lsquo;just enough design&rsquo;. We embrace this philosophy in everything we do, leading to numerous client success stories and indeed to our own success.</div><div>
We embrace change, empowering and trusting our people and building long and valuable relationships with our employees, our customers and our partners. We work flexibly, even adopting traditional/waterfall methods where circumstances demand it. At Hoonartek, the focus is always on delivery and value.</div></div></div></div></div></div></div></div>
<div><div><div><div><p><b>Job Description</b></p><div><div><div>
A Databricks Developer <b>designs, builds, and optimizes high-performance data pipelines and analytics solutions using Spark, Python (PySpark), and SQL within the Databricks Lakehouse platform</b>. Key responsibilities include implementing ELT/ETL processes using Delta Lake (Medallion Architecture), optimizing cluster performance, and automating workflows.</div><div>
Key Responsibilities</div><ul>
<li><b>Data Pipeline Development:</b> Design, develop, and maintain complex ELT/ETL pipelines using PySpark, Spark SQL, and Databricks notebooks to ingest data from various sources.</li>
<li><b>Performance Optimization:</b> Tune Spark jobs, optimize SQL queries, and manage Databricks cluster configurations to improve efficiency and reduce costs.</li>
<li><b>Streaming &amp; Batch Processing:</b> Develop and deploy real-time data ingestion pipelines using Spark Structured Streaming.</li>
<li><b>Workflow Automation:</b> Use tools like Azure Data Factory (ADF) or Databricks Workflows for scheduling and orchestrating data pipelines.</li>
<li><b>Collaboration &amp; Governance:</b> Work with data engineers, architects, and scientists to ensure data quality, security, and adherence to governance policies.</li></ul><div>
Required Technical Skills</div><ul>
<li><b>Core Technologies:</b> Strong proficiency in databricks, Apache Spark, PySpark, and SQL.</li>
<li><b>Cloud Platforms:</b> Experience with Azure Databricks (ADLS Gen2, ADF) or Databricks on AWS/GCP.</li>
<li><b>Programming &amp; Tools:</b> Proficiency in Python, Git, and CI/CD pipelines</li></ul><div>
Typical Qualifications</div><ul>
<li>Bachelor&rsquo;s degree in Computer Science, Engineering, or related field.</li>
<li>3-5 years of experience in data engineering, data warehousing, or big data processing.</li>
<li>Proven ability to troubleshoot and resolve performance bottlenecks in distributed computing environments.</li></ul></div></div></div></div></div></div></div><div></div>

Dadabricks Developer

<div></div><div><div><div><p><div><div><div><div><p><b>Must-Have**</b></p><br>
</div><div><ul><li> Strong hands-on experience of Hadoop, HIVE, Spark, PySpark, Big Data Eco System Tools.</li></ul>
<ul><li> Strong SQL experience.</li></ul>
<ul><li> Should be able to develop, tweak queries and work on performance enhancement.</li></ul>
<ul><li> Solid understanding of object-oriented programming and HDFS concepts.<br>
</li></ul></div></div><br>
<div><div><p><b>Good-to-Have</b></p><br>
</div><div><ul><li> Preferable to have good DWH/ Data Lake knowledge.</li></ul>
<ul><li> &amp;n bsp; The candidate should have good DevOps and Agile Development Framework knowledge.</li></ul>
</div></div><br>
</div></div></p></div></div><div><div><div><b>Location</b></div>
<div>Chennai</div>
</div><div><div><b>Job Function</b></div>
<div>TECHNOLOGY</div>
</div><div><div><b>Role</b></div>
<div>Developer</div>
</div><div><div><b>Job Id</b></div>
<div>397923</div>
</div><div><div><b>Desired Skills</b></div>
<div>Big Data | Hadoop | Hive</div>
</div></div><p>Desired Candidate Profile
</p><div><div><b>Qualifications</b> : BACHELOR OF TECHNOLOGY</div></div></div><div></div>

Big Data

Next Gen Software Solutions

<div><p><b>Job Title: Data Engineer (Spark, Hadoop, OzoneCH)
</b></p><p><b>Location: Berkley Heights, NJ (5 Days Onsite)</b></p>
<p><b>Type: Contract W2 only</b></p>
<p><b>Duration: Long term</b></p>
<p></p><p><b>Role Overview:</b></p>
<p>We are seeking a highly skilled Big Data Engineer with strong experience in Apache Spark, Hadoop ecosystem, and Apache Ozone. The ideal candidate will design, develop, and optimize large-scale data processing systems, ensuring high performance, scalability, and reliability for enterprise-level applications.</p>
<p><b>Key Responsibilities:</b></p>
<ul><li>Design and implement distributed data processing solutions using Apache Spark and Hadoop</li>
<li>Develop and maintain Spark applications for data transformation, aggregation, and ETL processes using Scala, Java, or Python</li>
<li>Utilize Apache Ozone for storing large-scale datasets, ensuring efficient data access and management in a distributed environment</li>
<li>Manage and optimize HDFS and Apache Ozone, Kafka for scalable and fault-tolerant storage.</li>
<li>Develop ETL pipelines for batch and real-time data ingestion and transformation.</li>
<li>Implement and ensure data validation, data security, integrity, and compliance across big data platforms.</li>
<li>Monitor and troubleshoot performance issues in large-scale clusters.</li>
<li>Collaborate with data scientists, analysts, and application teams to deliver high-quality data solutions.</li>
<li>Automate workflows and improve operational efficiency using scripting and orchestration tools.</li>
</ul><p><b>Required Skills &amp; Qualifications:</b></p>
<ul><li><b>Strong expertise in Apache Spark (Core, SQL, Streaming).</b></li>
<li><b>Hands-on experience with Hadoop ecosystem (HDFS, YARN, MapReduce).</b></li>
<li><b>Proficiency in Apache Ozone for object storage and integration with Hadoop.</b></li>
<li>Solid programming skills in <b>Java, Scala, or Python.</b></li>
<li>Experience with <b>Hive, HBase, and Kafka is a plus</b>.</li>
<li>Knowledge of cluster management and resource optimization.</li>
<li>Familiarity with Linux/Unix environments and shell scripting.</li>
<li>Understanding of data security, governance, and compliance standards.</li>
</ul><ul><li>Experience with cloud-based big data platforms</li>
<li>Exposure to containerization (Docker, Kubernetes) for big data workloads.</li>
<li>Knowledge of CI/CD pipelines for data engineering projects.</li>
</ul><p><b>Behavioral Skills:</b></p>
<ul><li>Good Communication skills</li>
<li>5 days' Work from Office at Berkley Heights, NJ</li>
<li>Team Player</li>
<li>Ability to work in a changing environment</li>
<li>Strong problem solving and analytical skills</li>
<li>Ability to work independently or within a team</li>
<li>Manage day-to-day challenges and communicate developmental risks with the technical team</li>
</ul><p><b>Qualifications:</b></p>
<ul><li>Bachelor's degree in computer science, Software Engineering, or a related field.</li>
<li>Proficiency in business process modeling and documentation tools.</li>
<li>Product implementation experience is preferred</li>
</ul><p><b>About Next Gen Software Solutions LLC:</b></p>
<p>Next Gen Software Solutions is a trusted provider of IT Staffing and consulting services dedicated to empowering businesses with cutting-edge technology solutions and exceptional talent. We specialize in delivering tailored IT consulting services, innovative software solutions, and connecting businesses with highly skilled IT professionals. Founded and led by a dedicated U.S. Army solider, Next Gen Software Solutions is deeply rooted in the core values of integrity, discipline, commitment, and experience-principles that guide every aspect of our operations.</p>
<p><b>Equal Employment Opportunity Statement:</b></p>
<p>Next Gen Software Solutions LLC is an Equal Opportunity Employer. We are committed to fostering an inclusive and diverse workplace where all employees and applicants are treated respect and dignity. We do not discriminate based on race, colour, religion, sex (including pregnancy, sexual orientation, or gender identity), national origin, age, genetic information, veteran status, or any other legally protected characteristic under applicable federal, state, or local laws.</p></div>

Data Engineer (Spark, Hadoop, OzoneCH) |Berkley Heights, NJ- 5 Days Onsite | Contract W2

Search 3314+ Spark developer jobs near you this March. New roles added regularly across leading industries. Apply today.