big data software engineer

Apple

<div>Apple is where individual imaginations gather together, committing to the values that lead to great work. Every new product we build, service we create, or Apple Store experience we deliver is the result of us making each other&rsquo;s ideas stronger. That happens because every one of us shares a belief that we can make something wonderful and share it with the world, changing lives for the better. It&rsquo;s the diversity of our people and their thinking that inspires the innovation that runs through everything we do. When we bring everybody in, we can do the best work of our lives. Here, you&rsquo;ll do more than join something - you&rsquo;ll add something.<br>
<br>
As a BigData Software Engineer, you are an integral part of a small data-centric team driving large-scale data infrastructure and processes development, implementation, and improvement. Our organization thrives on collaborative partnerships. Join and play a key role in developing and driving the adoption of Data Mesh and data-centric micro-services. Apple's Manufacturing Systems and Infrastructure (MSI) team is responsible for capturing, consolidating and tracking all manufacturing data for Apple&rsquo;s products and modules worldwide.
<br>
<br>
Our tools enable teams to confidently use data to shape the next generation of product manufacturing at Apple. We seek a practitioner with experience building large-scale data platforms, analytic tools, and solutions. If you are passionate about making data easily accessible, trusted, and available across the entire business at scale, we'd love to hear from you.<br>
<br>
<b>Description</b><br>
As a BigData Engineer, you will lead the development and implementation of our cloud-based data infrastructure. You will work closely with cross-functional teams to understand data requirements, design scalable solutions, and ensure the integrity and availability of our data. The ideal candidate will have a deep understanding of cloud technologies, data engineering best practices, and a proven track record of successfully delivering complex data projects.<br>
<br>
<ul><li>Lead the design and implementation of cloud-based data architectures.</li></ul><br>
<ul><li>Collaborate with data scientists, analysts, and business stakeholders to understand data requirements.</li></ul><br>
<ul><li>Implement data security and privacy best practices.</li></ul><br>
<ul><li>Optimize data storage, retrieval, and processing for performance and cost efficiency.</li></ul><br>
<ul><li>Lead efforts to ensure data quality, consistency, and accuracy.</li></ul><br>
<ul><li>Mentor and provide technical guidance to junior data engineers.</li></ul><br>
<ul><li>Stay current with industry trends and emerging technologies in cloud data engineering.</li></ul><br>
<b>
Preferred Qualifications</b><br>
Certification in cloud platforms (AWS Certified Big Data - Specialty, Microsoft Certified: Azure Data Engineer, etc.)<br>
Experience building data streaming solutions using Apache Spark, Storm, Link, Flume<br>
Experience using ETL tools and technologies like Reltio, Stibo, Informatics, Apache Atlas<br>
Knowledge of machine learning and data science concepts<br>
Contributions to the open source community<br>
Familiarity with data governance and compliance<br>
Strong problem-solving and analytical skills<br>
Excellent communication and leadership skills<br>
<br>
<b>Minimum Qualifications</b><br>
Bachelor&rsquo;s or Master's degree in Computer Science or a related field<br>
Proven experience as a Big Data developer, with a focus on cloud platforms<br>
Strong proficiency in cloud services such as AWS, Azure, or Google Cloud<br>
Expertise in data modelling and data integration<br>
Experience with big data technologies (Hadoop, Spark, etc.)<br>
Proficient in programming languages such as Python, Java, Scala, GoLang<br>
Knowledge of data warehouse solutions (Redshift, BigQuery, Snowflake, Druid)<br>
Experience with streaming data technologies (Kafka, Kinesis)&quot;,&quot;internalDetails&quot;:null,&quot;eeoContent&quot;:null</div>

Big Data Software Engineer - Manufacturing Systems & Infrastructure

Infinity Quest

<p>10 Experience</p><p>Job Details:<br/>Must Have Skills<br/>Big data</p><p>Client name : ltmindtree and microsoft</p><p>work type : Bellevue, WA &ndash; Remote</p><p>synapse and pyspark</p><p>Detailed Job Description<br/>Design the architecture of big data platformPerform and oversee tasks such as writing scripts, calling APIs, web scraping, and writing SQL queriesDesign and implement data stores that support the scalable processing and storage of our highfrequency dataMaintain our data pipelineCustomize and oversee integration tools, warehouses, databases, and analytical systemsConfigure and provide availability for dataaccess tools used by all data scientists</p><p>Job Type: Contract</p><p>Pay: From $65.00 per hour</p><p>Experience:</p><ul><li>synapse: 10 years (Preferred)</li><li>pyspark: 10 years (Preferred)</li><li>Big Data Engineer: 10 years (Preferred)</li></ul><p>Work Location: Remote</p>

Sr Big Data/Data Engineer

EPAM Systems

<div><p>We are looking for a skilled <b>Big Data DevOps Engineer </b>with profound knowledge of cloud technologies and experience with a variety of data tools and platforms.</p><p>
The ideal candidate will have proven experience in managing and optimizing big data operations across cloud environments.</p><p><b>
Responsibilities</b></p><ul><li>
Cloud Solutions Development: Build, deploy, and maintain scalable and secure big data solutions across major cloud platforms (AWS, Azure, Google Cloud) to support advanced data processing and analytics</li><li>
Data Tools Management: Configure and manage data processing tools such as Apache Spark, Kafka, Airflow, and Databricks to efficiently handle large-scale data workflows</li><li>
Automation and CI/CD: Utilize Terraform for infrastructure as code, automate workflows with Jenkins, and maintain code with GitLab for continuous integration and continuous delivery processes</li><li>
Performance Optimization: Monitor and optimize the performance of big data tools and cloud platforms, ensuring cost-effectiveness and operational efficiency</li><li>
Security and Compliance: Implement and enforce data security measures, ensuring compliance with data protection regulations and best practices in cloud environments</li><li>
Collaborative Development: Work collaboratively with data scientists, data engineers, and other stakeholders to understand requirements and deliver high-quality data solutions</li></ul><p><b>
Requirements</b></p><ul><li>
Strong experience with big data technologies, particularly Apache Spark or Databricks</li><li>
Extensive background in cloud services administration with platforms like AWS, Azure, or Google Cloud</li><li>
In-depth knowledge of infrastructure as code using Terraform</li><li>
Proficiency in implementing CI/CD pipelines with tools such as Jenkins, GitLab or GitHub Actions</li><li>
Solid scripting skills in Python, Scala, or similar languages</li><li>
Experience with SQL and relational/non-relational databases</li><li>
Excellent problem-solving abilities and a team-oriented attitude</li></ul><p><b>
Nice to have</b></p><ul><li>
Certifications in cloud technology (e.g., AWS Certified Solutions Architect, Google Cloud Certified Professional Cloud Architect)</li><li>
Additional experience with real-time data processing tools like Apache Kafka</li></ul><p><b>
We offer</b></p><ul><li>
CONTINUOUS UPSKILLING, LEARNING &amp; DEVELOPMENT<ul><li>
Diversity of tasks and projects</li><li>
Assessment center for objective review of competency level</li><li>
Personal development plan</li><li>
Mentoring programs and leadership development</li><li>
Certification and professional development support</li><li>
Access to learning platforms including more than 2,500 internal courses and the LinkedIn Learning library with 20,000+ courses</li><li>
English courses taught by certified teachers</li></ul></li><li>
CORPORATE BENEFITS<ul><li>
Extra leave days</li><li>
Referral bonuses</li></ul></li><li>
COMPENSATION PACKAGE<ul><li>
Competitive compensation paid in USD</li><li>
Regular salary and performance reviews</li></ul></li><li>
MEDICAL &amp; HEALTHCARE<ul><li>
Private health insurance</li><li>
Well-being events</li></ul></li><li>
WORKING ENVIRONMENT<ul><li>
Recreation areas and kitchens</li><li>
Tea, coffee, and snacks</li><li>
Well-being events</li><li>
Sports equipment and game consoles</li><li>
IT Equipment</li><li>
Microsoft's Software Assurance Home Use Program (HUP)</li></ul></li></ul><p><i>
EPAM is a leading global provider of digital platform engineering and development services. We are committed to having a positive impact on our customers, our employees, and our communities. We embrace a dynamic and inclusive culture. Here you will collaborate with multi-national teams, contribute to a myriad of innovative projects that deliver the most creative and cutting-edge solutions, and have an opportunity to continuously learn and grow. No matter where you are located, you will join a dedicated, creative, and diverse community that will help you discover your fullest potential.</i></p></div>

Middle Big Data DevOps Engineer

Vodafone

<div><div><div><div><h2 class="jobSectionHeader"><b>Who we are</b></h2>
</div><div><p>
VOIS (Vodafone Intelligent Solutions) is a strategic arm of Vodafone Group Plc, creating value for customers by delivering intelligent solutions through Talent, Technology &amp; Transformation.<br>
As the largest shared services organisation in the global telco industry with 30,000 FTE, our portfolio of next-generation solutions and services are designed in partnership with customers across Vodafone Group, local markets, and partner markets to simplify and drive growth. With our strategic partner Accenture, we work alongside our Vodafone customers, other Telco and tech companies to drive transformation, meet the challenges of our industry and ensure we stay relevant and resilient. This partnership is a unique, industry-first model which brings together the best of in-house and 3rd party capability.<br>
We work with customers across 28 countries from 10 VOIS locations: Albania, Egypt, Hungary, India, Romania, Spain, Turkey, UK, Germany, Ireland, and with a network of teams in Czech Republic, Italy, Greece, and Portugal.<br>
#VOIS #BeUnrivalled #CreateTheFuture</p></div></div><div><div><h2 class="jobSectionHeader"><b>
About this Role</b></h2>
</div><div><p>
We are seeking a skilled Big Data Engineer to join our Data and Analytics (DNA) team on a 6-month contract basis. Based in Pune, Bangalore, or Ahmedabad, this role is pivotal in designing and maintaining scalable big data solutions that support efficient data processing and analytics. The ideal candidate will collaborate with cross-functional teams to build robust data pipelines and leverage big data technologies to drive data-driven decision-making across the organisation.</p></div></div><div><div><h2 class="jobSectionHeader"><b>
What you&rsquo;ll do</b></h2>
</div><div><ul><li>Design, develop, and maintain scalable data pipelines for processing structured and unstructured data.</li><li>
Build and optimise data storage solutions including distributed file systems, NoSQL databases, and data warehouses.</li><li>
Utilise big data frameworks such as Apache Hadoop, Spark, Flink, and Kafka to develop data processing workflows.</li><li>
Ensure high performance and scalability through parallel processing, distributed computing, and caching strategies.</li><li>
Implement monitoring, alerting, and automated scaling mechanisms for big data systems.</li><li>
Maintain data quality and governance by applying validation, cleansing, and enrichment processes.</li><li>
Collaborate with data scientists, analysts, and engineers to align data solutions with business objectives.</li><li>
Document system architecture and processes, and provide training and support to stakeholders.</li></ul></div></div><div><div><h2 class="jobSectionHeader"><b>
Who you are</b></h2>
</div><div><ul><li>Graduate or Postgraduate in Computer Science, Engineering, or a related technical field.</li><li>
Over 5 years of experience in building cloud-based data engineering solutions.</li><li>
Proficient in SQL and programming languages such as Python, Java, or Scala.</li><li>
Hands-on experience with big data technologies including Apache Hadoop, Spark, Flink, and Kafka.</li><li>
Familiar with cloud platforms such as AWS, GCP, or Azure.</li><li>
Experienced in streaming tools like Amazon Kinesis, Apache Kafka, Flink, or Pulsar.</li><li>
Strong understanding of distributed systems and database management principles.</li><li>
Excellent communication and collaboration skills with a proactive approach to problem-solving.</li><li>
Ability to work independently and manage multiple priorities in a fast-paced environment.</li><li>
Familiarity with ITIL practices and experience in large global organisations is desirable.</li><li>
</li></ul></div></div><div><div><h2 class="jobSectionHeader"><b>
Not a perfect fit?</b></h2>
</div><div><p>
Concerned you may not meet every requirement? Vodafone is committed to creating an inclusive workplace where everyone can thrive. If you are excited about this role but your experience does not align exactly with every aspect of the job description, you are encouraged to apply. You may be the right candidate for this or another opportunity, and the recruitment team will support you in exploring where your skills fit best.</p></div></div><div><div><h2 class="jobSectionHeader"><b>
What's in it for you</b></h2>
</div><div><ul><li>Opportunity to work on cutting-edge big data technologies and cloud platforms.</li><li>
Exposure to large-scale data engineering projects within a global telecom environment.</li><li>
Collaborative work culture with cross-functional teams.</li><li>
Flexible work location options including Pune, Bangalore, and Ahmedabad.</li><li>
Be part of a dynamic team driving innovation and digital transformation.</li></ul></div></div><div><div><h2 class="jobSectionHeader"><b>
What skills you will learn</b></h2>
</div><div><ul><li>Advanced big data processing using Apache Hadoop, Spark, Flink, and Kafka.</li><li>
Cloud-based data engineering practices across AWS, GCP, or Azure.</li><li>
Real-time data streaming and processing with tools like Kinesis and Pulsar.</li><li>
Best practices in data governance, quality assurance, and system monitoring.</li><li>
Cross-functional collaboration and agile delivery in a global enterprise environment.</li></ul></div></div><div><div><h2 class="jobSectionHeader"><b>
VOIS Equal Opportunity Employer Commitment</b></h2>
</div><div><p>
Vodafone recognises and celebrates the value of diversity in building a workforce that reflects the customers and communities it serves. No form of discrimination is tolerated. This includes, but is not limited to, discrimination based on race, colour, age, veteran status, gender identity, gender expression, sexual orientation, pregnancy, maternity or parental status, ethnicity, disability, religion or belief, political affiliation, trade union membership, nationality, citizenship, indigenous status, medical condition, HIV status, neurodiversity, social origin, cultural background, marital or civil partnership status, or socio-economic background.</p></div></div><div><div><h2 class="jobSectionHeader"><b>
Join Us</b></h2>
</div><div><p>
At Vodafone, we&rsquo;re working hard to build a better future. A more connected, inclusive and sustainable world. As a dynamic global community, it's our human spirit, together with technology, that empowers us to achieve this.<br>
We challenge and innovate in order to connect people, businesses, and communities across the world. Delighting our customers and earning their loyalty drive us, and we experiment, learn fast and get it done, together.<br>
With us, you can be truly be yourself and belong, share inspiration, embrace new opportunities, thrive, and make a real difference.</p></div></div><div><div><h2 class="jobSectionHeader"><b>
Alert</b></h2>
</div><div><p>
Apply for Vodafone jobs only through the official Vodafone Careers website to avoid job scams and fraud.<br>
#JDEnhancedByTARA</p></div></div><div><div><h2 class="jobSectionHeader"><b>
Follow us on social media and #StayConnected</b></h2>
</div><div><ul><li>LinkedIn</li><li>
Facebook</li><li>
Instagram</li><li>
You can also chat with our employees to learn more about our projects: Click here</li></ul></div></div></div></div>

Big Data Engineer

LiveRamp

<div><div>LiveRamp is the data collaboration platform of choice for the world&rsquo;s most innovative companies. A groundbreaking leader in consumer privacy, data ethics, and foundational identity, LiveRamp is setting the new standard for building a connected customer view with unmatched clarity and context while protecting precious brand and consumer trust. LiveRamp offers complete flexibility to collaborate wherever data lives to support the widest range of data collaboration use cases&mdash;within organizations, between brands, and across its premier global network of top-quality partners.</div><div></div><div>
Hundreds of global innovators, from iconic consumer brands and tech giants to banks, retailers, and healthcare leaders turn to LiveRamp to build enduring brand and business value by deepening customer engagement and loyalty, activating new partnerships, and maximizing the value of their first-party data while staying on the forefront of rapidly evolving compliance and privacy requirements.</div><div><br>
LiveRamp is looking for a Senior Backend Engineer (SDE) to join our team focused on building scalable, high-performance backend systems that power our data collaboration platform.
</div><div></div><div>About you:</div><ul><li><div>
5+ years of Java programming experience</div></li><li><div>
3+ years of experience on streaming technologies like Red Panda</div></li><li><div>
High focus on quality, performance and cost of big data systems.</div></li><li><div>
Familiarity with with both batch and stream based data processing.</div></li><li><div>
Familiarity with big data databases like SingleStore / Big Query, relational databases.</div></li><li><div>
Familiarity with cloud computing systems like GCP.</div></li><li><div>
Familiarity with data-observability mechanisms and tools.</div></li><li><div>
Familiarity with big data processing technologies like Apache Spark</div></li></ul><div><br>
Benefits:</div><ul><li><div>
Flexible paid time off, paid holidays, options for working from home, and paid parental leave.</div></li><li><div>
Comprehensive Benefits Package: LiveRamp offers a comprehensive benefits package designed to help you be your best self in your personal and professional lives. Our benefits package offers medical, dental, vision, accident, life and disability, an employee assistance program, voluntary benefits as well as perks programs for your healthy lifestyle, career growth, and more. Your medical benefits extend to your dependents including parents.</div></li></ul><div></div><div>
More about us:</div><div></div><div><i>
LiveRamp&rsquo;s mission is to connect data in ways that matter, and doing so starts with our people. We know that inspired teams enlist people from a blend of backgrounds and experiences. And we know that individuals do their best when they not only bring their full selves to work but feel like they truly belong. Connecting LiveRampers to new ideas and one another is one of our guiding principles&mdash;one that informs how we hire, train, and grow our global team across nine countries and four continents. Click </i><div><i>here </i></div><i>to learn more about Diversity, Inclusion, &amp; Belonging (DIB) at LiveRamp.</i></div><div><br>
To all recruitment agencies: LiveRamp does not accept agency resumes. Please do not forward resumes to our jobs alias, LiveRamp employees or any other company location. LiveRamp is not responsible for any fees related to unsolicited resumes.</div></div>

Senior Backend Engineer (Big Data)

Recruitment Hub 365

<p><b>Job Title: Big Data Hadoop Developer</b><br/><b>Location: Hyderabad</b><br/><b>Work mode: 5 Days - (WFO)</b><br/><b>Experience Required: 4+ years</b><br/><b>Key Skills: Big Data, Hadoop, Spark, Hive</b></p><p><b>Job Overview</b><br/>We are looking for an experienced Big Data Hadoop Developer to design, build, and optimize large-scale data processing systems. The ideal candidate should have strong expertise in the Hadoop ecosystem, hands-on experience with Spark and Hive, and the ability to develop efficient ETL pipelines.</p><p><b>Key Responsibilities</b></p><ul><li>Design, develop, and maintain scalable big data systems and data pipelines.</li><li>Implement data processing frameworks using <b>Hadoop, Spark, and Hive</b>.</li><li>Develop and manage ETL workflows to ensure data accuracy, consistency, and availability.</li><li>Collaborate with data architects, analysts, and business teams to translate requirements into robust big data solutions.</li><li>Perform data validation, quality checks, and issue resolution for data pipelines.</li><li>Optimize data storage, query performance, and cluster utilization in the Hadoop ecosystem.</li><li>Ensure compliance with security, governance, and data management standards.</li></ul><p><b>Required Skills &amp; Qualifications</b></p><ul><li>4+ years of hands-on experience in <b>Big Data technologies</b>.</li><li>Strong understanding of the <b>Hadoop ecosystem</b> (HDFS, MapReduce, YARN).</li><li>Proficiency in <b>Apache Spark</b> (batch and streaming) and <b>Hive</b>.</li><li>Experience in building and maintaining data pipelines and ETL processes.</li><li>Strong knowledge of data optimization, partitioning, and performance tuning.</li><li>Familiarity with NoSQL databases such as <b>HBase, Cassandra, or MongoDB</b> is an advantage.</li><li>Experience with programming/scripting languages: <b>Java, Scala, Python, or Shell</b>.</li><li>Strong analytical and problem-solving skills.</li></ul><p>Job Type: Full-time</p><p>Pay: &#8377;1,500,000.00 - &#8377;2,400,000.00 per year</p><p>Application Question(s):</p><ul><li>How many years of experience you have with Big Data, Hadoop, Spark, Hive?</li><li>Mention your Notice Period, Current CTC and ECTC.</li><li>How many years of experience you have with Big Data / Hadoop Developer?</li></ul><p>Work Location: In person</p>

Big Data/Hadoop Developer

<div></div><div><div><div><div><ul><div>5 - 7 Years</div>
<div></div><div>5 Openings</div>
 <div>Hyderabad</div></ul></div><p></p><div><br>
<div><div>
</div></div></div></div><div></div><div><h3 class="jobSectionHeader"><b>Role description</b></h3>
<div><p><b>Key Responsibilities</b></p>
<div>
<ul><li>Develop and maintain ETL pipelines for data ingestion, transformation, and storage.</li></ul>
<ul><li>Design and optimize data models for analytics and reporting.</li></ul>
<ul><li>Ensure data quality, integrity, and security across all systems.</li></ul>
<ul><li>Collaborate with data scientists, analysts, and software engineers to deliver reliable data solutions.</li></ul>
<ul><li>Monitor and improve data workflows for performance and scalability.</li></ul>
</div>
<p><b>Required Skills</b></p>
<div>
<ul><li>Proficiency in SQL and experience with relational and NoSQL databases.</li></ul>
<ul><li>Hands-on experience with data pipeline tools (e.g., Apache Spark, Airflow).</li></ul>
<ul><li>Strong programming skills in Python or Java.</li></ul>
<ul><li>Familiarity with cloud platforms (AWS, Azure, or GCP).</li></ul>
<ul><li>Knowledge of data warehousing and big data technologies.</li></ul>
</div></div>
<h3 class="jobSectionHeader"><b>Skills</b></h3>
<div><p>Pyspark,Python,Aws</p></div></div></div></div><p></p>
<br><div><h3 class="jobSectionHeader"><b>About UST</b></h3>
<div>UST is a global digital transformation solutions provider. For more than 20 years, UST has worked side by side with the world&rsquo;s best companies to make a real impact through transformation. Powered by technology, inspired by people and led by purpose, UST partners with their clients from design to operation. With deep domain expertise and a future-proof philosophy, UST embeds innovation and agility into their clients&rsquo; organizations. With over 30,000 employees in 30 countries, UST builds for boundless impact&mdash;touching billions of lives in the process.</div></div>

Lead I - Data Engineering (Pyspark,Python,Aws)

Search 10000+ Big data software engineer jobs near you this November. New roles added regularly across leading industries. Apply today.