data platform engineer

Accenture

<p><b>Project Role : </b>Data Platform Engineer<br>
<b>Project Role Description : </b>Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
<br>
<b>Must have skills : </b>Databricks Unified Data Analytics Platform<br>
<b>Good to have skills : </b>NA<br>
Minimum <b>12</b> year(s) of experience is required<br>
<b>Educational Qualification : </b>15 years full time education<br>
<br>
Summary: Seeking a forward-thinking professional with an AI-first mindset to design, develop, and deploy enterprise-grade solutions using Generative and Agentic AI frameworks that drive innovation, efficiency, and business transformation. As a Data Platform Engineer, you will assist with the data platform blueprint and design, collaborating with Integration Architects and Data Architects to ensure cohesive integration between systems and data models. You will play a crucial role in shaping the data platform components. Roles and Responsibilities: Lead AI-driven solution design and delivery by applying GenAI and Agentic AI to address complex business challenges, automate processes, and integrate intelligent insights into enterprise workflows for measurable impact. &bull; Lead the design, development, and optimization of complex data pipelines, ensuring high performance and scalability using Py Spark, Spark, and Big Data technologies. &bull; Architect data engineering solutions across major cloud platforms like AWS, Azure, or GCP, enabling smooth and secure data ingestion, transformation, and storage. &bull; Lead efforts to design and build modular, reusable ETL/ELT pipelines to integrate data from various sources into cloud-based data lakes and data warehouses. &bull; Collaborate closely with cross-functional teams, including data scientists, analysts, and other engineering teams to develop and maintain large-scale data processing systems. &bull; Guide and mentor junior engineers in best practices for data engineering, cloud technologies, and big data solutions. &bull; Optimize and tune Spark and big data processing jobs to handle high-volume data efficiently and at scale. &bull; Ensure data security, privacy, and governance are maintained by enforcing best practices and policies across cloud platforms. &bull; Implement automated data testing and monitoring solutions to ensure data quality and pipeline reliability. &bull; Drive initiatives for real-time data processing, batch jobs, and analytics pipelines to support both operational and analytical needs. &bull; Advocate for continuous improvement, researching new tools, and technologies to drive data engineering excellence. Required Skills &amp; Qualifications: Strong grasp of Generative and Agentic AI, prompt engineering, and AI evaluation frameworks. Ability to align AI capabilities with business objectives while ensuring scalability, responsible use, and tangible value realization. &bull; Proven experience with AWS, Azure, or GCP in designing data engineering workflows and solutions. &bull; Expert-level knowledge of Py Spark, Spark, and Big Data technologies. &bull; Extensive experience in data pipeline architecture, design, and optimization. &bull; Strong hands-on experience with large-scale ETL/ELT processes and integrating data into cloud-based platforms like S3, Blob Storage, Big Query, and Redshift. &bull; Proficiency in Python, SQL, and scripting for data pipeline development. &bull; Ability to work with real-time data streaming tools like Kafka and batch processing tools. &bull; Experience leading cross-functional teams in the implementation of cloud-based data platforms and workflows. &bull; Solid understanding of data security, compliance, and governance in a cloud environment.Nice to Have:&bull;The candidate should have a minimum of 12 years of experience in Databricks Unified Data Analytics Platform. &bull; Familiarity with Dbt (Data Build Tool) for managing data transformation and data quality. &bull; Exposure to CI/CD pipelines, DevOps practices, and containerization technologies (Docker, Kubernetes). &bull; Certifications or experience with data tools like Apache Airflow, Five Tran, Informatica, or Talend. &bull; Experience with analytics and BI tools like Power BI, Tableau, or Looker for visualization over cloud-based data platforms. Educational Qualification:- 15 years full time education is required.</p>
<br>
15 years full time education

Data Platform Engineer

<p><b>Project Role : </b>Data Platform Engineer<br>
<b>Project Role Description : </b>Assists with the data platform blueprint and design, encompassing the relevant data platform components. Collaborates with the Integration Architects and Data Architects to ensure cohesive integration between systems and data models.
<br>
<b>Must have skills : </b>Data Science<br>
<b>Good to have skills : </b>Generative AI<br>
Minimum <b>7.5</b> year(s) of experience is required<br>
<b>Educational Qualification : </b>15 years full time education<br>
<br>
SUMMARY: We are seeking a Senior Technical &ndash; Gen AI Lead with deep expertise in Python, AI/ML concepts, and data engineering to spearhead the design and development of Data Agents powered by Agentic AI. As a senior technical leader, you will drive solution architecture, guide teams, and ensure the successful delivery of AI-enabled initiatives that align with business goals. The ideal AI Lead combines technical excellence, leadership ability, prompt engineering skills, and a strong product mindset to ensure solutions are impactful, scalable, and business driven. ROLES AND RESPONSIBILITIES: 1. Solution Architecture &amp; Delivery &bull; Lead the design, architecture, and implementation of AI-enabled solutions powered by Agentic AI. &bull; Ensure scalability, reliability, and alignment of solutions with business objectives. 2. Team Leadership &amp; Guidance &bull; Mentor and guide cross-functional teams in AI/ML, data engineering, and product development. &bull; Foster a collaborative environment that drives innovation and high-quality delivery. 3. AI/ML &amp; Data Engineering Expertise &bull; Spearhead the development of intelligent Data Agents leveraging Python, AI/ML models, and scalable data pipelines. &bull; Apply advanced prompt engineering techniques to optimize the performance of LLMs and agentic workflows. 4. Collaboration &amp; Stakeholder Engagement &bull; Work closely with product managers, data scientists, and engineering teams to define AI-driven use cases. &bull; Translate business goals into technical requirements and actionable AI solutions. 5. Innovation &amp; Emerging Technologies &bull; Stay ahead of trends in Gen AI, Agentic AI, ML, and autonomous systems. &bull; Introduce best practices, frameworks, and tools to enhance solution effectiveness and maintain competitive advantage. 6. Business-Driven Mindset &bull; Align AI solutions with organizational strategy to maximize impact and business value. &bull; Ensure delivered solutions are scalable, secure, and capable of driving measurable business outcomes. TECHNICAL EXPERIENCE: &bull; Min 7 years of professional experience in the required skills - having min 2 years of Gen AI experience &bull; Lead the design and development of Data Agents leveraging Agentic AI principles, Python, and AI/ML frameworks. &bull; Strong expertise in Python and Py Spark for large-scale data engineering. &bull; Solid foundation in AI/ML concepts (e.g., supervised/unsupervised learning, reinforcement learning, LLMs). &bull; Experience architecting and deploying AI-driven solutions in production environments. &bull; Proven leadership skills in managing teams, mentoring talent, and leading technical delivery. &bull; Strong prompt engineering skills to refine LLM interactions and improve system outcomes. &bull; Excellent communication and stakeholder management skills, with the ability to bridge business and technology. &bull; Product mindset with a focus on usability, scalability, and delivering business impact. &bull; Growth mindset and adaptability to thrive in an evolving digital and AI landscape &bull; Define and own the technical architecture for scalable, secure, and efficient agent driven systems. &bull; Apply prompt engineering techniques to optimize LLM performance and enhance agent effectiveness. &bull; Guide and mentor analysts and developers in Python, Py Spark, AI/ML best practices, and agentic system design. &bull; Partner with product managers, data scientists, and business stakeholders to translate requirements into AI-driven solutions. &bull; Drive a product-oriented approach, ensuring AI agents deliver measurable value aligned with business outcomes. &bull; Ensure code quality, performance optimization, and adherence to engineering best practices. &bull; Stay ahead of emerging trends in Agentic AI, LLMs, and autonomous systems to bring innovation into practice. &bull; Lead proofs-of-concept (POCs) and enterprise-scale implementations of AI-based automation. ADDITIONAL ATTRIBUTES: &bull; &bull; Hands-on experience with LLM frameworks (e.g., Lang Chain, Llama Index) and Agentic AI orchestration tools. &bull; &bull; Familiarity with cloud AI/ML platforms (AWS SageMaker, Azure ML, GCP Vertex AI). &bull; Exposure to data governance, security, and compliance practices in enterprise AI systems. &bull; &bull; Experience with visualization and reporting tools to support data-driven decision making. EDUCATION QUALIFICATIONS: &bull; A 15-year full-time education is required</p>
<br>
15 years full time education

Tala

<div><div><b>About Tala</b></div><div></div><div><br>
At Tala, we&rsquo;re applying advanced technology and human creativity to solve what legacy institutions can&rsquo;t or won't. We are a global financial infrastructure company on a mission to unleash the economic power of the global majority, recognizing that today&rsquo;s financial infrastructure doesn&rsquo;t work for most of the world&rsquo;s population.</div><div></div><div><br>
We&rsquo;re the first and only platform to combine the intelligence of a credit bureau, the payments execution of a fintech, and the relationship expertise of a bank into one vertically integrated solution. Our platform is powered by an expansive moat of proprietary data and AI/ML decisioning technology, enabling us to deliver instant, reliable liquidity personalized to every customer's needs. Through our flagship credit app, we&rsquo;ve disbursed over $7 billion in credit to more than 12 million customers across Latin America, Southeast Asia, and East Africa. These customers have leveraged Tala products to start and expand small businesses, manage day-to-day needs, and pursue their financial goals.</div><div></div><div><br>
Our pioneering work and proven impact have earned us consistent recognition, including being named to:</div><div></div><div><br>
CNBC&rsquo;s Disruptor 50 for five years.</div><div>
CNBC&rsquo;s World's Top Fintech Companies for two consecutive years.</div><div>
Forbes&rsquo; Fintech 50 list for nine consecutive years.</div><div></div><div><br>
Visionary investors, persuaded by the economic power of the global majority, have committed half a billion dollars in equity and debt to Tala's success.</div><div></div><div><br>
Given the global nature of our team, we operate on a remote-first approach with office hubs in Santa Monica, CA (HQ); Nairobi, Kenya; Mexico City, Mexico; Manila, the Philippines; and Bangalore, India.<br>
</div><div></div><br>
<div>Most Talazens join us because they connect with our mission. If you are energized by the impact you can make at Tala, we&rsquo;d love to hear from you!</div><div></div><div><br>
The Role<br>
</div><div></div><br>
<br>
<div>The Data Platform Engineering team is building the foundation for a best-in-class decision engine&mdash;empowering data scientists, machine learning engineers, and analytics teams to deliver insights and intelligence at scale.</div><div>
As a Data Platform Engineer, you&rsquo;ll design, build, and evolve the infrastructure and tooling that enable our internal teams to move fast, work efficiently, and make data-driven decisions. You&rsquo;ll collaborate closely with cross-functional partners to ensure reliability, scalability, and ease of use across the data ecosystem. You&rsquo;ll also help guide backend and frontend teams in adopting an Event-Driven Data Mesh architecture, ensuring consistency and high-quality data flows throughout the organization.</div>
<h3 class="jobSectionHeader"><b>What You'll Do
</b></h3><ul><li>Design, build, and maintain scalable data infrastructure to power analytics, machine learning, and business intelligence use cases.</li><li>
Develop and evolve data pipelines and streaming systems using technologies like Kafka, Spark, AWS Kinesis, and Flink.</li><li>
Implement infrastructure as code using Terraform or CloudFormation across AWS, GCP, or Azure environments.</li><li>
Partner with Data Scientists, MLEs, and Analytics Engineers to ensure data accessibility, quality, and reliability.</li><li>
Champion best practices for DataOps, observability, and reliability within the data platform.</li><li>
Contribute to architectural decisions and help drive adoption of an Event-Driven Data Mesh across teams.</li></ul><h3 class="jobSectionHeader"><b>
What You'll Need
</b></h3><ul><li>5+ years of experience in software engineering, with strong proficiency in Python.</li><li>
3+ years of experience building and maintaining streaming data systems (Kafka, Spark, AWS Kinesis, Flink).</li><li>
Proven experience deploying infrastructure as code using Terraform or CloudFormation.</li><li>
Background in a software or platform engineering environment, ideally supporting internal developer or data teams.</li><li>
Advanced degree in Engineering or a related technical field.</li><li>
Prior experience working remotely in a distributed team environment.</li></ul><br>
<div></div><ul><li>
Preferred Experience (Nice to Have)</li><li>
Experience building and maintaining data lakes, ideally using Delta Lake or Apache Iceberg.</li><li>
Experience collaborating with Machine Learning Engineers or Data Scientists on Feature Store design and maintenance.</li><li>
Familiarity with SRE principles and DevOps/DataOps best practices.</li></ul><div>
Our vision is to build a new financial ecosystem where everyone can participate on equal footing and access the tools they need to be financially healthy. We strongly believe that inclusion fosters innovation and we&rsquo;re proud to have a diverse global team that represents a multitude of backgrounds, cultures, and experience. We hire talented people regardless of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status.</div>
<p>
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.</p></div>

Senior Data Platform Engineer

Quizlet

<div><div><b>About Quizlet:</b></div><div></div><div><br>
At Quizlet, our mission is to help every learner achieve their outcomes in the most effective and delightful way. Our $1B+ learning platform serves tens of millions of students every month, including two-thirds of U.S. high schoolers and half of U.S. college students, powering over 2 billion learning interactions monthly.</div><div></div><div><br>
We blend cognitive science with machine learning to personalize and enhance the learning experience for students, professionals, and lifelong learners alike. We&rsquo;re energized by the potential to power more learners through multiple approaches and various tools.</div><div></div><div><b><br>
Let&rsquo;s Build the Future of Learning</b></div><div>
Join us to design and deliver AI-powered learning tools that scale across the world and unlock human potential.</div><div></div><div><br>
About the Team:</div><div></div><div><br>
The AI &amp; Data Platform team's mission is to build and maintain a rock-solid, scalable, and efficient foundation for all data and machine learning operations at Quizlet. This team is the bedrock upon which our AI ambitions are built. As we consolidate our AI efforts to create a unified and powerful product portfolio, the platform team's primary focus is on achieving operational excellence. We aggressively buy and leverage best-in-class MLOps and DataOps technologies to accelerate the pace of innovation across Quizlet.</div><div></div><div><br>
About the Role:</div><div></div><div><br>
As an AI &amp; Data Platform Engineer, you will help lead the design and development of Quizlet&rsquo;s analytical data and machine learning infrastructure, ensuring our platforms are scalable, reliable, and designed for long-term success. You&rsquo;ll partner closely with cross-functional teams to define technical direction, guide architectural decisions, and mentor engineers across Data Science, ML Engineering, and Product. This role requires strong technical depth, strategic thinking, and the ability to deliver systems that accelerate ML and Data innovation. You will measure success through platform adoption, reliability improvements, and developer productivity outcomes.</div><div></div><div><br>
We&rsquo;re happy to share that this is an<b> onsite position</b> in our San Francisco office. To help foster team collaboration, we require that employees be in the office a <b>minimum of three days per week</b>: Monday, Wednesday, and Thursday and as needed by your manager or the company. We believe that this working environment facilitates increased work efficiency, team partnership, and supports growth as an employee and organization.</div>
<h3 class="jobSectionHeader"><b>In this role, you will:</b></h3><ul><li>
Accelerate and amplify the work of our Machine Learning Engineers by building ML infrastructure that streamlines the development, testing, and deployment of machine learning models to production</li><li>
Collaborate with cross-functional teams, including Product Management, Data Analytics, and Product Engineering, to build systems that support a diverse range of data needs</li><li>
Build tools and services that improve developer efficiency and platform usability</li><li>
Enable the ingestion and processing of large-scale data, from real-time streaming to batch processing, by developing and implementing scalable and resilient services, platform, and infrastructure</li><li>
Own projects end-to-end, from conceptualization to launch and deployment</li><li>
Partner with Product Engineers and Data Scientists to ensure that critical data is delivered to its destination with maximum availability, recoverability, and operational efficiency</li><li>
Design and deliver reliable data and ML infrastructure, contributing to the team&rsquo;s roadmap and collaborating across functions to enable data-driven features and insights</li></ul><h3 class="jobSectionHeader"><b>
What you bring to the table:</b></h3><ul><li>
5-8+ years of experience building and maintaining large-scale data or ML platforms in production environments</li><li>
Experience contributing to team-wide best practices and advocating for reliability and efficiency in data workflows</li><li>
Deep experience with distributed systems, data pipelines, and ML infrastructure (e.g., Airflow, dbt, GCP, Kubernetes, feature stores)</li><li>
Excellent communication skills with the ability to influence and align diverse stakeholders</li></ul><h3 class="jobSectionHeader"><b>
Bonus points if you have:</b></h3><ul><li>
Experience owning platform architecture</li><li>
Expertise in cross-cloud data infrastructure or hybrid orchestration</li><li>
Contributions to open-source data/ML tooling or internal frameworks</li><li>
Demonstrated impact on developer experience or data reliability culture</li></ul><h3 class="jobSectionHeader"><b>
Compensation, Benefits &amp; Perks:</b></h3><ul><li>
Quizlet is an equal opportunity employer. We celebrate diversity and are committed to creating an inclusive environment for all employees. Salary transparency helps to mitigate unfair hiring practices when it comes to discrimination and pay gaps. Total compensation for this role is market competitive, including a starting base salary of $204,000 - $285,000, depending on location and experience, as well as company stock options</li><li>
Collaborate with your manager and team to create a healthy work-life balance</li><li>
20 vacation days that we expect you to take!</li><li>
Competitive health, dental, and vision insurance (100% employee and 75% dependent PPO, Dental, VSP Choice)</li><li>
Employer-sponsored 401k plan with company match</li><li>
Access to LinkedIn Learning and other resources to support professional growth</li><li>
Paid Family Leave, FSA, HSA, Commuter benefits, and Wellness benefits</li><li>
40 hours of annual paid time off to participate in volunteer programs of choice</li></ul><div><b>
Why Join Quizlet?</b></div><div></div><div><br>
 \uD83C\uDF0E Massive reach: 60M+ users, 1B+ interactions per week</div><div>
 \uD83E\uDDE0 Cutting-edge tech: Generative AI, adaptive learning, cognitive science</div><div>
 \uD83D\uDCC8 Strong momentum: Top-tier investors, sustainable business, real traction</div><div>
 \uD83C\uDFAF Mission-first: Work that makes a difference in people&rsquo;s lives</div><div>
 \uD83E\uDD1D Inclusive culture: Committed to equity, diversity, and belonging</div><div></div><div><b><i><br>
We strive to make everyone feel comfortable and welcome!</i></b></div><div>
We work to create a holistic interview process, where both Quizlet and candidates have an opportunity to view what it would be like to work together, in exploring a mutually beneficial partnership.</div><div></div><div><br>
We provide a transparent setting that gives a comprehensive view of who we are!</div><div></div><div><b><br>
In Closing:</b></div><div></div><div><br>
At Quizlet, we&rsquo;re excited about passionate people joining our team&mdash;even if you don&rsquo;t check every box on the requirements list. We value unique perspectives and believe everyone has something meaningful to contribute. Our culture is all about taking initiative, learning through challenges, and striving for high-quality work while staying curious and open to new ideas. We believe in honest, respectful communication, thoughtful collaboration, and creating a supportive space where everyone can grow and succeed together.&rdquo;</div><div></div><div><br>
Quizlet&rsquo;s success as an online learning community depends on a strong commitment to diversity, equity, and inclusion.</div><div></div><div><br>
As an equal opportunity employer and a tech company committed to societal change, we welcome applicants from all backgrounds. Women, people of color, members of the LGBTQ+ community, individuals with disabilities, and veterans are strongly encouraged to apply. Come join us!</div><div></div><div><b><br>
To All Recruiters and Placement Agencies:</b></div><div></div><div><br>
At this time, Quizlet does not accept unsolicited agency resumes and/or profiles.</div><div>
Please do not forward unsolicited agency resumes to our website or to any Quizlet employee. Quizlet will not pay fees to any third-party agency or firm nor will it be responsible for any agency fees associated with unsolicited resumes. All unsolicited resumes received will be considered the property of Quizlet.</div><div>
#LI-FT</div>
<p>
We may use artificial intelligence (AI) tools to support parts of the hiring process, such as reviewing applications, analyzing resumes, or assessing responses. These tools assist our recruitment team but do not replace human judgment. Final hiring decisions are ultimately made by humans. If you would like more information about how your data is processed, please contact us.</p></div>

Sr./Staff AI & Data Platform Engineer

Citi

<div><p><b><i>Engineer the future of global finance. At Citi, our Tech team doesn&rsquo;t just support finance &ndash; we are helping to redefine it. Every day, $5 trillion crosses through our network. We do business in 180+ countries operating at a scale few can match. From deploying advanced AI to helping shape global markets, we build systems that matter. Look to join a team where your work helps influence economies, your ideas can drive innovation and outcomes, and your growth is backed by mentorship, continuous learning and flexibility with potential hybrid work opportunities. Help solve real-world challenges that touch millions and get the opportunity to build the future of finance with Citi Tech.</i></b></p><p></p><p>
Our Data Platform is the backbone of Fixed Income Capital Markets trading, providing the low latency, high concurrency, scalability, and availability needed to power our groundbreaking solutions.</p><p><br>
Our Data Platform Engineering team is on the cutting edge. We research, adapt, and deploy the latest open-source data platforms to meet Citi's unique needs. We're a collaborative group that thrives on technical challenges and the satisfaction of building highly performant systems.</p><p><br>
We're seeking a passionate and highly skilled Lead Java Data Engineer to guide and mentor a talented team of engineers in building and maintaining Citi's next-generation data platform. If you're a natural leader with a deep understanding of Java, distributed systems, and a passion for pushing the boundaries of Big Data technology, we want to hear from you.<br>
<br>
<b>Responsibilities:</b></p><ul><li>
Technical Leadership: Provide technical leadership and mentorship to a team of data engineers, fostering a culture of collaboration, innovation, and continuous learning.</li><li>
Architecture &amp; Design: Lead the design and development of highly scalable, low-latency, fault-tolerant data pipelines and platform components that meet Citi's evolving business needs.</li><li>
Open Source Expertise: Stay abreast of emerging open-source data technologies and evaluate their suitability for integration into Citi's platform.</li><li>
Performance Optimization: Continuously identify and implement performance optimizations across the data platform to ensure optimal efficiency and responsiveness.</li><li>
Collaboration &amp; Communication: Partner closely with stakeholders across engineering, data science, and business teams to understand requirements and translate them into robust technical solutions.</li><li>
Delivery &amp; Execution: Drive the timely and high-quality delivery of data platform projects, adhering to agile methodologies and best practices.</li></ul><p><b><br>
Qualifications:</b></p><ul><li>
Java Expertise: Significant hands-on experience developing high-performance Java applications (Java 11+ preferred) with a strong foundation in core Java concepts, OOP, and OOAD.</li><li>
Data Engineering Fundamentals: Proven experience building and maintaining data pipelines using technologies like Kafka, Apache Spark, or Apache Flink. Familiarity with event-driven architectures and experience in developing real-time, low-latency applications is essential.</li><li>
Distributed Systems: Deep understanding of distributed systems concepts and experience with MPP platforms such as Trino (Presto) or Snowflake.</li><li>
Deployment &amp; Orchestration: Experience deploying and managing applications on container orchestration platforms like Kubernetes, OpenShift, or ECS.</li><li>
Leadership &amp; Communication: Demonstrated ability to lead and mentor engineering teams, communicate complex technical concepts effectively, and collaborate across diverse teams.</li><li>
Problem Solving &amp; Analytical Skills: Excellent problem-solving skills and a data-driven approach to decision-making.</li></ul><p><b><br>
Preferred:
</b></p><ul><li>Previous experience at tier-1 investment banks or major financial institutions</li><li>
Experience with real-time streaming analytics and low-latency system optimization</li></ul><p></p><p>
-</p><p><b>
Job Family Group:
</b></p>Technology<p>
-</p><p><b>
Job Family:</b></p>
Applications Development<p>
-</p><p><b>
Time Type:</b></p>
Full time<p>
-</p><p><b>
Most Relevant Skills
</b></p>Please see the requirements listed above.<p>
-</p><p><b>
Other Relevant Skills
</b></p>For complementary skills, please see above and/or contact the recruiter.<p>
-</p><p><i>
Citi is an equal opportunity employer, and qualified candidates will receive consideration without regard to their race, color, religion, sex, sexual orientation, gender identity, national origin, disability, status as a protected veteran, or any other characteristic protected by law.</i></p><p></p><p><i><br>
If you are a person with a disability and need a reasonable accommodation to use our search tools and/or apply for a career opportunity review </i><i>Accessibility at Citi</i><i>.</i><br>
<br>
<i>View Citi&rsquo;s </i><i>EEO Policy Statement</i><i> and the </i><i>Know Your Rights</i><i> poster.</i></p></div>

Big Data Platform Senior Engineer

The Modern Data Company

<div><div><h3 class="jobSectionHeader"><b>Job Information</b></h3>
<ul><div><p>Date Opened</p>
<div>11/07/2025
</div></div><div><p>Industry</p>
<div>Software Product
</div></div><div><p>Job Type</p>
<div>Full time
</div></div><div><p>City</p>
<div>Hyderabad
</div></div><div><p>State/Province</p>
<div>Telangana
</div></div><div><p>Country</p>
<div>India
</div></div><div><p>Zip/Postal Code</p>
<div>500081
</div></div></ul></div><div><h3 class="jobSectionHeader"><b>About Us</b></h3>
<div><p>The Modern Data Company is redefining data management for the AI era, transforming data from a technical challenge into a company&rsquo;s most powerful business asset. Modern's DataOS platform is the world's first operating system for data, providing a breakthrough layer that integrates with any data stack, no rip and replace.</p><p><br>
Fortune 1000+ enterprises use DataOS to scale AI and solve mission-critical data challenges. With DataOS, enterprises are accelerating AI adoption by up to 90% while reducing data consumption and platform costs by 50%. Modern&rsquo;s rapidly expanding customer base includes global category leaders across a wide range of global industries. They trust DataOS to power their AI and business transformation.</p><br>
<div></div>
</div><p></p><h3 class="jobSectionHeader"><b>Job Description</b></h3>
<div><div>Role Overview:The Senior Data Platform Engineer is an experienced professional who designs and optimizes complex data processing solutions to address sophisticated business needs. In this role, you will tackle large-scale batch and streaming data projects, ensuring that data pipelines and platforms are scalable, high-performing, and secure. A Senior Data Platform Engineer acts as a technical expert in the team, often mentoring junior engineers and leading critical implementation efforts. This role requires deep expertise in data engineering tools and frameworks &ndash; including proficiency in Apache Spark, Trino, or Apache Iceberg for big data processing and analytics &ndash; and the ability to leverage these technologies to build robust data products. This position is also client-focused, demanding excellent communication to collaborate with client stakeholders on a near-daily basis.</div><div></div><div><br>
Key Responsibilities:</div><br>
<div></div>
Architect and Optimize Data Pipelines: Lead the design, development, and optimization of large-scale data pipelines, handling both batch data movement and real-time streaming data. Ensure pipelines are built for scalability (able to handle growing data volumes) and low-latency data delivery where needed.<br>
<ul><li>
</li>
Technical Leadership: Serve as a subject matter expert in the data platform team. Review code, enforce engineering best practices, and mentor junior data engineers in improving pipeline reliability, data modeling, and coding standards. Proactively introduce improvements or new technologies to enhance the data platform.<br>
<li>
</li>
Advanced Data Processing &amp; Analytics: Utilize advanced big data frameworks and query engines (e.g., Spark, Trino, etc.) to implement complex data transformations and aggregations. For example, use Apache Spark (a unified analytics engine for large-scale data processing) or Trino (a distributed SQL query engine for big data analytics) to process and query massive datasets efficiently. Leverage Apache Iceberg (a high-performance table format for huge analytic tables) for managing large-scale data lake storage and enabling fast analytical queries.<br>
<li>
</li>
Data Flow Management: Oversee the scheduling and orchestration of data workflows to ensure timely availability of data for analytics. Implement monitoring and observability on data pipelines to detect issues, and ensure data reliability and quality for downstream use.<br>
<li>
</li>
Collaborate with Stakeholders: Work closely with data analysts, data scientists, and other engineering team members to understand data needs and ensure the platform meets business requirements. Liaise with client teams regularly to gather requirements and present data solutions that address their problems. As a senior engineer, you may lead technical discussions with clients, offering guidance on data best practices and platform capabilities.<br>
<li>
</li>
Data Governance and Security: Champion data governance standards within projects &ndash; implement data validation, access controls, and compliance measures. Ensure that the data platform adheres to security best practices and that sensitive data is handled in accordance with corporate and regulatory guidelines.<br>
<li>
</li><li>
Data Infrastructure Management: Support the underlying data infrastructure (e.g., databases, cloud storage, processing frameworks) to ensure high availability and optimal performance of data systems. Uphold best practices in data security and governance to protect sensitive information and comply with relevant policies.</li></ul><div></div><h3 class="jobSectionHeader"><b><br>
Requirements</b></h3>
Extensive Data Engineering Experience: Several years (around 5 years) of hands-on experience in data engineering or related fields. Demonstrated ability to design and manage complex data ingestion frameworks for both structured and unstructured data.<br>
<ul><li>
</li>
Expertise in Big Data Technologies: Proven expertise in one or more modern big data processing technologies. Experience with Apache Spark, Presto/Trino, or Apache Iceberg is required for this role, as you will be leveraging these systems to handle large-scale data processing and querying. Familiarity with distributed data storage and indexing techniques is expected.<br>
<li>
</li>
Proficient in SQL and Python: Mastery of SQL for complex querying (including performance tuning on large datasets) and Python for building data pipeline scripts or tooling. Ability to work with other programming languages (Scala, Java, etc.) for Spark or similar frameworks is a plus.<br>
<li>
</li>
Pipeline and API Integration Skills: Strong experience in developing ETL/ELT pipelines and using workflow orchestration tools. Comfortable working with streaming data platforms (e.g., Kafka or Flink) and event-driven architectures. Also, proficient in integrating external data sources via APIs and handling real-time data feeds.<br>
<li>
</li>
Data Architecture and Modeling: Solid understanding of data modeling principles and the design of data warehouses/data lakes. Ability to design efficient schema and storage solutions for analytics. Experience with optimizing query performance and resource utilization in big data environments.<br>
<li>
</li>
Problem Solving and Leadership: Excellent problem-solving skills applied to debugging data issues and optimizing pipeline performance. Demonstrated leadership in taking ownership of projects, and ability to communicate technical concepts to both technical teams and non-technical stakeholders clearly. Experience in a client-facing environment or consulting projects is highly valued, as the role involves guiding clients through technical decisions.<br>
<li>
</li><li>
Education: Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, Data Engineering, or a related discipline. Continuous learning mindset to stay updated with the latest data engineering technologies and best practices.</li></ul></div></div></div><p></p>

Senior Data Engineer

Virtusa

<div>
 <strong>
  About The Role
  <br/>
  <br/>
 </strong>
 We are seeking a highly skilled Data Engineer with deep expertise in PySpark and the Cloudera Data Platform (CDP) to join our data engineering team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable data pipelines that ensure high data quality and availability across the organization. This role requires a strong background in big data ecosystems, cloud-native tools, and advanced data processing techniques.
 <br/>
 <br/>
 The ideal candidate has hands-on experience with data ingestion, transformation, and optimization on the Cloudera Data Platform, along with a proven track record of implementing data engineering best practices. You will work closely with other data engineers to build solutions that drive impactful business insights.
 <br/>
 <br/>
 <strong>
  Responsibilities
  <br/>
  <br/>
 </strong>
 Data Pipeline Development: Design, develop, and maintain highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform, ensuring data integrity and accuracy.
 <br/>
 <br/>
 Data Ingestion: Implement and manage data ingestion processes from a variety of sources (e.g., relational databases, APIs, file systems) to the data lake or data warehouse on CDP.
 <br/>
 <br/>
 Data Transformation and Processing: Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs and business requirements.
 <br/>
 <br/>
 Performance Optimization: Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes.
 <br/>
 <br/>
 Data Quality and Validation: Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline.
 <br/>
 <br/>
 Automation and Orchestration: Automate data workflows using tools like Apache Oozie, Airflow, or similar orchestration tools within the Cloudera ecosystem.
 <br/>
 <br/>
 Education and Experience
 <br/>
 <br/>
 Bachelors or Masters degree in Computer Science, Data Engineering, Information Systems, or a related field.
 <br/>
 <br/>
 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform.
 <br/>
 <br/>
 Technical Skills
 <br/>
 <br/>
 PySpark: Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques.
 <br/>
 <br/>
 Cloudera Data Platform: Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase.
 <br/>
 <br/>
 Data Warehousing: Knowledge of data warehousing concepts, ETL best practices, and experience with SQL-based tools (e.g., Hive, Impala).
 <br/>
 <br/>
 Big Data Technologies: Familiarity with Hadoop, Kafka, and other distributed computing tools.
 <br/>
 <br/>
 Orchestration and Scheduling: Experience with Apache Oozie, Airflow, or similar orchestration frameworks.
 <br/>
 <br/>
 Scripting and Automation: Strong scripting skills in Linux.
</div>


Data Engineer (Pyspark)

Search 10000+ Data platform engineer jobs near you this November. New roles added regularly across leading industries. Apply today.

10000 data platform engineer jobs NEAR_ME

Data Platform Engineer

Data Platform Engineer

Senior Data Platform Engineer

Sr./Staff AI & Data Platform Engineer

Big Data Platform Senior Engineer

Senior Data Engineer

Data Engineer (Pyspark)