ai infrastructure engineer

Cantina Labs

<div><div><h2 class="jobSectionHeader">Location</h2>
Remote (U.S. or Europe)
</div><div><h2 class="jobSectionHeader">Employment Type</h2>
Full time
</div><div><h2 class="jobSectionHeader">Department</h2>
Research</div></div>
 <div><div>About Cantina:
Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning.
If you're excited about the potential AI has to shape human creativity and social interactions, join us in building the future! 
<div></div>About the Role:
We are looking for an MLOps Engineer to build and scale the inference infrastructure for our generative audio models, including Text-to-Speech (TTS), voice conversion, and Automatic Speech Recognition (ASR). You will be responsible for designing and deploying high-performance systems that ensure low-latency, reliable, and scalable model serving for both streaming and batch inference. This role is central to bridging the gap between research and production, ensuring our audio models are optimized for performance and cost-efficiency as we scale.
What You&rsquo;ll Do:
<ul><li>Design and maintain inference infrastructure for generative audio model architectures.
</li><li>Implement and manage high-performance inference engines.
</li><li>Orchestrate service deployments using Kubernetes (K8S), implementing advanced autoscaling paradigms to handle varying traffic loads efficiently.
</li><li>Develop and automate robust CI/CD pipelines to streamline the testing and deployment of model artifacts and inference configurations.
</li><li>Monitor production systems, establishing observability practices to track latency, resource utilization, and overall model performance.
</li><li>Collaborate closely with research teams to optimize model serving paths and evaluate various inference strategies.
</li><li>Optimize inference performance for both streaming and batch applications.
</li></ul>What You&rsquo;ll Bring:
<ul><li>Deep understanding of modern audio model architectures (e.g., TTS, ASR) and their specific inference requirements.
</li><li>Strong hands-on experience with Kubernetes (K8S), container orchestration, and implementing autoscaling strategies for production workloads.
</li><li>Solid background in MLOps, including CI/CD automation and managing scalable cloud infrastructure.
</li><li>Proficiency in software engineering principles and experience with Python or Go for infrastructure tooling and backend services.
</li><li>Experience with GPU-accelerated inference and performance profiling techniques.
</li><li>Familiarity with high-performance inference engines (e.g., Triton Inference Server, vLLM-Omni) is a plus.
</li></ul>Compensation:
The anticipated annual base salary range for this role is between $125,000-$165,000 (&euro;110,000-&euro;145,000). When determining compensation, a number of factors will be considered, including skills, experience, job scope, location, and competitive compensation market data. 
<div></div>Benefits for U.S.-based roles:
<ul><li>Competitive salary and generous company equity
</li><li>Medical, dental, and vision insurance &ndash; 99.99% of premiums covered by Cantina
</li><li>42 days of paid time off, including:
<ul><li>15 PTO days
</li><li>10 sick days
</li><li>15 company holidays
</li><li>2 floating holidays
</li></ul></li><li>Generous parental leave &amp; fertility support
</li><li>401(k) retirement savings plan
</li><li>Lifestyle spending account &ndash; $500/month to use however you&rsquo;d like
</li><li>Complimentary lunch and snacks for in-office employees
</li><li>One Medical membership, and more!</li></ul></div></div>

Machine Learning Engineer, Ops

Speechify

<div>The mission of Speechify is to make sure that reading is never a barrier to learning.

Over 50 million people use Speechify's text-to-speech products to turn whatever they're reading &ndash; PDFs, books, Google Docs, news articles, websites &ndash; into audio, so they can read faster, read more, and remember more. Speechify's text-to-speech reading products include its iOS app, Android App, Mac App, Chrome Extension, and Web App. Google recently named Speechify the Chrome Extension of the Year and Apple named Speechify its 2025 Design Award winner for Inclusivity.
Today, nearly 200 people around the globe work on Speechify in a 100% distributed setting &ndash; Speechify has no office. These include frontend and backend engineers, AI research scientists, and others from Amazon, Microsoft, and Google, leading PhD programs like Stanford, high growth startups like Stripe, Vercel, Bolt, and many founders of their own companies.

Overview

We're looking to hire for our Data side of our AI team at Speechify. This role is responsible for all aspects of data collection to support our model training operations. We are able to build high-quality datasets at petabyte-scale and low cost through a tight integration of infrastructure, engineering, and research work. We are looking for a skilled Software Engineer to join us.

What You'll Do
<ul>
<li>Be scrappy to find new sources of audio data and bring it into our ingestion pipeline</li>
<li>Operate and extend the cloud infrastructure for our ingestion pipeline, currently running on GCP and managed with Terraform.</li>
<li>Collaborate closely with our Scientists to shift the cost/throughput/quality frontier, delivering richer data at bigger scale and lower cost to power our next-generation models.</li>
<li>Collaborate with others on the AI Team and Speechify Leadership to craft the AI Team's dataset roadmap to power Speechify's next-generation consumer and enterprise products.</li>
</ul>
An Ideal Candidate Should Have
<ul>
<li>BS/MS/PhD in Computer Science or a related field.</li>
<li>5+ years of industry experience in software development.</li>
<li>Proficiency with bash/Python scripting in Linux environments</li>
<li>Proficiency in Docker and Infrastructure-as-Code concepts and professional experience with at least one major Cloud Provider (we use GCP)</li>
<li>Experience with web crawlers, large-scale data processing workflows is a plus</li>
<li>Ability to handle multiple tasks and adapt to changing priorities.</li>
<li>Strong communication skills, both written and verbal.</li>
</ul>
What we offer
<ul>
<li>A fast-growing environment where you can help shape the company and product.</li>
<li>An entrepreneurial-minded team that supports risk, intuition, and hustle.</li>
<li>A hands-off management approach so you can focus and do your best work.</li>
<li>An opportunity to make a big impact in a transformative industry.</li>
<li>Competitive salaries, a friendly and laid-back atmosphere, and a commitment to building a great asynchronous culture.</li>
<li>Opportunity to work on a life-changing product that millions of people use.</li>
<li>Build products that directly impact and support people with learning differences like dyslexia, ADD, low vision, concussions, autism, and more.</li>
<li>Work in one of the fastest-growing sectors of tech, the intersection of artificial intelligence and audio.</li>
</ul>
Compensation: The United States base salary range for this full-time position is $140,000-$200,000 + bonus + equity depending on experience

Think you're a good fit for this job?

Tell us more about yourself and why you're interested in the role when you apply.
 
And don't forget to include links to your portfolio and LinkedIn.

Not looking but know someone who would make a great fit?

Refer them!

Speechify is committed to a diverse and inclusive workplace.

Speechify does not discriminate on the basis of race, national origin, gender, gender identity, sexual orientation, protected veteran status, disability, age, or other legally protected status.</div>

Software Engineer, Data Infrastructure & Acquisition - Honolulu, HI, USA

Zscaler

<div><div>About Zscaler

Zscaler accelerates digital transformation to ensure our customers can be more agile, efficient, resilient, and secure. As an AI-forward enterprise, we are constantly pushing the envelope, leveraging the world's largest security data lake to power our cloud-native Zero Trust Exchange platform. This innovation protects our customers from cyberattacks and data loss by securely connecting users, devices, and applications in any location.

Here, impact in your role matters more than title and trust is built on results. We say, impact over activity. We seek innovators who actively use AI to amplify their impact and who thrive in an environment where we leverage intelligent systems to stay ahead of evolving threats. We believe in transparency and value constructive, honest debate&mdash;we're focused on getting to the best ideas, faster. We build high-performing teams that can make an impact quickly and with high quality. To do this, we are building a culture of execution centered on customer obsession, collaboration, ownership, and accountability.

We value high-impact, high-accountability with a sense of urgency where you're enabled to do your best work and embrace your potential. If you're driven by purpose, thrive on solving complex challenges, and want to be part of the team that's helping to secure the AI age, we invite you to bring your talents to Zscaler and help shape the future of cybersecurity.</div>
Role

We are looking for a Principal, AI Research Engineer to join our AI &amp; Data Protection team. This role is hybrid based in San Jose, CA, and reports to the EVP, AI Security and Strategic Initiatives.

Zscaler's mission is to secure the world's data in the age of AI. As the leader of the world's largest security cloud, the Zero Trust Exchange, we are uniquely positioned to define how the global enterprise safely adopts, scales, and secures Artificial Intelligence. You will be the primary architect of Zscaler's AI ecosystem, bridging the gap between our core Zero Trust platform and the rapidly evolving world of LLMs, GPU infrastructure, and AI application frameworks. This role is ideal for a technical leader who deeply understands the &quot;AI Stack&quot; and has a proven track record of building thriving developer and partner ecosystems at scale.

What you'll do (Role Expectations)
<ul>
<li>Define the AI Ecosystem Strategy by leading the vision for integrations with frontier model providers, vector database companies, and AI infrastructure leaders</li>
<li>Lead the &quot;Secure AI&quot; roadmap by owning the lifecycle for features that enable safe use of 3rd-party LLMs and prevent data leakage</li>
<li>Build a developer ecosystem by defining and launching APIs, SDKs, and integration patterns for &quot;Zero Trust-native&quot; applications</li>
<li>Drive market leadership and cross-functional partnerships to turn complex technical integrations into high-growth go-to-market motions</li>
</ul>
Who You Are (Success Profile)
<ul>
<li>You thrive in ambiguity. You're comfortable building the path as you walk it. You thrive in a dynamic environment, seeing ambiguity not as a hindrance, but as the raw material to build something meaningful.</li>
<li>You act like an owner. Your passion for the mission fuels your bias for action. You operate with integrity because you genuinely care about the outcome. True ownership involves leveraging dynamic range: the ability to navigate seamlessly between high-level strategy and hands-on execution.</li>
<li>You are a problem-solver. You love running towards the challenges because you are laser-focused on finding the solution, knowing that solving the hard problems delivers the biggest impact.</li>
<li>You are a high-trust collaborator. You are ambitious for the team, not just yourself. You embrace our challenge culture by giving and receiving ongoing feedback&mdash;knowing that candor delivered with clarity and respect is the truest form of teamwork and the fastest way to earn trust.</li>
<li>You are a learner. You have a true growth mindset and are obsessed with your own development, actively seeking feedback to become a better partner and a stronger teammate. You love what you do and you do it with purpose.</li>
</ul>
What We're Looking for (Minimum Qualifications)
<ul>
<li>8+ years of experience in technical leadership or engineering partnerships with a history of building impactful industry relationships</li>
<li>Deep technical fluency in the AI lifecycle including training vs. inference, GPU roles, context windows, and RAG architectures</li>
<li>Ability to apply first-principles thinking to fundamental security and networking challenges in enterprise LLM deployment</li>
<li>Strong bias for action with a &quot;Ship Early, Iterate Fast&quot; mentality and focus on delivering high-value MVPs</li>
<li>Foundational understanding of AI/ML technologies and experience leveraging, securing, or positioning AI-driven solutions to optimize outcomes within your functional domain</li>
</ul>
What Will Make You Stand Out (Preferred Qualifications)
<ul>
<li>Proven track record of building thriving developer and partner ecosystems at scale</li>
<li>Deep understanding of the full &quot;AI Stack&quot; from inference and RAG to data governance</li>
<li>Experience in high-velocity environments where AI security and networking solutions are evolving daily</li>
</ul>
#LI-Hybrid

#LI-CM3<div>
At Zscaler, we are committed to building a team that reflects the communities we serve and the customers we work with. We foster an inclusive environment that values all backgrounds and perspectives, emphasizing collaboration and belonging. Join us in our mission to make doing business seamless and secure.

Our Benefits program is one of the most important ways we support our employees. Zscaler proudly offers comprehensive and inclusive benefits to meet the diverse needs of our employees and their families throughout their life stages, including:
<div>
<ul><li>Various health plans</li>
<li>Time off plans for vacation and sick time</li>
<li>Parental leave options</li>
<li>Retirement options</li>
<li>Education reimbursement</li>
<li>In-office perks, and more!</li>
</ul>
<div>Learn more about Zscaler's hybrid working model and benefits here.
</div>
</div>By applying for this role, you adhere to applicable laws, regulations, and Zscaler policies, including those related to security and privacy standards and guidelines.

Zscaler is committed to providing equal employment opportunities to all individuals. We strive to create a workplace where employees are treated with respect and have the chance to succeed. All qualified applicants will be considered for employment without regard to race, color, religion, sex (including pregnancy or related medical conditions), age, national origin, sexual orientation, gender identity or expression, genetic information, disability status, protected veteran status, or any other characteristic protected by federal, state, or local laws. See more information by clicking on the Know Your Rights: Workplace Discrimination is Illegal link.

Pay Transparency

Zscaler complies with all applicable federal, state, and local pay transparency rules.

Zscaler is committed to providing reasonable support (called accommodations or adjustments) in our recruiting processes for candidates who are differently abled, have long term conditions, mental health conditions or sincerely held religious beliefs, or who are neurodivergent or require pregnancy-related support.</div></div>

Principal, AI Research Engineering

Indeed

Our Mission As the world&rsquo;s number 1 job site*, our mission is to help people get jobs. We strive to cultivate an inclusive and accessible workplace where all people feel comfortable being themselves. We're looking to grow our teams with more people who share our enthusiasm for innovation and creating the best experience for job seekers. (*Comscore, Total Visits, March 2026) Day to Day As a Software Engineer IV on the AI Infrastructure team, you will own the design, development, and operation of foundational AI platform systems that support critical AI products and workflows across Indeed. You will take on broad technical ownership for critical AI services, ensuring these systems are reliable, scalable, maintainable, and ready to support growing adoption across job seeker and employer AI products. In this role, you will work closely with engineers, product managers, data scientists, SRE partners, and dependent product teams to translate AI platform requirements into well-designed solutions. You will help drive architecture and execution for multiple AI Infrastructure projects and related platform capabilities. You will identify opportunities to improve developer productivity, system reliability, operational excellence, and the overall quality of Indeed&rsquo;s AI product development lifecycle. Responsibilities Own the design, development, and evolution of complex systems, frameworks, or platforms. Drive technical decision-making, balancing short-term delivery with long-term maintainability and scalability. Architect new solutions, evaluate trade-offs, and validate ideas through prototyping, experimentation, or iteration on existing systems. Participate in and influence code and design reviews across teams to uphold high engineering standards. Identify performance, reliability, and scalability improvements and drive enhancements to existing systems. Mentor and guide other engineers, supporting technical growth and best practices across teams. Communicate clearly and effectively with engineers, product managers, and other business partners to align on technical direction and execution. Skills/Competencies Requires a Bachelor&rsquo;s degree in Computer Science, Mathematics, Computer Engineering, Electrical Engineering, or related field and a minimum of 8 years of related experience; or a Master&rsquo;s degree with a minimum of 6 years of experience; or a PhD with 3 years experience Proficiency in software design, data structures, algorithms, and computer science fundamentals. Experience designing, building, and operating scalable, reliable software systems or platforms. Demonstrated ownership and accountability for technical outcomes and system quality. Excellent collaboration and communication skills, with the ability to influence technical direction across teams. Salary Range Transparency Tier 1 - United States of America 140,000 - 210,000 USD per year Tier 2 - United States of America 155,000 - 233,000 USD per year Tier 3 - United States of America 171,000 - 257,000 USD per year Tier 4 - United States of America N/A Tier 5 - United States of America 195,000 - 293,000 USD per year Salary Range Disclaimer The salary range for this role reflects the minimum and maximum compensation for the role. Offers are typically made between the range minimum and the range midpoint. Actual compensation will be determined based on job-related skills, experience, and expertise, as evaluated during the interview process. The range(s) listed is just one component of Indeed's total compensation package for employees. Other rewards may include quarterly bonuses, Restricted Stock Units (RSUs), a Paid Time Off policy, and many region-specific benefits. Compensation may also vary based on where a role is performed, as work locations are grouped into geographic pay tiers to reflect cost of labor differences in different geographic markets. Candidates can view geographic pay tiers by location on our career site (https://www.indeed.com/careers/paytiers), and recruiters can confirm how location is considered for a specific role. Benefits - Health, Work/Life Harmony, &amp; Wellbeing We care about what you care about. We have a multitude of benefits to support Indeedians, as well as their pets, kids, and partners including medical, dental, vision, disability and life insurance. Indeedians are able to enroll in our company&rsquo;s 401k plan, as well as an equity-based incentive program. Indeedians will also receive open paid time off, 11 paid holidays a year and up to 26 weeks of paid parental leave. For more information, select your country and learn more about our employee benefits, program, &amp; perks at https://www.indeed.com/careers/benefits! Equal Opportunities and Accommodations Statement Indeed is deeply committed to building a workplace and global community where inclusion is not only valued, but prioritized. We&rsquo;re proud to be an equal opportunity employer, seeking to create a welcoming and diverse environment. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, family status, marital status, sexual orientation, national origin, genetics, neuro-diversity, disability, age, or veteran status, or any other non-merit based or legally protected grounds. Indeed provides reasonable accommodations to qualified individuals with disabilities in the employment application process. To request an accommodation, please visit https://www.indeed.com/careers/accommodations. If you are requesting accommodation for an interview, please reach out at least one week in advance of your interview. For more information about our commitment to equal opportunity/affirmative action, please visit our Careers page (https://www.indeed.com/careers). Inclusion &amp; Belonging Inclusion and belonging are fundamental to our hiring practices and company culture, forming an integral part of our vision for a better world of work. At Indeed, we&rsquo;re committed to the wellbeing of our employees and on a mission to make this the best place to work and thrive. We believe that fostering an inclusive environment where every employee feels respected and accepted benefits everyone, fueling innovation and creativity. We value diverse experiences, including those who have had prior contact with the criminal legal system. We are committed to providing individuals with criminal records, including formerly incarcerated individuals, a fair chance at employment. Those with military experience are encouraged to apply. Equivalent expertise demonstrated through a combination of work experience, training, military experience, or education is welcome. Indeed&rsquo;s Employee Recruiting Privacy Policy Like other employers Indeed uses our own technologies to help us find and attract top talent from around the world. In addition to our site&rsquo;s user and privacy policy found at https://www.indeed.com/legal, we also want to make you aware of our recruitment specific privacy policy found at https://www.indeed.com/legal/indeed-jobs. Agency Disclaimer Indeed does not pay placement fees for unsolicited resumes or referrals from non-candidates, including search firms, staffing agencies, professional recruiters, fee-based referral services, and recruiting agencies (each individually, an &quot;Agency&quot;), subject to local laws. An Agency seeking a placement fee must obtain advance written approval from Indeed's internal Talent Acquisition team and execute a fee agreement with Indeed for each job opening before making a referral or submitting a resume for that opening. AI Notice Indeed is committed to ensuring fairness and transparency throughout our hiring process. We use artificial intelligence (AI) tools to assist in the screening, assessment, and selection of applicants for this position by analyzing information provided in resumes and applications. Our use of AI does not replace human decision-making. Unless otherwise notified, Indeed does not use AI constituting an AEDT or an ADMT as those tools are defined in applicable laws. Reference ID: 47399 The deadline to apply to this position is 8/8/2026. Job postings may be extended at the hiring team&rsquo;s discretion based on applicant volume. It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability. Reference ID: 47399

Staff Software Engineer- AI Platform

NVIDIA

<div>NVIDIA is hiring experienced software engineers to help scale up its AI Infrastructure. We expect you to have significant software engineering experience with cluster operations, operator development, node health monitoring and working with GPU resource scheduling. We welcome out-of-the-box thinkers who can provide new ideas with strong execution bias. Expect to be constantly challenged, improving, and evolving for the better. You will help advance NVIDIA's capacity to build and deploy leading infrastructure solutions for a broad range of AI-based applications. If you're creative, passionate about GPUs, and love having fun, please apply today! 
For two decades, we have pioneered visual computing, the art and science of computer graphics. With the invention of the GPU - the engine of modern visual computing - the field has expanded to encompass video games, movie production, product design, medical diagnosis and scientific research. Today, we stand at the beginning of the next era, the AI computing era, ignited by a new computing model, GPU deep learning. 
What you will be doing:<ul><li>
You will be part of an DGX Cloud team responsible for production systems that enable large scalable GPU clusters to be used for a variety of AI workloads.</li><li>
Designing and developing a massively distributed scalable platform which would be used to identify, diagnose and remediate non-performant GPU assets.</li><li>
Working with teams across NVIDIA to ensure production AI clusters run reliability and consistently with maximum performance. Evaluating system failures and improving services based on a well-defined incident management process.</li><li>
Working across all of our product stack: React, Web Components, TypeScript, Golang, PostgreSQL, Temporal, Bazel, Kubernetes</li></ul> 
What we need to see:<ul><li>
Direct experience in a software engineering role within a highly technical organization with demonstrable impact from your work.</li><li>
Highly motivated with strong communication skills, you can work successfully with multi-functional teams, principles, and architects and coordinate effectively across organizational boundaries and geographies.</li><li>
5+ years in similar role and experience on large-scale production systems. Experience with common software engineering principles, tools and techniques</li><li>
You possess a BS in Computer Science or Engineering or equivalent experience</li><li>
6+ years of experience doing full-stack engineering</li><li>
3+ years building and shipping consumer-facing products</li><li>
Proficiency in React, TypeScript/JavaScript, and Golang</li><li>
Proficiency with a SQL database</li></ul> 
Ways to stand out from the crowd:<ul><li>
Technical competency in managing and automating large-scale distributed systems independent of cloud providers. Advanced hands-on experience and deep understanding of cluster management systems (Kubernetes, Slurm, Base Command Manager).</li><li>
Empathy for users, attention to detail, and a passion for creating world-class user experiences</li><li>
Prior experience in asynchronous workflows and/or event driven architecture.</li><li>
Proven operational excellence in maintaining reliable and performant infrastructure.</li><li>
A good understanding of how to use LLMs responsibly and the perils of blindly consuming their output</li></ul> 
NVIDIA is widely considered to be one of the technology world&rsquo;s most desirable employers. We have some of the most forward-thinking and hardworking people on the planet working for us. If you are creative and autonomous, we want to hear from you!
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 184,000 USD - 287,500 USD for Level 4, and 224,000 USD - 356,500 USD for Level 5.<div><div><div><div><div><div><div>
You will also be eligible for equity and benefits.</div></div></div></div></div></div></div>
Applications for this job will be accepted at least until August 3, 2026.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.
NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.</div>

Senior Full Stack Software Engineer - DGX Cloud

DigitalOcean

<div><div>Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you'll find your place here. We value winning together&mdash;while learning, having fun, and making a profound difference for the dreamers and builders in the world.</div>
DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. We are seeking a Senior Engineer 2 to join our AI Inference Data Plane team. In this role, you will be a key technical leader responsible for designing, developing, and delivering high-scale, resilient data plane services that power our &quot;Inference as a Service&quot; offering. You will work at the intersection of distributed systems and specialized AI hardware to ensure our customers can deploy and scale their models with industry-leading performance and reliability. This is a hands-on role, requiring you to be able to develop high quality software while availing of all the productivity boosts granted by the latest AI coding agents.
<h2 class="jobSectionHeader">
What You'll Do:</h2>
<ul><li>Technical Leadership: Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models.</li>
<li>System Design: Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards.</li>
<li>Performance Optimization: Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing.</li>
<li>Collaboration: Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs.</li>
<li>Distributed Serving at Scale: Build on Kubernetes-native distributed inference frameworks like llm-d (or alternatives such as NVIDIA Dynamo, Ray Serve, KServe) to deliver prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models.</li>
<li>Flow Control &amp; Load Balancing: Solve the distributed-systems problems unique to LLM serving &mdash; inference-aware load balancing on queue depth, cache locality, and predicted latency; flow control and fairness across tenants; autoscaling inference pools; and moving gigabytes of KV-cache between prefill and decode instances with negligible overhead.</li>
<li>Open Source Contributions: Contribute upstream to llm-d, vLLM, and the inference gateway ecosystem, and represent DigitalOcean in these communities.</li>
<li>Mentorship: Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement.</li>
<li>Operational Excellence: Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health.</li>
</ul><h2 class="jobSectionHeader">What You'll Bring to DigitalOcean:</h2>
<ul><li>AI/ML Domain Knowledge: Hands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or TensorRT.</li>
<li>Inference Frameworks: Familiarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray Serve.</li>
<li>Inference Engine Depth: Hands-on experience with vLLM or alternatives (SGLang, TensorRT-LLM, TGI, Modular MAX), including internals like continuous batching, paged attention, and prefix caching.</li>
<li>Distributed Inference Fluency: Understanding of why cluster-scale serving is hard: KV-cache locality is partitioned across workers, naive round-robin routing destroys cache hit rates and tail latency, and disaggregated prefill/decode requires fast cross-pod KV transfer (e.g., NIXL).</li>
<li>Upstream Track Record: Merged contributions to vLLM, llm-d, SGLang, or similar projects strongly preferred.</li>
<li>Architecture Proficiency: Knowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization).</li>
<li>Software Engineering: Expert-level proficiency in GoLang or Python and familiarity with gRPC.</li>
<li>Cloud Operations: Proven experience shipping customer-facing software products and running critical services in a high-scale environment similar to DigitalOcean.</li>
<li>Open Source Mindset: Experience integrating and building with open-source software.</li>
</ul><h2 class="jobSectionHeader">Compensation Range:</h2>
<ul><li>$139,200 - $174,000</li>
</ul>
<ul><li>This is a remote role</li></ul>

JR: 2026-7624

#LI-Remote<div><h2 class="jobSectionHeader">
Why You'll Like Working for DigitalOcean</h2>
<ul><li>We innovate with purpose. You'll be a part of a cutting-edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.</li>
<li>We prioritize career development. At DO, you'll do the best work of your career. You will work with some of the smartest and most interesting people in the industry. We are a high-performance organization that will always challenge you to think big. Our organizational development team will provide you with resources to ensure you keep growing. We provide employees with reimbursement for relevant conferences, training, and education. All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.</li>
<li>We care about your well-being. Regardless of your location, we will provide you with a competitive array of benefits to support you from our Employee Assistance Program to Local Employee Meetups to flexible time off policy, to name a few. While the philosophy around our benefits is the same worldwide, specific benefits may vary based on local regulations and preferences.</li>
<li>We reward our employees. The salary range for this position is based on market data, relevant years of experience, and skills. You may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance. We also provide equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.</li>
<li>DigitalOcean is an equal-opportunity employer. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.</li>
</ul>
Application Limit: You may apply to a maximum of 3 positions within any 180-day period. This policy promotes better role-candidate matching and encourages thoughtful applications where your qualifications align most strongly.</div></div>

Senior Software Engineer I - AI Inference Data Plane

<div><div>Dive in and do the best work of your career at DigitalOcean. Journey alongside a strong community of top talent who are relentless in their drive to build the simplest scalable cloud. If you have a growth mindset, naturally like to think big and bold, and are energized by the fast-paced environment of a true industry disruptor, you'll find your place here. We value winning together&mdash;while learning, having fun, and making a profound difference for the dreamers and builders in the world.</div>
DigitalOcean is expanding its AI Infrastructure layer to support the next generation of AI-driven applications. We are seeking a Senior Engineer 2 to join our AI Inference Data Plane team. In this role, you will be a key technical leader responsible for designing, developing, and delivering high-scale, resilient data plane services that power our &quot;Inference as a Service&quot; offering. You will work at the intersection of distributed systems and specialized AI hardware to ensure our customers can deploy and scale their models with industry-leading performance and reliability. This is a hands-on role, requiring you to be able to develop high quality software while availing of all the productivity boosts granted by the latest AI coding agents.
<h2 class="jobSectionHeader">
What You'll Do:</h2>
<ul><li>Technical Leadership: Act as a technical leader on the team, driving the end-to-end design, development, and delivery of critical data plane components hosting large generative AI models.</li>
<li>System Design: Architect and refine system design proposals for our high-scale, multi-tenant AI inference cloud ecosystem, ensuring they meet rigorous availability and resiliency standards.</li>
<li>Performance Optimization: Implement and optimize distributed inference hosting using techniques like tensor/data parallelism, KV cache optimizations, and smart routing.</li>
<li>Collaboration: Work cross-functionally with Product Managers, customer-facing teams, and other engineering teams to align technical roadmaps with customer needs.</li>
<li>Distributed Serving at Scale: Build on Kubernetes-native distributed inference frameworks like llm-d (or alternatives such as NVIDIA Dynamo, Ray Serve, KServe) to deliver prefill/decode disaggregation, KV-cache-aware routing, tiered prefix caching, and wide expert parallelism for MoE models.</li>
<li>Flow Control &amp; Load Balancing: Solve the distributed-systems problems unique to LLM serving &mdash; inference-aware load balancing on queue depth, cache locality, and predicted latency; flow control and fairness across tenants; autoscaling inference pools; and moving gigabytes of KV-cache between prefill and decode instances with negligible overhead.</li>
<li>Open Source Contributions: Contribute upstream to llm-d, vLLM, and the inference gateway ecosystem, and represent DigitalOcean in these communities.</li>
<li>Mentorship: Coach and mentor junior engineers, fostering a culture of technical excellence and continuous improvement.</li>
<li>Operational Excellence: Maintain and operate critical, high-scale services, utilizing observability tools and defining SLOs to ensure superior platform health.</li>
</ul><h2 class="jobSectionHeader">What You'll Bring to DigitalOcean:</h2>
<ul><li>AI/ML Domain Knowledge: Hands-on experience hosting large language or multimodal models using inference engines like vLLM, SGLang, or TensorRT.</li>
<li>Inference Frameworks: Familiarity with distributed inference serving frameworks such as llm-d, NVIDIA Dynamo, or Ray Serve.</li>
<li>Inference Engine Depth: Hands-on experience with vLLM or alternatives (SGLang, TensorRT-LLM, TGI, Modular MAX), including internals like continuous batching, paged attention, and prefix caching.</li>
<li>Distributed Inference Fluency: Understanding of why cluster-scale serving is hard: KV-cache locality is partitioned across workers, naive round-robin routing destroys cache hit rates and tail latency, and disaggregated prefill/decode requires fast cross-pod KV transfer (e.g., NIXL).</li>
<li>Upstream Track Record: Merged contributions to vLLM, llm-d, SGLang, or similar projects strongly preferred.</li>
<li>Architecture Proficiency: Knowledge of common LLM architectures and optimization techniques (e.g., continuous batching, quantization).</li>
<li>Software Engineering: Expert-level proficiency in GoLang or Python and familiarity with gRPC.</li>
<li>Cloud Operations: Proven experience shipping customer-facing software products and running critical services in a high-scale environment similar to DigitalOcean.</li>
<li>Open Source Mindset: Experience integrating and building with open-source software.</li>
</ul><h2 class="jobSectionHeader">Compensation Range:</h2>
<ul><li>$139,200 - $174,000</li>
</ul>
<ul><li>This is a hybrid role</li></ul>

JR: 2026-7624

#LI-Hybrid<div><h2 class="jobSectionHeader">
Why You'll Like Working for DigitalOcean</h2>
<ul><li>We innovate with purpose. You'll be a part of a cutting-edge technology company with an upward trajectory, who are proud to simplify cloud and AI so builders can spend more time creating software that changes the world. As a member of the team, you will be a Shark who thinks big, bold, and scrappy, like an owner with a bias for action and a powerful sense of responsibility for customers, products, employees, and decisions.</li>
<li>We prioritize career development. At DO, you'll do the best work of your career. You will work with some of the smartest and most interesting people in the industry. We are a high-performance organization that will always challenge you to think big. Our organizational development team will provide you with resources to ensure you keep growing. We provide employees with reimbursement for relevant conferences, training, and education. All employees have access to LinkedIn Learning's 10,000+ courses to support their continued growth and development.</li>
<li>We care about your well-being. Regardless of your location, we will provide you with a competitive array of benefits to support you from our Employee Assistance Program to Local Employee Meetups to flexible time off policy, to name a few. While the philosophy around our benefits is the same worldwide, specific benefits may vary based on local regulations and preferences.</li>
<li>We reward our employees. The salary range for this position is based on market data, relevant years of experience, and skills. You may qualify for a bonus in addition to base salary; bonus amounts are determined based on company and individual performance. We also provide equity compensation to eligible employees, including equity grants upon hire and the option to participate in our Employee Stock Purchase Program.</li>
<li>DigitalOcean is an equal-opportunity employer. We do not discriminate on the basis of race, religion, color, ancestry, national origin, caste, sex, sexual orientation, gender, gender identity or expression, age, disability, medical condition, pregnancy, genetic makeup, marital status, or military service.</li>
</ul>
Application Limit: You may apply to a maximum of 3 positions within any 180-day period. This policy promotes better role-candidate matching and encourages thoughtful applications where your qualifications align most strongly.</div></div>

Senior Engineer, Inference Data Plane

Search 4505+ Ai infrastructure engineer jobs near you this August. New roles added regularly across leading industries. Apply today.