applied data scientist, evaluation & model behavior

data engineer

data engineer - finance Jobs in San Francisco

data engineer - finance

data scientist

applied data scientist, evaluation & model behavior Jobs in San Francisco

associate machine learning engineer Jobs in San Francisco

associate machine learning engineer

applied data scientist evaluation model behavior

AGI, Inc.

<div><h1 class="jobSectionHeader">Think Different. Build the Future.
</h1><h2 class="jobSectionHeader">Our Mission</h2>
Build everyday AGI. Trustworthy, consumer-grade agents that redefine human&ndash;AI collaboration for millions. Software shouldn&rsquo;t wait for commands; it should partner with you, amplifying what you can do every single day.<h2 class="jobSectionHeader">
Why AGI, Inc.</h2>
We&rsquo;re a stealth team of elite founders and AI researchers, with backgrounds spanning Stanford, OpenAI, and DeepMind. We&rsquo;re industry leaders in mobile and computer-use agents, bringing these capabilities to consumer scale.
Grounded in years of agent research, our AI is designed with trustworthiness and reliability as core pillars, not afterthoughts.
We are supported by tier-1 investors who funded the first generation of AI giants; now they&rsquo;re backing us to build the next: everyday AGI. (Watch the demo)
If you see possibility where others see limits, read on.<h2 class="jobSectionHeader">
About the Role</h2>
As an Applied Scientist focused on Evaluation &amp; Model Behavior, you will design and implement the systems used to measure and improve the performance of Computer Use Agents.
This is not a support role. You will be responsible for the technical definition of model quality, including the design of evaluation metrics, the curation of training datasets, and the engineering of system prompts. You'll work directly with the engineering team to translate product requirements into technical specifications and quantifiable benchmarks.
You'll focus on rigor, clarity, and impact, ensuring every metric, dataset, and prompt moves us toward more reliable, trustworthy agents.<h2 class="jobSectionHeader">
What You'll Do</h2>
Model Behavior Design: Translate product requirements into technical specifications for model behavior. Engineer system prompts and few-shot examples to address specific capability gaps and behavioral failures.
Evaluation Design: Define metrics for reasoning, tool usage, and safety, and validate these metrics against human judgment to ensure statistical rigor.
Data Strategy: Design algorithms to filter, score, and select training data. Write Python scripts to sanitize inputs and manage the training data lifecycle from raw logs to high-quality datasets.
Failure Analysis: Investigate regressions in model benchmarks. Diagnose root causes, distinguishing between data quality issues, prompt instruction failures, or underlying model capability gaps and implement fixes.
Ground Truth Management: Define rubrics and guidelines for human annotation. Maintain reference datasets (&quot;Golden Sets&quot;) to establish a consistent baseline for model performance evaluation.<h2 class="jobSectionHeader">
Minimum Qualifications</h2><ul><li>
Master's degree or PhD in Computer Science, Data Science, Statistics, or a related technical field, or equivalent practical experience</li><li>
3+ years of experience in Data Science, Machine Learning, or Applied Science</li><li>
Proficiency in Python, with experience writing production-quality code for data pipelines or evaluation harnesses</li><li>
Experience with experimental design, A/B testing, or statistical analysis</li></ul><h2 class="jobSectionHeader">
Preferred Qualifications</h2><ul><li>
Experience with Large Language Models (LLMs), including prompt engineering, fine-tuning, or RLHF workflows</li><li>
Experience building automated evaluation systems or implementing model-based evaluation frameworks</li><li>
Ability to translate product requirements into measurable technical metrics</li><li>
Experience managing human-in-the-loop data pipelines or annotation quality control</li></ul><h2 class="jobSectionHeader">
Why This Role Matters</h2>
You can't improve what you can't measure. You can't ship what you can't trust.
You will define the technical definition of quality for our agents &mdash; the metrics that predict real-world success, the datasets that encode user intent, and the prompts that shape model behavior. Your work will directly determine how quickly we can iterate and how confidently we can ship.<h2 class="jobSectionHeader">
Our Culture</h2>
All in, in person &mdash; work moves faster face-to-face
 
Ship by default &mdash; speed and polish can coexist
 
One band, one sound &mdash; radical candor, zero politics<h2 class="jobSectionHeader">
Perks</h2>
Competitive company-sponsored medical, dental, and vision insurance
 
Top-tier relocation and immigration support<h2 class="jobSectionHeader">
How to Apply</h2>
Send us:<ul><li>
A link &mdash; or 60-second video &mdash; of something you built and why it matters</li><li>
Your resume or LinkedIn</li><li>
Two sentences on the hardest problem you've cracked</li></ul>
Every exceptional candidate hears back within 48 hours.
If you see possibility where others see limits, we'd love to meet you.</div>

Applied Data Scientist, Evaluation & Model Behavior

Prolific

<div>
 
 AI Trainer &ndash; Machine Learning Specialists
 
 
 
 
 About Prolific
 
 
 
 Prolific is not just another player in the AI space &ndash; we are building the biggest pool of quality human data in the world.
 
 
 Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid study participants with a wide variety of experiences, knowledge, and skills.
 
 
 
 The role
 
 
 
 We&rsquo;re looking for
 
 AI Trainer &ndash; Machine Learning Specialists
 
 to help train and evaluate cutting-edge AI models using real ML expertise. If you have the necessary experience, we&rsquo;ll send you a quick
 
 10- to 15-minute test
 
 to assess your skills and suitability for AI tasks. If successful, you&rsquo;ll be invited to join Prolific as a participant, where you&rsquo;ll get paid to train and evaluate powerful AI models.
 
 
 Researchers looking for your skills tend to pay up to
 
 $150/hr per AI task completed
 
 . You must be prepared to complete paid tasks that require
 
 one hour of uninterrupted work
 
 , though many are shorter.
 
 
 What you&rsquo;ll bring
 
 
 <ul>
 <li>
 AI Training task skills and verifiable professional experience as a Machine Learning Specialist (e.g., ML engineer, data scientist, applied scientist, research engineer)
 </li>
 <li>
 A willingness to take our skills verification test to assess your suitability for our participant pool
 </li>
 <li>
 Strong attention to detail and the ability to concentrate on complex tasks for up to one hour at a time
 </li>
 <li>
 A reliable and fast internet connection and access to a computer
 </li>
 <li>
 A willingness to self-declare your earnings, as our participants are self-employed
 </li>
 <li>
 A Paypal account to receive payment from our clients
 
 
 </li>
 </ul>
 
 What you&rsquo;ll be doing in the role
 
 
 
 <ul>
 <li>
 Completing AI training tasks such as analyzing, editing, and writing annotations (including technical reasoning and structured evaluation)
 </li>
 <li>
 Judging the performance of AI in performing ML-relevant tasks (e.g., model/experiment critique, data leakage detection, metrics interpretation, debugging approach, methodology review)
 </li>
 <li>
 Improving cutting-edge AI models by providing expert feedback on correctness, robustness, clarity, and technical depth
 
 
 </li>
 </ul>
 
 Key Technologies
 
 
 
 <ul>
 <li>
 General AI Training
 </li>
 <li>
 Model Evaluation and Evals
 </li>
 <li>
 Trust and Safety
 </li>
 <li>
 Red Teaming
 </li>
 <li>
 Quality Analytics
 </li>
 <li>
 Data/ML Concepts (e.g., supervised learning, deep learning, NLP, CV, statistics, experimentation)
 
 
 </li>
 </ul>
 
 Why Prolific is a great platform to join as a Participant
 
 
 
 Joining our platform as a Prolific participant will give you the chance to influence the AI models of the future using professional machine learning expertise. Once you pass our assessment, you can join Prolific in just 15 minutes, and start enjoying competitive pay rates, flexible hours, and the ability to work from home.
 
 
 We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioural data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.
 
 
 We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviours into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation &ndash; one that reflects the breath and the best of humanity.
 
 
 
 Links to more information on Prolific
 
 
 
 Website
 
 
 Youtube
 
 
 Privacy Statement
 
 
 By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal personal information.
</div>

AI Training - Machine Learning Specialist (PST)

Triumph

<div><h1 class="jobSectionHeader">About Us</h1>
Triumph makes mobile gaming more thrilling by letting players wager - and win - real money, play in mass multiplayer games, and compete in social tournaments. We've built the top app in our App Store category and sustained exponential month-over-month growth on our revenue and active players. We're hyper-scaling our team and continuing to innovate, launching new products like Rips, our collectibles app, which has found huge success and is continuing to expand. And we're just getting started.
Triumph is backed by some of the top consumer VCs including Goodwater Capital, General Catalyst, and DraftKings Drive Fund.<h1 class="jobSectionHeader">
The Role</h1>
As a Data Scientist, you'll own the quantitative systems that drive how millions of real-money players experience Triumph's products, from their first session to long-term retention and monetization. You'll build the models and frameworks behind our most critical business decisions: how we price, how we pay out, how we match players, and how we grow.
You'd be joining a small, high-output quant team (4 people today) that operates like a trading desk. We build the mathematical systems that power Triumph's core business: pricing engines, payout distributions, matchmaking algorithms, risk models, and player behavior systems. Every model we ship touches real money and real users. You see the impact in the numbers the next day.<h1 class="jobSectionHeader">
What You'll Do</h1><ul><li>
Monetization &amp; Pricing: Develop and optimize the pricing engines, payout structures, and edge calculations that are the mathematical backbone of Triumph's revenue. Own pack economics, rarity calibration, and pricing models for Rips by Triumph.</li><li>
User Journey &amp; Retention: Build models that map the full player lifecycle: acquisition, activation, engagement, monetization, churn risk. Identify the quantitative levers that move retention and LTV, and design interventions that act on them.</li><li>
Experimentation: Design and analyze experiments (A/B tests and beyond) with rigorous statistical methodology. Own the measurement framework that tells us what's actually working across the product.</li><li>
Behavioral Modeling: Develop ML and statistical models on rich, high-frequency user behavior data (session patterns, spend curves, matchmaking outcomes, gameplay trajectories) to drive both product decisions and real-time production systems.</li><li>
Growth &amp; Acquisition: Build models that directly inform acquisition spend and channel optimization, connecting upstream marketing decisions to downstream LTV and monetization outcomes.</li><li>
Cross-Functional Impact: Partner closely with engineering, product, and leadership to translate model outputs into shipped features and strategic decisions. Identify high-leverage quantitative problems across the business and drive them from formulation to production impact.</li></ul><h1 class="jobSectionHeader">
Qualifications</h1><ul><li>
Bachelor's degree in a quantitative subject: math, physics, computer science, statistics, economics, or a related discipline.</li><li>
True depth and mastery in at least one quantitative domain: probability, statistics, applied ML, causal inference, or mathematics. We want spiky people who are confident they are among the best in their discipline.</li><li>
Proficiency in Python and SQL.</li><li>
Experience working with large-scale user or behavioral datasets.</li></ul><h1 class="jobSectionHeader">
Preferred Qualifications</h1><ul><li>
Experience in consumer tech, gaming, fintech, or marketplace data science, particularly in monetization, LTV modeling, or experimentation.</li><li>
Prior experience as a quantitative trader or quantitative researcher.</li><li>
Experience in competitive math, physics, or CS olympiads, or a graduate degree in a quantitative discipline.</li><li>
Nationally competitive in any activity. Some members of our team include national champions in debate, Clash Royale, and Poker.</li></ul><h1 class="jobSectionHeader">
Why Triumph?</h1><ul><li>
High growth. Build a high-scale consumer platform that touches gaming, finance, and social with the autonomy to set our web direction.</li><li>
High agency. Small, high-impact engineering team that is growing rapidly with significant opportunity for leadership and growth.</li><li>
High energy. Passionate team who are proud of our work and velocity (16x year over year growth).</li><li>
Competitive salary and benefits. $400/mo lunch credit, healthcare, vision, dental, 401k, etc.</li></ul>
Our team gathers 5 days a week at Triumph&rsquo;s headquarters at Levi&rsquo;s Plaza in San Francisco.
Compensation Range: $160K - $265K</div>

Data Scientist

OpenAI

<div>
 
 About The Team
 
 
 
 Our Applied team brings OpenAI technologies to consumers and businesses around the world. We collaborate across research, engineering, design and business functions to turn cutting-edge AI advancements into impactful real-world applications. Our team has been behind notable product launches (ChatGPT, API, Sora), creating tools that help developers write code, enable businesses to operate more efficiently, and empower individuals to learn and create. As AI capabilities rapidly evolve, we focus on ensuring that our products are safe, accessible, and beneficial to all.
 
 
 
 About The Role
 
 
 
 As a Data Scientist on the Applied Product team, you will contribute to a data-driven product development culture for consumer and enterprise products at OpenAI. This is critical as our products reach millions of users and businesses worldwide. We are focused on aligning both research and product development to drive measurable impact for these individuals and organizations alike.
 
 
 You should expect to define our north-star metrics, design A/B tests, and establish source-of-truth dashboards that the entire company can use to answer their own product questions. Most importantly, you should expect to be a core member of the product development team.
 
 
 This role is based in San Francisco, CA or Seattle, WA. We use a hybrid work model of 3 days in the office per week and offer relocation assistance to new employees.
 
 
 
 In This Role, You Will
 
 
 
 <ul>
 <li>
 Embed with the product development team as a trusted partner, uncovering new ways to improve the product and drive growth
 </li>
 <li>
 Define and interpret A/B tests that help answer critical questions about the impact of model and UX changes to our product
 </li>
 <li>
 Establish a data-driven product development culture by defining, tracking, and operationalizing feature-, product-, and company-level metrics
 </li>
 <li>
 Develop and socialize dashboards, reports, and other ways of enabling the team and company to answer product data questions in a self-serve way
 
 
 </li>
 </ul>
 
 You Might Thrive In This Role If You Have
 
 
 
 <ul>
 <li>
 5+ years experience in a quantitative role navigating highly ambiguous environments, ideally as an early data scientist or product analyst at a hyper-growth product company or research org
 </li>
 <li>
 Proposed, designed, and run rigorous experiments with clear insights and product recommendations utilizing SQL and Python
 </li>
 <li>
 Defined, implemented, and operationalized new feature and product-level metrics from scratch
 </li>
 <li>
 Excellent communication skills with demonstrated ability to communicate with product managers, engineers, and executives alike
 </li>
 <li>
 Strategic insights beyond the paradigm of statistical significance testing
 
 
 </li>
 </ul>
 
 You could be an especially great fit if you have:
 
 
 
 <ul>
 <li>
 Strong programming background, with ability to run simulations and prototype variants
 </li>
 <li>
 Experience validating quantitative insights with qualitative methods (e.g. surveys, UXR)
 </li>
 <li>
 Demonstrated prior experience in NLP, large language models, or generative AI
 
 
 </li>
 </ul>
 
 About OpenAI
 
 
 
 OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
 
 
 We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.
 
 
 For additional information, please see OpenAI&rsquo;s Affirmative Action and Equal Employment Opportunity Policy Statement.
 
 
 Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.
 
 
 To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.
 
 
 We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.
 
 
 OpenAI Global Applicant Privacy Policy
 
 
 At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
 
 
 Compensation Range: $230K - $385K
</div>

Data Scientist, Product

VARIANCE

<div><h2 class="jobSectionHeader">Role</h2>
At Variance, we are teaching machines to make the hardest judgment calls at scale. That means building AI agents for the high-stakes gray area of risk investigations, fraud, and identity reviews.
We&rsquo;re a small, talent-dense team in San Francisco working on a problem at the edge of what AI systems can reliably do: making good decisions in messy, adversarial, real-world environments. We focus on building, high-consequence systems problems where the edge cases matter most.
We&rsquo;re looking for a Research Engineer to help define how we measure and improve model quality. You&rsquo;ll build the benchmarks, datasets, tooling, and evaluation loops that tell us whether our systems are actually getting better on the tasks that matter. This role sits at the center of research, product, and engineering. It is about creating rigorous, domain-specific evaluations that reflect real customer workflows, expose meaningful failure modes, and drive the next generation of model and agent improvements.<h2 class="jobSectionHeader">
You&rsquo;re a fit if you:</h2><ul><li>
Care deeply about craftsmanship and have strong opinions about model quality, measurement, and experimental rigor</li><li>
Want to work on core model and agent behavior, not just surface-level product metrics</li><li>
Are excited by the challenge of defining what &ldquo;good&rdquo; looks like in messy, high-stakes environments</li><li>
Think in tight loops: hypothesis, benchmark design, evaluation, failure analysis, iteration</li><li>
Have strong engineering fundamentals and like building robust systems around ambiguous research problems</li><li>
Thrive in environments where success criteria are initially underspecified and need to be sharpened through work</li><li>
Are willing to do the work in the trenches: reviewing outputs, grading edge cases, curating datasets, and refining tasks until the evaluation actually measures what matters</li><li>
Care deeply about building systems that protect people from fraud, scams, and abuse</li></ul><h2 class="jobSectionHeader">
What you&rsquo;ll do</h2><ul><li>
Build proprietary benchmarks and datasets to evaluate models and model systems on fraud, identity, and risk workflows</li><li>
Design and run offline and online evals that measure model performance on real customer tasks, not just abstract benchmarks</li><li>
Define quality metrics for judgment systems, including precision, calibration, consistency, abstention, and failure handling</li><li>
Study where models and agents break, and turn those failures into better evals, better datasets, and better training loops</li><li>
Build reusable evaluation tools and quality building blocks that can be used across different product surfaces and workflows</li><li>
Partner closely with research, engineering, product, and design to improve system quality through rigorous experimentation</li><li>
Help create a strong culture of scientific experimentation, clear measurement, and continuous iteration</li><li>
Push the boundary of how AI systems are evaluated in regulated, adversarial, and high-consequence environments</li></ul><h2 class="jobSectionHeader">
What success looks like</h2><ul><li>
We have a clear, trusted view of how our systems perform across the workflows that matter most</li><li>
Our evals predict real-world quality better than generic benchmarks</li><li>
We identify meaningful failure modes earlier and improve system behavior faster</li><li>
We develop differentiated datasets, benchmarks, and quality loops that compound over time</li><li>
Research and engineering teams use your work to make better decisions about what to train, ship, and improve next</li><li>
Variance becomes known for rigorous, domain-specific evaluation of judgment systems</li></ul><h2 class="jobSectionHeader">
Preferred background</h2><ul><li>
Experience training, evaluating, or improving modern ML systems</li><li>
Strong programming skills and comfort working in research-heavy codebases</li><li>
Experience building benchmarks, datasets, evaluation pipelines, or quality systems</li><li>
Familiarity with LLMs, agent systems, retrieval, post-training, or adjacent areas</li><li>
Ability to design clean experiments and draw reliable conclusions from noisy results</li><li>
Strong engineering judgment and a bias toward building</li><li>
Interest in fraud, risk, trust and safety, compliance, or other regulated and adversarial domains</li></ul><h2 class="jobSectionHeader">
Our culture</h2>
We believe in ownership, urgency, and craft. We enjoy spirited debate, wild ideas, and building things we&rsquo;re proud of. We&rsquo;re fully in-person in San Francisco.<h2 class="jobSectionHeader">
What we offer</h2><ul><li>
Competitive salary and meaningful equity</li><li>
Platinum-level medical, dental, and vision insurance</li><li>
Unlimited PTO, sick leave, and parental leave</li><li>
Up to $100 per month in reimbursement for personal health and wellness expenses</li><li>
401(k) plan</li></ul>
Compensation Range: $250K - $400K</div>

Research Engineer, Evals

Valence

<div>Valence has built the only first-to-market AI native coaching platform for enterprise, offering personalized, expert, and human-like guidance and support to any leader or employee. We&rsquo;re not just talking about the future of work &mdash; we&rsquo;re building it now, with the most innovative Fortune 500 companies across healthcare, financial services, manufacturing, and technology. 
Our focus is on the problems that actually decide whether AI changes how organizations operate &mdash; the ones with no playbook, no obvious answers, and no guarantee of success. If you want to be part of the small group that defines how AI transforms the future at a global scale, this is your chance. 
And this isn&rsquo;t for everyone. We&rsquo;re not looking for people who want predictability or incremental progress. We only want those who are restless at the edge of what&rsquo;s possible, who get bored when things feel &ldquo;done,&rdquo; and who are driven to redefine what AI can mean for leaders, companies, and the world. Because at Valence, the work worth doing is the kind that redefines work itself. 
The Role
As a founding Applied AI Engineer at Valence, you will help define and build the future of AI-powered leadership coaching, working directly with our Head of AI and cross-functional product and engineering teams. You&rsquo;ll be at the intersection of generative AI research and product execution - helping design, build, and refine intelligent systems that deliver context-aware, personalized coaching at enterprise scale. This role is purposefully broad and adaptable: we don&rsquo;t expect any one candidate to check every technical box, but we do look for engineers who are eager to learn, iterate rapidly, and contribute meaningfully across a range of challenges from model behavior to production systems. 
You&rsquo;ll tackle real-world AI problems - from transforming enterprise data into actionable context, to optimizing conversational experiences, to shaping how AI engages with users in meaningful and responsible ways. Your work will directly impact how our platform performs in high-stakes enterprise deployments and how leaders around the world grow through AI-facilitated insights. 
About Valence
We're the only company pioneering leadership coaching for large enterprises in an AI-first way. Our mission is to transform how the world's biggest companies approach learning and development, helping teams work better together through AI-powered personalization that adapts to individual goals and organizational culture using the latest advances in machine learning and natural language processing. 
We've been featured in Harvard Business Review, TIME, World Economic Forum, Financial Times, Forbes and an Inc. 5000 fastest-growing private companies in America. Our clients represent the most diverse and sophisticated enterprise AI implementations globally, including Coca-Cola, Delta, Nestl&eacute;, General Mills, Schneider Electric, Deutsche Telekom, AstraZeneca, Prudential, CVS and Bristol Myers Squibb. 
Working at Valence means you'll work directly with Fortune 500 technology leaders, building expertise through the most complex enterprise deployments while gaining insight into diverse organizational approaches to AI transformation. These aren't just any enterprise clients - they're the companies defining what AI-first business transformation looks like across every major industry.
What You'll Do<ul><li>
Architect and build enterprise-grade AI and conversational systems that power coaching workflows and user experiences.</li><li>
Develop, evaluate, and refine LLM-based components - balancing performance, scalability, and reliability in real use cases.</li><li>
Integrate and manage diverse sources of structured and unstructured data to improve contextual understanding and output quality.</li><li>
Partner closely with product, engineering, and design to translate user needs into impactful technical solutions.</li><li>
Rapidly prototype and iterate on systems that span backend services, data pipelines, and frontend interactions as needed.</li><li>
Build tooling, tests, and automation to support reliable model deployment, observability, and continuous improvement.</li><li>
Help streamline data and science workflows, enabling fast experimentation and data-driven decisions.</li><li>
We recognize that not every candidate will have experience in every area above - we value growth potential, curiosity, and the ability to learn on the job.</li></ul>
What We're Looking For<ul><li>
Technical foundation: 8+ years of experience in software engineering, AI/ML, data-intensive systems, AI/ML development (ideally including a Master's or Ph.D. in Computer Science, ML, Data Science, or a related field).</li><li>
AI systems mindset: Familiarity with language systems (e.g., NLP, conversational interfaces, IR) and comfort reasoning about model behavior, context, and evaluation - both theoretical and practical knowledge.</li><li>
Data tooling &amp; analysis: Experience with core data science tools such as NumPy, scikit-learn, Pandas, PySpark, plus SQL and common visualization tools (e.g., matplotlib, Seaborn, Plotly, or BI tools) to explore and communicate insights.</li><li>
Cloud &amp; deployment experience: Comfortable developing and deploying services in cloud environments (AWS, GCP, Azure) and working with containerization/orchestration (Docker, Kubernetes).</li><li>
Engineering excellence: Strong software engineering skills, including writing maintainable code, debugging distributed systems, and collaborating in cross-functional teams.</li><li>
Growth orientation: Eagerness to tackle unfamiliar problems, learn new technologies, and contribute to shaping our platform and culture.</li><li>
Communication: Ability to explain technical ideas clearly and work effectively with both technical and non-technical stakeholders.</li><li>
Nice-to-have (but not required): experience with ML lifecycle tools (e.g., MLflow, Weights &amp; Biases), familiarity with Cloud ML services, or past work building generative AI applications.</li></ul>
What You'll Get<ul><li>
Ownership &amp; Rapid Growth</li><li>
Outsized missions from day one, with direct responsibility for company-defining projects</li><li>
Work alongside the executive team with transparency into strategy and decision-making</li><li>
Influence on direction through real-time customer feedback and market insights</li></ul><ul><li> 
AI-First Operator</li><li>
Work directly with cutting-edge AI models and next-generation platforms</li><li>
Build expertise in enterprise AI implementation across Fortune 500 companies and multiple industries</li><li>
Establish yourself as a recognized leader among peers in shaping how AI transforms work at a global scale</li></ul><ul><li> 
Compensation</li><li>
Competitive salary including base + bonuses</li><li>
Comprehensive health coverage (medical, dental, vision) from day one</li><li>
Generous PTO, company-wide R&amp;R shutdowns, and paid parental leave</li><li>
Retirement plan support for US and global employees</li></ul><ul><li> 
Equity</li><li>
Meaningful ownership in a venture-backed company at a growth inflection point</li><li>
Financial upside that comes from scaling fast</li><li>
Top-up grants as we scale and you deliver exceptional performance &mdash; your compensation grows alongside your impact</li></ul><ul><li> 
Top-Performing Culture</li><li>
A culture built for top talent: intensity to win, growth without limits, and a team that solves hard problems and celebrates big wins together</li></ul> 
Learn more about us and meet our team here
Location and Work Environment
If candidates are based in NYC or Toronto they can work hybrid in our offices, otherwise this role can be remote. Candidates must be comfortable working with colleagues in different time zones (UK), and have valid travel documents without work authorization restrictions in the US. 
Diversity and Inclusion
We are dedicated to creating a diverse and inclusive environment where everyone feels valued and supported. We encourage applications from candidates of all backgrounds and offer accommodations upon request throughout the hiring process. If you have any questions, please reach out to Allison Langille, Head of People, at jobs@valence.co. 
Employment Verification &amp; Commitment
We use third-party services to verify employment history, education, and other information relevant to your candidacy. Employment is contingent upon the successful completion of these verification checks. This is a full-time role that requires a high level of focus, availability, and commitment. Employees may not hold concurrent full-time employment with another organization while employed at Valence. Any outside consulting, advisory, freelance, or other professional work must be disclosed and approved in advance and must not interfere with job responsibilities, availability, performance, or create a conflict of interest - including risks related to confidentiality, intellectual property, or competition. 
#LI-HYBRID</div>

Staff Software Engineer, Applied AI

Baseten

<div>
 
 About Baseten
 
 
 
 Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $300M Series E, backed by investors including BOND, IVP, Spark Capital, Greylock, and Conviction. Join us and help build the platform engineers turn to to ship AI products.
 
 
 
 THE ROLE
 
 
 
 This role sits at the applied end of our post-training research efforts. You will work directly with stakeholders from the world&rsquo;s fastest-growing AI companies to post-train open-source models that outperform frontier closed models on their specialised tasks. Your day-to-day is finding creative ways to extract signal from complex, domain-specific datasets and building the reward functions, environments, eval harnesses, and training pipelines that turn that signal into better models. The models you train ship to production and reach millions of users.
 
 
 We are looking for people with hands-on LLM fine-tuning and RL experience. Researchers who are excited by the prospect of shipping models into production, who can translate a customer's domain-specific requirements into an effective training curriculum, and who know when to be rigorous and when to iterate fast.
 
 
 
 RECENT RESEARCH
 
 
 
 <ul>
 <li>
 Dense, on-policy or both?
 </li>
 <li>
 Repeated kv cache for long-running agents
 </li>
 <li>
 Distillation without the dark &ndash; replicating black-box on-policy distillation on Baseten
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Design and run post-training pipelines: SFT, GRPO, DPO, RLVR, reward function engineering, and synthetic data generation.
 </li>
 <li>
 Build task-specific training environments and evals tailored to customer domains like healthcare, code generation, and legal, spanning multi-turn tool use, sandboxed execution, and agentic workflows.
 </li>
 <li>
 Work directly with customers to translate production data into training signal, designing reward loops from real usage patterns and handling distribution shift.
 </li>
 <li>
 Run and analyze training experiments end-to-end: diagnose reward hacking, importance sampling drift, and advantage estimation instabilities.
 </li>
 <li>
 Publish findings at top venues and contribute to Baseten's open-source training libraries.
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Hands-on experience training LLMs with reinforcement learning &mdash; demonstrated understanding of GRPO or PPO beyond recipe-level reproduction, including group advantage computation, clipped objectives, and KL penalty design
 </li>
 <li>
 Strong intuition for reward engineering: the ability to distinguish between a reward that trains effectively and one that will exploit at scale
 </li>
 <li>
 Experience building multi-turn agent environments with tool use, not limited to single-turn question-answering setups
 </li>
 <li>
 Comfort working across the full pipeline from dataset construction through training, evaluation, and deployment
 </li>
 <li>
 Experience with production ML systems. Preference for candidates who have closed a training&ndash;inference loop where production data feeds back into model improvement
 
 
 </li>
 </ul>
 
 Preferred Qualifications
 
 
 
 <ul>
 <li>
 Experience with RL training frameworks
 </li>
 <li>
 Publications at NeurIPS, ICML, ICLR, focused on RL for LLMs, reward modeling, or alignment
 
 
 </li>
 </ul>
 
 Benefits
 
 
 
 <ul>
 <li>
 Competitive compensation, including meaningful equity.
 </li>
 <li>
 100% coverage of medical, dental, and vision insurance for employee and dependents
 </li>
 <li>
 Generous PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
 </li>
 <li>
 Paid parental leave
 </li>
 <li>
 Company-facilitated 401(k)
 </li>
 <li>
 Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
 
 
 </li>
 </ul>
 
 Apply now
 
 to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
 
 
 
 At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.
 
 
 
 
 We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).
 
 
 
 Compensation Range: $200K - $275K
 
 
</div>

Post-Training Applied Researcher

Explore 296+ Applied data scientist evaluation model behavior in San Francisco this April. Find roles across top companies and apply now to boost your career.