software engineer

data engineer

ai engineer

ai/generative ai engineer Jobs in San Francisco

ai/generative ai engineer

associate machine learning engineer Jobs in San Francisco

associate machine learning engineer

data pipeline engineer

Mercor

<div>
 
 About The Job
 
 
 
 
 Mercor
 
 connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
 
 Benchmark
 
 ,
 
 General Catalyst
 
 ,
 
 Peter Thiel
 
 ,
 
 Adam D'Angelo
 
 ,
 
 Larry Summers
 
 , and
 
 Jack Dorsey
 
 .
 
 
 
 Position:
 
 Software Engineer II
 
 
 
 Type:
 
 
 Full-Time
 
 
 
 
 Compensation:
 
 
 $70&ndash;$95/hour
 
 
 
 
 Location:
 
 
 Remote
 
 
 
 
 Role Responsibilities
 
 
 
 <ul>
 <li>
 Drive technical implementation and testing of complex data pipeline features within the SPECTRA platform. Ensure robust end-to-end functionality and meet project release timelines.
 </li>
 <li>
 Design and develop data processing pipelines using SQL and Python. Handle large-scale data collection, transformation, and quality monitoring workflows.
 </li>
 <li>
 Build and maintain dashboards and visualization tools. Provide visibility into data quality, rater performance, and platform metrics for stakeholders across multiple teams.
 </li>
 <li>
 Create comprehensive end-to-end tests and monitoring alerts. Ensure platform reliability, data integrity, and proactive identification of issues in the data collection workflow.
 </li>
 <li>
 Debug and troubleshoot complex data flow issues across the SPECTRA platform. Identify root causes and implement solutions to maintain high system availability.
 </li>
 <li>
 Collaborate closely with Technical Leads and engineering teams. Implement backend data processing logic and ensure adherence to development standards and best practices.
 
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 
 Must-Have
 
 
 
 <ul>
 <li>
 Experience in SQL development writing complex queries, data transformations, and working with large datasets in production environments.
 </li>
 <li>
 Proficient in Python with experience in data processing, automation, and pipeline development using object-oriented programming principles.
 </li>
 <li>
 Experience with data engineering pipelines including ETL processes, data modeling, and building scalable data processing workflows.
 </li>
 <li>
 Hands-on experience with data visualization and dashboard tools such as Tableau or similar platforms.
 </li>
 <li>
 Strong analytical and problem-solving skills with demonstrated ability to debug complex data issues and implement effective solutions.
 </li>
 <li>
 Experience with data quality monitoring and automated testing frameworks.
 
 
 
 </li>
 </ul>
 
 Preferred
 
 
 
 <ul>
 <li>
 Degree in Computer Science or related field.
 </li>
 <li>
 Machine Learning experience with understanding of ML model training data requirements, feature engineering, and model evaluation metrics.
 </li>
 <li>
 Experience building and scaling large products or systems.
 </li>
 <li>
 Experience with building privacy and ads-related products.
 </li>
 <li>
 Experience working with human labeling and annotation systems including workflow design, quality assurance, and vendor management in data collection operations.
 </li>
 <li>
 Prior experience working with the company or its internal tools.
 
 
 
 </li>
 </ul>
 
 Compensation &amp; Legal
 
 
 
 <ul>
 <li>
 Pursuant to the California Fair Chance Act, Los Angeles County Fair Chance Ordinance for Employers, Los Angeles Fair Chance Initiative for Hiring Ordinance, and San Francisco Fair Chance Ordinance, qualified applicants will be considered for assignment with arrest and conviction records.
 
 
 
 </li>
 </ul>
 
 Application Process (Takes 20&ndash;30 mins to complete)
 
 
 
 <ul>
 <li>
 Upload resume
 </li>
 <li>
 AI interview based on your resume
 </li>
 <li>
 Submit form
 
 
 
 </li>
 </ul>
 
 Resources &amp; Support
 
 
 
 <ul>
 <li>
 For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
 </li>
 <li>
 For any help or support, reach out to: support@mercor.com
 
 
 
 </li>
 </ul>
 
 PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
 
</div>

Software Engineer I - Junior Level

Checkr

<div><div>About Checkr 
Checkr is building the data platform to power safe and fair decisions. Established in 2014, Checkr's innovative technology and robust data platform help customers assess risk and ensure safety and compliance to build trusted workplaces and communities. Checkr has over 100,000 customers including DoorDash, Coinbase, Lyft, Instacart, and Airtable.
 
 
We're a team that thrives on solving complex problems with innovative solutions that advance our mission. Checkr is recognized on Forbes Cloud 100 2025 List and is a Y Combinator 2024 Breakthrough Company.</div>
About the Role

We are seeking a strong Senior Data Engineer to build and maintain scalable, high-quality data pipelines powering Checkr's centralized data platform. As a Senior Engineer, you will independently deliver complex features, contribute to system design, and collaborate with cross-functional partners to support the next generation of our data products.
<h2 class="jobSectionHeader">
What You'll Do</h2>
<ul><li>Independently design and implement complex batch and streaming pipelines using PySpark, SQL, and AWS services.</li>
<li>Navigate ambiguity with guidance, translating high-level direction into well-scoped, high-quality technical solutions.</li>
<li>Work cross-functionally with product, design, analysts, and engineers to ship impactful features and improve data workflows.</li>
<li>Contribute to architectural discussions and system improvements without owning long-term strategy.</li>
<li>Ensure pipeline reliability and data quality, implementing testing, monitoring, and observability best practices.</li>
<li>Investigate and resolve production issues for services owned by the team.</li>
<li>Write performant, maintainable code that aligns with engineering standards.</li>
<li>Support the team in building foundational datasets that enable analytics, ML, and customer-facing features.</li>
</ul><h2 class="jobSectionHeader">What You Bring</h2>
<ul><li>6&ndash;7+ years of experience in data engineering with strong hands-on execution ability.</li>
<li>Proficiency with PySpark, Python, and SQL, including debugging and performance optimization.</li>
<li>Experience building large-scale pipelines (up to terabytes or larger), with exposure to streaming systems such as Kafka.</li>
<li>Strong knowledge of data modeling, relational databases, and NoSQL stores.</li>
<li>Experience with AWS services such as EMR, Glue, Athena, Lambda, and S3.</li>
<li>Exposure to Iceberg or other lakehouse technologies (nice to have).</li>
<li>Understanding of security and data privacy fundamentals.</li>
<li>Strong problem-solving skills, attention to detail, and ability to execute independently.</li>
<li>Knowledge of Databricks, Snowflake, or Graph/Vector stores is a plus.</li>
</ul><div>
At Checkr, we believe a hybrid work environment strengthens collaboration, drives innovation, and encourages connection. Our hub locations are Denver, CO, San Francisco, CA, Nashville, TN, and Santiago, Chile. Individuals are expected to work from the office 3 days a week. In-office perks are provided, such as lunch five times a week, a commuter stipend, and an abundance of snacks and beverages. A relocation stipend may be available for those willing to relocate to a Checkr hub location. 

Equal Employment Opportunities at Checkr 
Checkr is committed to building the best product and company, which requires hiring talented and qualified individuals with a diverse set of perspectives and lived experiences. Checkr believes in hiring people of all backgrounds, including those whose histories are impacted by the justice system in accordance with local, state, and/or federal laws, including the San Francisco's Fair Chance Ordinance. 

Applicant Privacy Policy 
If you are a California resident or are located in Alberta or British Columbia, our Applicant Privacy Policy applies to our collection and processing of your personal information when you apply for a role with us or otherwise participate in our recruitment process. 

<ul><li>Legitimate Checkr emails will always include our official domain name after the @ symbol (e.g., name@checkr.com or name@ext.checkr.com).</li></ul></div></div>

Senior Data Engineer

Airtable

<div><div>Airtable is the no-code app platform that empowers people closest to the work to accelerate their most critical business processes. More than 500,000 organizations, including 80% of the Fortune 100, rely on Airtable to transform how work gets done.</div>
Airtable is the no-code app platform that empowers people closest to the work to accelerate their most critical business processes. More than 500,000 organizations, including 80% of the Fortune 100, rely on Airtable to transform how work gets done.

At Airtable, we're passionate about democratizing software creation &mdash; empowering anyone to build powerful, flexible tools without writing code. With our shift to an AI-native platform, customers can now generate full apps and deploy AI agents directly into their workflows. Data engineering plays a critical role in this evolution by delivering the insights our teams rely on to improve user experience, measure agent impact, and understand how the business is performing at scale.

As a Software Engineer, Data at Airtable, you'll make an enormous contribution to our data engineering efforts. You'll design and own mission-critical data pipelines to enable decision-making, partner with company leaders to create scalable data solutions, and launch innovative alerting and visualization solutions.
<h2 class="jobSectionHeader">
The Teams</h2>

We're hiring across two pods. Both teams work closely with Data Science and Analytics and are central to how Airtable understands itself as a business. You'll find your fit based on where your interests and experience align. And uniquely: the platform you're measuring and building on is the same one your customers use every day. When Airtable ships a new AI agent capability, you're among the first to instrument it, understand its adoption, and help shape what comes next.

Product &amp; AI Data Infrastructure This team builds and owns the foundational data pipelines that power product analytics across Airtable. As Airtable shifts to an AI-native platform, our work increasingly involves instrumenting and measuring AI product usage, building event pipelines for AI agents, surfacing AI-native adoption metrics in core business tables, and developing AI-powered data discovery tooling, including vector search over our catalog metadata. We partner closely with product analytics, product engineering, and data infrastructure to turn business questions into well-modeled, trustworthy data.

GTM &amp; Business Data Engineering This team owns the datasets that power go-to-market and business operations at Airtable. We build and govern the business critical pipelines and core tables used to report AI usage metrics across the company, create and maintain data models that support RevOps, Marketing, and BizTech stakeholders, and increasingly leverage AI tools, including writing Claude skills and incorporating AI context guidance, to accelerate and enhance our day-to-day pipeline work. If you're energized by working at the intersection of clean data architecture and cross-functional business impact, this is your team.
<h1 class="jobSectionHeader">
What you'll do</h1>
<ul><li>Work across our engineering organization and stakeholders from data science, growth, sales, marketing, and product to understand the data needs of the business and produce pipelines, data marts, and other solutions that enable better decision-making.</li>
<li>Design and maintain our foundational business tables in order to simplify analysis and reporting across the entire company, including AI usage metrics surfaced to executive stakeholders.</li>
<li>Use AI tools as a daily part of how you work, from LLM-assisted pipeline development and debugging to exploring our catalog through AI-powered discovery, and bring a curiosity for where this tooling is heading next.</li>
<li>Build and enforce a pattern language across our data stack, ensuring pipelines and tables are consistent, accurate, and well-understood.</li>
<li>Continue to improve the performance and reliability of our data warehouse.</li>
<li>Partner with data scientists, analytics engineers, and business stakeholders to translate ambiguous business questions into well-scoped data solutions.</li>
</ul><h1 class="jobSectionHeader">Who you are</h1>
<ul><li>You have 3-8+ years of professional experience designing, creating, and maintaining scalable data pipelines, preferably in Airflow.</li>
<li>You've wrangled enough data to understand how often the complex systems that produce it can go wrong, and you build with that in mind.</li>
<li>You are proficient in at least one programming language (preferably Python) and are willing to pick up others as the work demands.</li>
<li>You are highly effective with SQL and understand how to write and tune complex queries.</li>
<li>You're genuinely curious about how AI is reshaping data engineering and you're actively experimenting, not just watching from the sidelines. Whether that's using LLMs to write and debug pipelines faster, thinking through how to model agent behavior as data, or exploring what smarter data discovery could look like, you bring enthusiasm for figuring it out.</li>
<li>You're passionate and thoughtful about building systems that enhance human understanding.</li>
<li>You communicate with clarity and precision in written form and have experience conveying findings through graphs and visualizations.</li></ul><div> 
Please see our Privacy Notice for details regarding Airtable's collection and use of personal data relating to the application and recruitment process by clicking here. 
 
For applicants that live in or have a link to Australia, please see this Privacy Collection Statement for details regarding Airtable's collection and use of personal data relating to the application and recruitment process.

 Stay Safe from Job Scams 
All official Airtable communication will come from an @airtable.com email address. We will never ask you to share sensitive information or purchase equipment during the hiring process. If in doubt, contact us at hr@airtable.com. Learn more about avoiding job scams here.</div></div>

Software Engineer, Data

Cognition

<div><h2 class="jobSectionHeader">We are an applied AI lab building end-to-end software agents.</h2>
We're the makers of Devin, the first AI software engineer, and Windsurf, the AI-native IDE. Together, they represent our vision for collaborative AI teammates that enable engineers to focus on more interesting problems and empower teams to strive for more ambitious goals.
Our team is small and talent-dense. Among our founding team, we have world-class competitive programmers, former founders, and leaders from companies at the cutting edge of AI including Scale AI, Palantir, Cursor, Waymo, Tesla, Lunchclub, Modal, Google DeepMind, and Nuro.
Building Devin and Windsurf is just the first step&mdash;our hardest challenges still lie ahead. If you&rsquo;re excited to solve some of the world&rsquo;s biggest problems and build AI that can reason on real-world tasks, apply to join us.<h2 class="jobSectionHeader">
About the Role</h2>
We&rsquo;re hiring a Data Engineer to help own our data stack &mdash; from database architecture and pipelines to integrations and reporting. You&rsquo;ll design and maintain the systems that keep our data reliable, accessible, and actionable across the Finance and Accounting organization.
In this role you will:<ul><li>
Design and manage database architecture and data models with a focus on finance and accounting use cases</li><li>
Build and maintain ETL/ELT pipelines and orchestration workflows</li><li>
Create and manage data integrations across internal and external financial systems</li><li>
Partner with Finance and Accounting teams to build auditable, automated reporting systems, dashboards, and datasets</li><li>
Support close processes, FP&amp;A workflows, and revenue reporting through scalable data infrastructure</li><li>
Own business reporting: datasets, dashboards, metrics, and self-serve analytics</li><li>
Ensure data quality, observability, governance, and documentation</li></ul>
Requirements for the role:<ul><li>
5+ years in a data engineering, analytics engineering, or a full-stack data role</li><li>
Expert SQL and strong Python (or R)</li><li>
Experience with data modeling, warehouse architecture, and BI-oriented schema design</li><li>
Hands-on experience with ETL/ELT tools (dbt, Airflow, Dagster, etc.)</li><li>
Experience building or maintaining BI reporting (Metabase a plus)</li><li>
Proven track record collaborating with finance stakeholders to deliver impactful solutions</li><li>
Experience with FP&amp;A processes, close workflows, and financial reporting</li><li>
Familiarity with GAAP vs. non-GAAP reporting and revenue recognition concepts</li><li>
Based in SF and willing to be in office 5 days a week</li></ul></div>

Data Engineer - Finance

Medeloop

<div><h2 class="jobSectionHeader">The Role</h2>

This is a full-ownership data engineering role at the center of Medeloop's AI platform. You won't be maintaining pipelines someone else built, you'll be architecting the data backbone that powers AI agents doing real operations at scale. You'll work directly with data scientists, AI engineers, and product teams to turn raw, complex healthcare data into the clean, structured, semantically-rich foundation our AI scientists depend on. Your work shows up in customer products and research outcomes, not internal dashboards that no one reads.Candidates who currently perform these tasks exclusively through manual processes are unlikely to be suitable for this role. We require an individual who has already adopted and integrated AI techniques to enhance operational velocity, rather than one who is contemplating future experimentation.If you want to build something that genuinely changes how medical research gets done, this is the role.
<h2 class="jobSectionHeader">
What You'll Own</h2>
<ul><li>The healthcare data lake: curating, extending, and evolving it through new concepts, derived variables, and data models that directly inform our AI engines and customer products</li>
<li>AI-native data workflows: designing and operating AI-powered pipelines (using tools like Claude Code and agent frameworks) to automate harmonization, cleaning, quality checks, and summarization at scale</li>
<li>NLP and semantic infrastructure: building pipelines for entity extraction, concept normalization, embedding-based retrieval, and semantic search that power the AI Scientist platform</li>
<li>Novel data extraction approaches: experimenting with and building new methodologies for working with unstructured clinical data, not just applying existing playbooks</li>
<li>Research-grade data products: delivering analytical samples, cohorts, and final datasets that withstand scientific scrutiny and are actively used by researchers and customers</li>
<li>Data governance and observability protocols: including access controls, PHI/PII handling, data classification, compliance, monitoring, alerting, data freshness, and comprehensive documentation to enable self-service capabilities.</li>
</ul><h2 class="jobSectionHeader">What We're Looking For</h2>
<ul><li>3+ years of relevant data engineering or data management within an analytics-driven organization, with end-to-end ownership from raw ingestion to final data product</li>
<li>Deep hands-on experience with healthcare CDMs (OMOP, FHIR, PCORnet) &mdash; designing or extending them, not just querying</li>
<li>Knowledge of medical ontologies: UMLS, SNOMED CT, RxNorm</li>
<li>Experience with big data, data pipelines and tooling that support retrieval-augmented generation (RAG), vector integrations, embedding workflows, and other AI/ML workloads. Experience in big data tooling such as Spark, Iceberg, EMR</li>
<li>Fluent in Python and SQL; comfortable across structured and unstructured data</li>
<li>Proven NLP experience: semantic search, entity recognition, concept normalization, embedding pipelines</li>
<li>Strong grasp of inferential statistics and cohort methodology to be a real partner to data scientists and customers (as part of onboarding)</li>
<li>Experience contributing to an AI/ML product, especially in automated research or scientific discovery</li>
<li>Experience mentoring other engineers and providing technical leadership</li>
</ul><h2 class="jobSectionHeader">Bonus Points</h2>
<ul><li>Multi-cloud experience (AWS, Azure, GCP)</li>
<li>Authorship or contribution to peer-reviewed publications or technical reports</li>
</ul><h2 class="jobSectionHeader">Why Medeloop</h2>
<ul><li>Ownership from day one: small team, high-trust, no layers between your work and its impact</li>
<li>Technically ambitious: you'll build AI-powered workflows, not just support them</li>
<li>Real-world stakes: your work accelerates drug development, addresses health equity, and improves clinical research for institutions that matter</li>
<li>Strong foundation: Series A, top-tier investors, and a data asset (200M+ patient records) that most companies spend years trying to build</li></ul></div>

Senior AI Data Engineer– Agentic Healthcare Platform

KREA

<div>About Krea
At Krea, we are building next-generation AI creative tools.
We are dedicated to making AI intuitive and controllable for creatives. Our mission is to build tools that empower human creativity, not replace it.
We believe AI is a new medium that allows us to express ourselves through various formats&mdash;text, images, video, sound, and even 3D. We're building better, smarter, and more controllable tools to harness this medium.
Supercomputing / AI Infra at Krea
We build and operate the infrastructure for Krea's research and inference. Distributed training, 1000+ K8s GPU clusters, petabyte scale data pipelines, etc. We build a lot of this from scratch &mdash; custom distributed datastores, job orchestration systems, and streaming pipelines that replace tools like Kafka and Ray for modern AI workloads at scale.
Example projects:
Distributed data systems<ul><li>
Design multi-stage pipelines that turn petabytes of raw data into clean, annotated datasets</li><li>
Run classification models on billions of images</li><li>
Deploy and combine LLMs to caption massive multimedia data</li></ul>
GPU infrastructure<ul><li>
Manage distributed training and inference on 1000+ GPU Kubernetes clusters</li><li>
Solve orchestration and scaling for large-scale GPU job processing</li><li>
Scale workloads and research between clusters in multiple datacenters</li></ul>
Distributed training<ul><li>
Profile and optimize dataloaders streaming thousands of images per second</li><li>
Profile and debug InfiniBand networking on huge training runs</li><li>
Build fault tolerance systems for large-scale pretraining</li><li>
Collaborate with researchers on evolving RL infrastructure</li></ul>
Applied ML pipelines<ul><li>
Find clean scenes in millions of videos using distributed shot-boundary detection</li><li>
Customize and train models to filter billions of images for questions like &quot;is this a screenshot?&quot;</li><li>
Build the systems that bridge raw cluster capacity and research output</li></ul>
Who we're looking for:
Systems people. If you've read a blog post about InfiniBand debugging or building a custom distributed database and thought &quot;I want to do that&quot; &mdash; this is that team.
You'll spend your time working heavily with Python, Kubernetes, Torch, and data tools like DuckDB, Arrow, etc. It's OK if you don't have K8s or ML experience &mdash; the main thing we hire for is an intuition for distributed systems, and a great mental model of how systems interact and function under different conditions.
Strong candidates may have experience with&hellip;<ul><li>
Python, PyArrow, DuckDB, SQL, massive relational databases, PyTorch, Pandas, NumPy&hellip;</li><li>
Kubernetes</li><li>
Designing and implementing large-scale ETL systems</li><li>
Fundamental knowledge of containerization, operating systems, file-systems, and networking</li><li>
Distributed systems design</li><li>
Distributed training systems (NCCL, InfiniBand, RDMA)</li><li>
Streaming and event processing systems (Kafka, Pulsar, or similar)</li><li>
PyTorch internals, custom dataloaders, and training infrastructure</li></ul></div>

Engineer, Supercomputing & Distributed Systems

Uber

<div>
 
 About The Role
 
 
 
 The Growth and Marketing Platform Data Team is looking for a highly skilled and passionate Data Engineer to build and scale Uber's data infrastructure. In this role, you'll develop high-performance data pipelines, real-time streaming systems, and robust data architectures that power Uber's growth and marketing initiatives.
 
 
 You'll collaborate with Product Managers, Data Scientists, Data Platform Engineers, and Marketing Operations to design scalable, efficient, and reliable data solutions that drive business impact. This is a highly visible and impactful role for someone who thrives on data at scale, technical leadership, and solving ambiguous problems with clarity and precision.
 
 
 
 What You Will Do
 
 
 
 <ul>
 <li>
 Design, develop, and maintain scalable data pipelines and backend services for processing large-scale web event analytics.
 </li>
 <li>
 Lead the development of core data infrastructure, ensuring high availability, performance, and reliability.
 </li>
 <li>
 Solve complex challenges at the intersection of low latency, high correctness, and full determinism.
 </li>
 <li>
 Architect and optimize data models, ETL/ELT workflows, and distributed data processing frameworks.
 </li>
 <li>
 Participate in code reviews, mentor junior engineers, and enforce best practices in data engineering.
 </li>
 <li>
 Diagnose and resolve data quality issues, ensuring accuracy, efficiency, and security.
 
 
 </li>
 </ul>
 
 Basic Qualifications
 
 
 
 <ul>
 <li>
 4+ years of experience in data engineering or backend software development.
 </li>
 <li>
 Proficiency in Python, SQL, Java, or Scala for data pipeline development.
 </li>
 <li>
 Proficiency in Big Data frameworks (e.g., Spark, Flink, Presto, MapReduce) and distributed computing.
 </li>
 <li>
 Data warehouse and Data Modeling experience
 
 
 </li>
 </ul>
 
 Preferred Qualifications
 
 
 
 <ul>
 <li>
 Experience in Web and SEO Analytics
 </li>
 <li>
 Experience in Marketing Technologies
 </li>
 <li>
 Designing and executing on Real-time and Near Real-time Systems
 
 
 </li>
 </ul>
 For San Francisco, CA-based roles: The base salary range for this role is USD$171,000 per year - USD$190,000 per year. You will be eligible to participate in Uber's bonus program, and may be offered an equity award &amp; other types of comp. All full-time employees are eligible to participate in a 401(k) plan. You will also be eligible for various benefits. More details can be found at the following link https://jobs.uber.com/en/benefits., For San Francisco, CA-based roles: The base salary range for this role is USD$171,000 per year - USD$190,000 per year. You will be eligible to participate in Uber's bonus program, and may be offered an equity award &amp; other types of comp. All full-time employees are eligible to participate in a 401(k) plan. You will also be eligible for various benefits. More details can be found at the following link https://jobs.uber.com/en/benefits.
</div>

Data Engineer - Growth and Marketing Platform

Explore 224+ Data pipeline engineer in San Francisco this April. Find roles across top companies and apply now to boost your career.