software engineer

machine learning engineer Jobs in San Francisco

machine learning engineer

data scientist, product analytics Jobs in San Francisco

data scientist, product analytics

senior data engineer Jobs in San Francisco

senior data engineer

ai application engineer Jobs in San Francisco

ai application engineer

data engineer

Adobe

<div>
 <strong>
  Our Company
  <br/>
  <br/>
 </strong>
 Changing the world through digital experiences is what Adobe&rsquo;s all about. We give everyone&mdash;from emerging artists to global brands&mdash;everything they need to design and deliver exceptional digital experiences! We&rsquo;re passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies interact with customers across every screen.
 <br/>
 <br/>
 We&rsquo;re on a mission to hire the very best and are committed to creating exceptional employee experiences where everyone is respected and has access to equal opportunity. We realize that new ideas can come from everywhere in the organization, and we know the next big idea could be yours!
 <br/>
 <br/>
 Adobe Customer Solutions is looking for a full time Data Engineer with experience in building data integrations using AWS technology stack as part of the team's Data as a Service portfolio for Adobe&rsquo;s Digital Experience enterprise customers.
 <br/>
 <br/>
 Customer facing Engineers who enjoy tackling complex technical challenges, have a passion for delighting customers and who are self-motivated to push themselves in a team oriented culture will thrive in our environment
 <br/>
 <br/>
 What you&rsquo;ll do
 <br/>
 <br/>
 <ul>
  <li>
   Collaborate with Data architects, Enterprise architects, Solution consultants and Product engineering teams to capture customer data integration requirements, conceptualize solutions &amp; build required technology stack
  </li>
  <li>
   Collaborate with enterprise customer's engineering team to identify data sources, profile and quantify quality of data sources, develop tools to prepare data and build data pipelines for integrating customer data sources and third party data sources with Adobe solutions
  </li>
  <li>
   Develop new features and improve existing data integrations with customer data ecosystem
  </li>
  <li>
   Encourage team to think out-of-the-box and overcome engineering obstacles while incorporating new innovative design principles.
  </li>
  <li>
   Collaborate with a Project Manager to bill and forecast time for customer solutions
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Requirements
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Experience as an enterprise Data Engineer from a consulting background
  </li>
  <li>
   AWS Certified Data Engineer &ndash; Associate or AWS Certified Cloud Practitioner
  </li>
  <li>
   5+ years experience in building/operating/maintaining fault tolerant and scalable data processing integrations using AWS
  </li>
  <li>
   7+ years experience in Python programming language preferably using PySpark
  </li>
  <li>
   Software development experience working with Apache Airflow, MongoDB, MySQL
  </li>
  <li>
   Strong capacity to manage numerous projects are a must
  </li>
  <li>
   Experience using Docker or Kubernetes is a plus
  </li>
  <li>
   BS/MS degree in Computer Science or equivalent industry experience
  </li>
  <li>
   Ability to identify and resolve problems associated with production grade large scale data processing workflows
  </li>
  <li>
   Excellent communication skills (we&rsquo;re a geographically distributed team)
  </li>
  <li>
   Experience creating and maintaining unit tests and continuous integration.
  </li>
  <li>
   Passion for creating Intelligent data pipelines that customers love to use.
   <br/>
   <br/>
  </li>
 </ul>
 Special Consideration given for
 <br/>
 <br/>
 <ul>
  <li>
   Experience &amp; knowledge with Web Analytics or Digital Marketing
  </li>
  <li>
   Experience &amp; knowledge with Customer Data Platform (CDP) or Data Management Platform (DMP)
  </li>
  <li>
   Experience &amp; knowledge with Adobe Experience Cloud solutions
   <br/>
   <br/>
  </li>
 </ul>
 Our compensation reflects the cost of labor across several  U.S. geographic markets, and we pay differently based on those defined markets. The U.S. pay range for this position is $114,100 -- $214,950 annually. Pay within this range varies by work location and may also depend on job-related knowledge, skills, and experience. Your recruiter can share more about the specific salary range for the job location during the hiring process.
 <br/>
 <br/>
 In California, the pay range for this position is $148,500 - $214,950 In New York, the pay range for this position is $148,500 - $214,950 In Washington, the pay range for this position is $135,100 - $195,550
 <br/>
 <br/>
 At Adobe, for sales roles starting salaries are expressed as total target compensation (TTC = base + commission), and short-term incentives are in the form of sales commission plans. Non-sales roles starting salaries are expressed as base salary and short-term incentives are in the form of the Annual Incentive Plan (AIP).
 <br/>
 <br/>
 In addition, certain roles may be eligible for long-term incentives in the form of a new hire equity award.
 <br/>
 <br/>
 <strong>
  State-Specific Notices:
  <br/>
  <br/>
 </strong>
 <strong>
  California:
  <br/>
  <br/>
 </strong>
 <strong>
  Fair Chance Ordinances
  <br/>
  <br/>
 </strong>
 Adobe will consider qualified applicants with arrest or conviction records for employment in accordance with state and local laws and &ldquo;fair chance&rdquo; ordinances.
 <br/>
 <br/>
 <strong>
  Colorado:
  <br/>
  <br/>
 </strong>
 <strong>
  Application Window Notice
  <br/>
  <br/>
 </strong>
 If this role is open to hiring in Colorado (as listed on the job posting), the application window will remain open until at least the date and time stated above in Pacific Time, in compliance with Colorado pay transparency regulations. If this role does not have Colorado listed as a hiring location, no specific application window applies, and the posting may close at any time based on hiring needs.
 <br/>
 <br/>
 <strong>
  Massachusetts:
  <br/>
  <br/>
 </strong>
 <strong>
  Massachusetts Legal Notice
  <br/>
  <br/>
 </strong>
 It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.
 <br/>
 <br/>
 Adobe is proud to be an Equal Employment Opportunity employer. We do not discriminate based on gender, race or color, ethnicity or national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other applicable characteristics protected by law. Learn more.
 <br/>
 <br/>
 Adobe aims to make Adobe.com accessible to any and all users. If you have a disability or special need that requires accommodation to navigate our website or complete the application process, email accommodations@adobe.com or call (408) 536-3015.
</div>


Data Engineer

Baselayer

<div>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   About Baselayer
   <br/>
  </strong>
  Trusted by 2,200+ financial institutions, Baselayer is the intelligent business identity platform that helps verify any business, automate KYB, and monitor real-time risk. Baselayer&rsquo;s B2B risk solutions and identity graph network leverage state and federal government filings and proprietary data sources to prevent fraud, accelerate onboarding, and lower credit losses.
 </p>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   About the Role
   <br/>
  </strong>
  We are looking for a Data Engineer to build and scale Baselayer&rsquo;s data infrastructure. You will own the pipelines and data systems that power analytics, reporting, and machine learning across the company, with a focus on reliability, performance, and data quality.
 </p>
 <p>
  This role is hands-on and highly cross-functional. You will work closely with Product and Engineering to ensure data is accessible, trusted, and delivered in a way that supports product capabilities in a regulated environment.
 </p>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   What You&rsquo;ll Do
  </strong>
 </p>
 <ul>
  <li>
   <p>
    Design, build, and maintain scalable data pipelines that ingest, clean, validate, and transform data from internal systems and external sources
   </p>
  </li>
  <li>
   <p>
    Own data reliability and quality through monitoring, alerting, lineage, and validation frameworks
   </p>
  </li>
  <li>
   <p>
    Build and maintain data models and curated datasets that support analytics, dashboards, customer reporting, and downstream ML use cases
   </p>
  </li>
  <li>
   <p>
    Partner with Engineering to define best practices for data architecture, storage, access controls, and performance
   </p>
  </li>
  <li>
   <p>
    Implement orchestration and scheduling for batch and near-real-time workflows as needed
   </p>
  </li>
  <li>
   <p>
    Optimize pipeline performance, cost, and scalability as data volumes grow
   </p>
  </li>
  <li>
   <p>
    Develop and maintain documentation and runbooks for pipelines, datasets, and operational procedures
   </p>
  </li>
  <li>
   <p>
    Identify data gaps and instrumentation needs, and work with engineering teams to improve event capture and logging
   </p>
  </li>
 </ul>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   About You
   <br/>
  </strong>
  You want to learn fast, take ownership, and do work that matters. You are not just doing this for the win. You are doing it because you have something to prove and want to be great.
 </p>
 <p>
  You thrive in the details, care about correctness, and take pride in building robust systems that other teams can rely on. You operate with urgency, handle ambiguity well, and consistently raise the bar on data quality and reliability.
 </p>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   Required Experience and Skills
  </strong>
 </p>
 <ul>
  <li>
   <p>
    1 to 3 years of experience in data engineering, analytics engineering, or backend engineering with significant data pipeline ownership
   </p>
  </li>
  <li>
   <p>
    Strong Python skills and experience building production-grade data workflows
   </p>
  </li>
  <li>
   <p>
    Strong SQL skills with experience designing data models and transforming large datasets
   </p>
  </li>
  <li>
   <p>
    Experience building and maintaining ETL or ELT pipelines and working with data warehouses or analytics databases
   </p>
  </li>
  <li>
   <p>
    Familiarity with orchestration tools and workflow scheduling (for example Airflow, Dagster, Prefect, or similar)
   </p>
  </li>
  <li>
   <p>
    Strong understanding of data quality, testing, observability, and operational best practices
   </p>
  </li>
  <li>
   <p>
    Comfort working with large-scale datasets and troubleshooting performance issues
   </p>
  </li>
  <li>
   <p>
    Ability to communicate clearly with technical and non-technical stakeholders
   </p>
  </li>
 </ul>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   What Sets You Apart
  </strong>
 </p>
 <ul>
  <li>
   <p>
    Experience working with identity, fraud, risk, compliance, or other regulated datasets
   </p>
  </li>
  <li>
   <p>
    Experience integrating with external data sources, APIs, and government or registry data
   </p>
  </li>
  <li>
   <p>
    Familiarity with streaming or near-real-time data patterns
   </p>
  </li>
  <li>
   <p>
    Highly feedback-oriented with a desire for continuous improvement
   </p>
  </li>
 </ul>
 <p>
  &nbsp;
 </p>
 <p>
  <strong>
   Work Location
  </strong>
 </p>
 <ul>
  <li>
   <p>
    Hybrid in SF, in office 3 days per week
   </p>
  </li>
 </ul>
 <p>
  <strong>
   Compensation and Benefits
  </strong>
 </p>
 <ul>
  <li>
   <p>
    Salary range of $122,000 to $167,000
   </p>
  </li>
  <li>
   <p>
    Equity package
   </p>
  </li>
  <li>
   <p>
    Unlimited vacation
   </p>
  </li>
  <li>
   <p>
    Comprehensive health coverage
   </p>
  </li>
  <li>
   <p>
    401(k) with company match
   </p>
  </li>
 </ul>
</div>


Apple

<p>Imagine what you could do here. At Apple, new ideas have a way of becoming great products, services, and customer experiences very quickly. Bring passion and dedication to your job and there's no telling what you can accomplish. We are seeking an exceptional engineer with a background in large scale software development and delivery in hyper-scale distributed computing environments. You will work within cybersecurity software engineering team that drives Apache Spark&rsquo;s state-of-the art features and pushes the boundaries of what is possible in our industry.<br>
<br>
<b>Description</b><br>
<br>
Apple is seeking an extraordinary individual to work on a team supporting and developing services which underpin mission critical information security capabilities. Spark expertise is a prerequisite. While previous experience in information security is not required, a sincere drive for security and privacy is required. This role will work closely with our engineering teams and with partner teams to ensure the successful design, development, and delivery of product objectives. The successful candidate will have a passion for technical excellence and team collaboration with a heavy focus on using cutting-edge technologies to build data intensive systems, designing and optimizing data models, and collaborating with cross-functional teams to solve complex data challenges.<br>
<br>
<b>Preferred Qualifications</b><br>
<br>
5+ years experience in data engineering and demonstrated expertise with data modeling<br>
<br>
5+ years of experience with JVM languages: Java or Scala<br>
<br>
5+ years experience with big data platforms<br>
<br>
Familiarity with ML/AI workflows and feature engineering to support analytics, reporting, and machine learning use cases<br>
<br>
Knowledge engineering expertise including semantic models and knowledge graphs<br>
<br>
Experience working with cloud compute environments like OpenStack, AWS, and Azure<br>
<br>
Experience mentoring engineers, conducting code reviews, and contributing to technical best practices and documentation<br>
<br>
Knowledge of data quality frameworks and validation techniques<br>
<br>
Knowledge of data governance and compliance frameworks<br>
<br>
<b>Minimum Qualifications</b><br>
<br>
Demonstrated expertise in designing and implementing relational and medallion data architectures<br>
<br>
Demonstrated expertise in the design and implementation of production data pipelines at scale<br>
<br>
Demonstrated proficiency with JVM languages (Scala or Java preferred), Golang, or Swift<br>
<br>
Apple is an equal opportunity employer that is committed to inclusion and diversity. We seek to promote equal opportunity for all applicants without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, Veteran status, or other legally protected characteristics. Learn more about your EEO rights as an applicant .<br>
<br>
<b>Pay &amp; Benefits</b><br>
<br>
At Apple, base pay is one part of our total compensation package and is determined within a range. This provides the opportunity to progress as you grow and develop within a role. The base pay range for this role is between $147,400 and $272,100, and your base pay will depend on your skills, qualifications, experience, and location.<br>
<br>
Apple employees also have the opportunity to become an Apple shareholder through participation in Apple's discretionary employee stock programs. Apple employees are eligible for discretionary restricted stock unit awards, and can purchase Apple stock at a discount if voluntarily participating in Apple's Employee Stock Purchase Plan. You'll also receive benefits including: Comprehensive medical and dental coverage, retirement benefits, a range of discounted products and free services, and for formal education related to advancing your career at Apple, reimbursement for certain educational expenses - including tuition. Additionally, this role might be eligible for discretionary bonuses or commission payments as well as relocation. Learn more about Apple Benefits.<br>
<br>
Note: Apple benefit, compensation and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.</p>

Qcells

<div><p><b>Description</b>
</p><h3 class="jobSectionHeader"><b>POSITION DESCRIPTION:</b></h3><p>
We are seeking a <b>Lead Data Engineer</b> to architect, build, and lead the development of scalable, cloud-based data platforms that support enterprise analytics, operational reporting, and advanced data use cases. This role provides technical leadership in designing and optimizing ETL/ELT frameworks using Azure data services (Fabric, Data Lake, Data Factory), integrating data from ERP, CRM, and operational systems, and establishing robust data models within a modern lakehouse architecture.</p><p>
The ideal candidate brings deep SQL and Python expertise, extensive experience with distributed data platforms, and strong knowledge of data architecture, governance, and performance optimization. This individual will serve as a technical leader and mentor, partnering closely with architects, analysts, application teams, and business stakeholders to deliver reliable, scalable, and well-governed enterprise data solutions.</p><h3 class="jobSectionHeader"><b>
RESPONSIBILITIES</b></h3><ul><li>
Lead the design, development, and optimization of scalable data pipelines supporting ingestion, transformation, and enterprise-wide data consumption.</li><li>
Architect and implement enterprise-grade ETL/ELT frameworks using Azure Fabric or comparable cloud data platforms.</li><li>
Oversee and optimize data integrations from ERP (NetSuite/SAP), CRM (Salesforce), internal systems, APIs, and third-party data sources.</li><li>
Design and govern high-quality, scalable data models supporting analytics, reporting, operational systems, and advanced use cases.</li><li>
Partner with Data Architects to define and implement lakehouse patterns, Delta Lake strategies, medallion architecture, and domain-driven design principles.</li><li>
Establish and enforce data quality frameworks, validation standards, lineage tracking, and observability practices.</li><li>
Drive performance optimization, scalability, reliability, and cost governance across cloud environments.</li><li>
Provide technical leadership and mentorship to data engineers; conduct design reviews and enforce engineering best practices.</li><li>
Collaborate cross-functionally with analysts, application teams, and business stakeholders to translate requirements into scalable data solutions.</li><li>
Lead MDM, metadata management, governance, and data standardization initiatives.</li><li>
Oversee CI/CD automation, DevOps integration, testing frameworks, and monitoring strategies for data workflows.</li><li>
Evaluate emerging technologies and recommend platform improvements aligned with enterprise strategy.</li></ul><h3 class="jobSectionHeader"><b>
MINIMUM QUALIFICATIONS</b></h3><ul><li>
10+ years of experience in data engineering, data architecture, or related roles.</li><li>
Proven experience leading large-scale data platform initiatives in cloud environments.</li><li>
Extensive hands-on experience with Azure data services (Data Lake, Data Factory, Fabric, Synapse, or similar).</li><li>
Advanced proficiency in SQL and Python; experience with Spark or distributed processing frameworks.</li><li>
Deep experience designing and implementing enterprise ETL/ELT frameworks.</li><li>
Strong expertise in data modeling (dimensional modeling, star schema, lakehouse/Delta modeling).</li><li>
Experience integrating complex enterprise systems (ERP, CRM, operational platforms).</li><li>
Strong understanding of data governance, metadata management, MDM, and data quality frameworks.</li><li>
Experience with performance tuning, workload optimization, and cloud cost management.</li><li>
Demonstrated ability to lead technical teams, conduct architecture reviews, and mentor engineers.</li><li>
Strong problem-solving, debugging, and system design skills.</li><li>
Travel may be required up to 5%, depending on business needs.</li></ul><h3 class="jobSectionHeader"><b>
PREFERRED QUALIFICATIONS</b></h3><ul><li>
Experience with Delta Tables, Snowflake, Synapse, or comparable cloud data platforms.</li><li>
Experience with event-driven and streaming architectures (Kafka, Event Hub, streaming pipelines).</li><li>
Familiarity with finance, operations, energy, or ERP-driven data domains.</li><li>
Experience designing API-based data integrations and modern integration patterns.</li><li>
Azure certifications (Data Engineer Associate, Solutions Architect, or equivalent).</li><li>
Experience enabling analytics teams, data science workflows, or ML pipelines.</li><li>
Experience implementing enterprise data security and compliance frameworks.</li></ul><br>
Hanwha Q CELLS Technologies, Inc. a subsidiary of Hanwha Q CELLS, one of the world&acute;s largest and most recognized photovoltaic manufacturers for its high-performance, high-quality solar cells and modules. It is headquartered in Seoul, South Korea (Global Executive HQ) Talheim, Germany (Technology &amp; Innovation HQ) and Santa Clara, CA, USA (HW and SW Product Development HQ). Through its growing global business network spanning Europe, North America, Asia, South America, Africa, and the Middle East, the company provides excellent services and long-term partnerships to its customers in the utility, commercial, government, and residential markets. Hanwha Q CELLS is a flagship company of Hanwha Group, a FORTUNE Global 500 firm and a Top 7 business enterprise in South Korea.<br>
<br>
<div><div><div><br>
<b>PHYSICAL, MENTAL &amp; ENVIRONMENTAL DEMANDS:</b><br>
To comply with the Rehabilitation Act of 1973 the essential physical, mental and environmental requirements for this job are listed below. These are requirements <i>normally expected</i> to perform <i>regular</i> job duties. Incumbent must be able to successfully perform all of the functions of the job with or without reasonable accommodation.</div></div><div><div><div><div><b><br>
Mobility</b></div></div><br>
<div><div><br>
Standing</div><div><br>
20% of time</div></div><br>
<div><div><br>
Sitting</div><div><br>
70% of time</div></div><br>
<div><div><br>
Walking</div><div><br>
10% of time</div></div><br>
<div><div><br>
<b>
Strength</b></div></div><br>
<div><div><br>
Pulling</div><div><br>
up to 10 Pounds</div></div><br>
<div><div><br>
Pushing</div><div><br>
up to 10 Pounds</div></div><br>
<div><div><br>
Carrying</div><div><br>
up to 10 Pounds</div></div><br>
<div><div><br>
Lifting</div><div><br>
up to 10 Pounds</div></div><br>
<div><div><br>
<b>
Dexterity</b> <b>(F = Frequently, O = Occasionally, N = Never)</b></div></div><br>
<div><div><br>
Typing</div><div><br>
F</div></div><br>
<div><div><br>
Handling</div><div><br>
F</div></div><br>
<div><div><br>
Reaching</div><div><br>
F</div></div></div><div><div><div><b><br>
Agility</b> <b>(F = Frequently, O = Occasionally, N = Never)</b></div></div><br>
<div><div><br>
Turning</div><div><br>
F</div></div><br>
<div><div><br>
Twisting</div><div><br>
F</div></div><br>
<div><div><br>
Bending</div><div><br>
O</div></div><br>
<div><div><br>
Crouching</div><div><br>
O</div></div><br>
<div><div><br>
Balancing</div><div><br>
N</div></div><br>
<div><div><br>
Climbing</div><div><br>
N</div></div><br>
<div><div><br>
Crawling</div><div><br>
N</div></div><br>
<div><div><br>
Kneeling</div><div><br>
N</div></div></div></div></div><div></div><p><br>
The salary range is required by the California Pay Transparency Act and may differ depending on the location of those candidates hired nationwide. Actual compensation is influenced by a wide array of factors including but not limited to, skill set, education, licenses and certifications, essential job duties and requirements, and the necessary experience relative to the job&rsquo;s minimum qualifications.</p>
<ul><li>This target salary range is for CA positions only and should not be interpreted as an offer of compensation.</li></ul>
You may view your privacy rights by reviewing Qcells' Privacy Policy or by contacting our HR team for a copy.</div>

Lead Data Engineer

OpenAI

<div>
 <strong>
  About The Team
  <br/>
  <br/>
 </strong>
 The Applied team works across research, engineering, product, and design to bring OpenAI&rsquo;s technology to consumers and businesses.
 <br/>
 <br/>
 We seek to learn from deployment and distribute the benefits of AI, while ensuring that this powerful tool is used responsibly and safely. Safety is more important to us than unfettered growth.
 <br/>
 <br/>
 <strong>
  About The Role
  <br/>
  <br/>
 </strong>
 We're seeking a Data Engineer to take the lead in building our data pipelines and core tables for OpenAI. These pipelines are crucial for powering analyses, safety systems that guide business decisions, product growth, and prevent bad actors. If you're passionate about working with data and are eager to create solutions with significant impact, we'd love to hear from you. This role also provides the opportunity to collaborate closely with the researchers behind ChatGPT and help them train new models to deliver to users. As we continue our rapid growth, we value data-driven insights, and your contributions will play a pivotal role in our trajectory. Join us in shaping the future of OpenAI!
 <br/>
 <br/>
 <strong>
  In This Role, You Will
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Design, build and manage our data pipelines, ensuring all user event data is seamlessly integrated into our data warehouse.
  </li>
  <li>
   Develop canonical datasets to track key product metrics including user growth, engagement, and revenue.
  </li>
  <li>
   Work collaboratively with various teams, including, Infrastructure, Data Science, Product, Marketing, Finance, and Research to understand their data needs and provide solutions.
  </li>
  <li>
   Implement robust and fault-tolerant systems for data ingestion and processing.
  </li>
  <li>
   Participate in data architecture and engineering decisions, bringing your strong experience and knowledge to bear.
  </li>
  <li>
   Ensure the security, integrity, and compliance of data according to industry and company standards.
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  You Might Thrive In This Role If You
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Have 3+ years of experience as a data engineer and 8+ years of any software engineering experience(including data engineering).
  </li>
  <li>
   Proficiency in at least one programming language commonly used within Data Engineering, such as Python, Scala, or Java.
  </li>
  <li>
   Experience with distributed processing technologies and frameworks, such as Hadoop, Flink and distributed storage systems (e.g., HDFS, S3).
  </li>
  <li>
   Expertise with any of ETL schedulers such as Airflow, Dagster, Prefect or similar frameworks.
  </li>
  <li>
   Solid understanding of Spark and ability to write, debug and optimize Spark code.
   <br/>
   <br/>
  </li>
 </ul>
 This role is exclusively based in our San Francisco HQ. We offer relocation assistance to new employees.
 <br/>
 <br/>
 <strong>
  About OpenAI
  <br/>
  <br/>
 </strong>
 OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. We push the boundaries of the capabilities of AI systems and seek to safely deploy them to the world through our products. AI is an extremely powerful tool that must be created with safety and human needs at its core, and to achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity.
 <br/>
 <br/>
 We are an equal opportunity employer, and we do not discriminate on the basis of race, religion, color, national origin, sex, sexual orientation, age, veteran status, disability, genetic information, or other applicable legally protected characteristic.
 <br/>
 <br/>
 For additional information, please see OpenAI&rsquo;s Affirmative Action and Equal Employment Opportunity Policy Statement.
 <br/>
 <br/>
 Background checks for applicants will be administered in accordance with applicable law, and qualified applicants with arrest or conviction records will be considered for employment consistent with those laws, including the San Francisco Fair Chance Ordinance, the Los Angeles County Fair Chance Ordinance for Employers, and the California Fair Chance Act, for US-based candidates. For unincorporated Los Angeles County workers: we reasonably believe that criminal history may have a direct, adverse and negative relationship with the following job duties, potentially resulting in the withdrawal of a conditional offer of employment: protect computer hardware entrusted to you from theft, loss or damage; return all computer hardware in your possession (including the data contained therein) upon termination of employment or end of assignment; and maintain the confidentiality of proprietary, confidential, and non-public information. In addition, job duties require access to secure and protected information technology systems and related data security obligations.
 <br/>
 <br/>
 To notify OpenAI that you believe this job posting is non-compliant, please submit a report through this form. No response will be provided to inquiries unrelated to job posting compliance.
 <br/>
 <br/>
 We are committed to providing reasonable accommodations to applicants with disabilities, and requests can be made via this link.
 <br/>
 <br/>
 OpenAI Global Applicant Privacy Policy
 <br/>
 <br/>
 At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.
 <br/>
 <br/>
 Compensation Range: $230K - $385K
</div>


Data Engineer, Analytics

Afresh

<div><div><p>Afresh is the leading AI company in fresh food&mdash;partnering with grocers like Albertsons, Wakefern, Meijer, and Stater Bros to order billions of dollars of fresh food in over 12,000 grocery departments nationwide.</p>
<p>
Following record-breaking 70% growth in 2025, we've expanded our platform to cover all fresh departments, launched our full store suite, and debuted DC Fresh Buying.</p>
<p>
We're on a mission to eliminate food waste and make fresh food accessible to all. In 2025 alone our software helped save 200M lbs of food waste. If you're looking for a role where your work directly translates into massive scale and social good, and you want to be part of the team that defines the future of fresh, there is no better time to join us.</p></div><p><b>
About the Role</b></p>
<p>
As a Senior Data Engineer, you'll play a key role in scaling and improving how we integrate and process customer data. You will design and implement ETLs that reliably process large volumes of customer-provided data and build tools/improve the platform to make customer integrations faster, more accurate, and more scalable. You'll also contribute to the development of new features that support our expanding product lines. Your work will have a direct and visible impact on our ability to onboard customers more easily and quickly and power our machine learning grocery solution.</p>
<p><b>
What You'll Do</b></p>
<ul>
<li>Build tools and frameworks that streamline customer integrations, enabling faster onboarding and better handling of customer data.</li>
<li>Create robust ETLs in PySpark and DBT to process billions of records from customer datasets, ensuring data is accurate, reliable, and ready for downstream use.</li>
<li>Investigate and implement new technologies into the data platform, focusing on practical solutions that address current pain points and anticipate future needs.</li>
<li>Collaborate with product, engineering, and go-to-market teams to design and deliver data solutions for new products and features.</li>
<li>Identify and implement optimizations to improve ETL runtime and data processing scalability, reducing the time and effort required for integrations.</li>
<li>Solve real-world data quality challenges by working directly with messy, incomplete, or inconsistent customer data to extract the signal we need.</li>
<li>Support team members by mentoring engineers, leading technical discussions, and providing clear, actionable feedback.</li>
</ul>
<p><b>What Makes You a Great Fit</b></p>
<p>
We encourage all highly-qualified candidates to apply, even if they don't meet every listed qualification.</p>
<ul>
<li>Significant experience designing and maintaining ETLs that process large-scale datasets.</li>
<li>Proficiency with Python, PySpark, SQL, and experience working on platforms/tools like Databricks, Snowflake, or DBT.</li>
<li>Strong problem-solving skills and the ability to work with ambiguous or incomplete requirements to deliver concrete, impactful solutions.</li>
<li>A focus on practical outcomes&mdash;you're skilled at balancing technical rigor with the need to get things done.</li>
<li>Experience working directly with complex, unclean datasets and finding innovative ways to process and analyze them.</li>
<li>A knack for identifying areas where tooling or automation can simplify workflows and reduce manual effort.</li>
<li>Excellent communication skills&mdash;you're able to explain your ideas clearly to both technical and non-technical audiences.</li>
<li>Proven leadership in technical projects, with a willingness to mentor and help others grow.</li>
</ul>
<p><b>Why Work Here</b></p>
<ul>
<li>Join a mission-driven company reducing millions of pounds of food waste in grocery stores per year.</li>
<li>Work on challenging, real-world problems that have a direct impact on our customers.</li>
<li>Be part of a collaborative, supportive team where your ideas are valued and acted on.</li>
<li>Use cutting-edge tools and platforms to solve meaningful data challenges.</li>
</ul>
<p>We're looking for someone who thrives on tackling complex data problems and takes pride in building systems that work seamlessly at scale. If that sounds like you, we'd love to hear from you!</p>
<p><b>
This position is not eligible for employer sponsorship.</b></p>
<p>
Salary Range in U.S.: $156,000 - $211,000</p>
<p>
Salary Range in Canada (CAD): $137,000 - $185,000</p><p></p><div><p><b><br>
About Afresh</b><br>
<br>
Founded in 2017, Afresh is working on the #1 solution to curb climate change: reducing food waste. By combining human insight and transformative technology, we're helping grocers provide fresher food to customers at more affordable prices.
<br>
<br>
Afresh sits at an incredible intersection of positive social impact, rocket ship financial growth, and cutting-edge technology. Our best-in-class AI research has been published in top journals including ICML, and we've raised over $148 million in funding from investors including former co-CEO of Whole Foods Market Walter Robb and Eric Schmidt's Innovation Endeavors.</p>
<p>
Fresh is the past, present, and future of our food system &ndash; the waste we create today will impact our planet for years to come. Join us as we continue to build a vibrant, diverse, and inclusive team that embodies our company's values of proactivity, kindness, candor, and humility.<br>
<br>
<b>Afresh provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, sex, national origin, age, disability, genetics, sexual orientation, gender identity/expression, marital status, pregnancy or related condition, or any other basis protected by law.</b></p>
<p><b>
Here at Afresh, many of our employees work remotely provided that they reside in one of the following states: AL, AR, CA, CO, FL, GA, IL, KY, MA, MI, MT, MO, NV, NJ, NY, NC, OR, PA, TX, WA, UT, VA, WI</b>.</p></div></div>

Senior Data Engineer

Data Engineer, Product Analytics

Explore 269+ Data engineer in San Francisco this March. Find roles across top companies and apply now to boost your career.

269 data engineer jobs In San Francisco

Data Engineer

Data Engineer

Data Engineer

Lead Data Engineer

Data Engineer, Analytics

Senior Data Engineer

Data Engineer, Product Analytics