Senior Data Engineer

The Senior Data Engineer within the AI Platform team is responsible for designing, building, and scaling enterprise-grade data pipelines and data services that power AI/ML applications, analytics, and operational intelligence use cases.

The team focuses on enabling secure, high-quality, and real-time data access across enterprise platforms, including data ingestion, transformation, API integration, and feature engineering to support machine learning workflows.

This role partners closely with AI/ML engineers, platform engineering (CAP/EDP), and product teams to develop scalable and compliant data solutions that support healthcare use cases, including handling PHI/PII data under strict governance and security standards. The position plays a key role in modernizing data platforms, enabling reusable data services, and improving performance, reliability, and observability of enterprise data systems.

Required Qualifications

• 5+ years of experience in Data Engineering, Software Engineering, or related field
• 3+ years of hands-on experience developing ETL/ELT pipelines using tools/frameworks such as Spark, Databricks, or similar distributed processing systems
• Strong proficiency in SQL and at least one programming language such as Python or Scala
• Experience working with at least one cloud platform (AWS, Azure, or GCP)
• Experience designing and implementing data pipelines for batch and/or streaming data processing
• Experience with data modeling, data warehousing, or lakehouse architectures
• Experience integrating systems using APIs, event-driven architectures (e.g., Kafka), or microservices
• Experience implementing data quality, validation, and monitoring frameworks
• Experience working with large-scale structured and unstructured datasets
• Experience handling sensitive data (PHI/PII) or working in regulated environments
• Experience using version control (e.g., Git) and CI/CD pipelines

Preferred Qualifications (non-measurable / behavioral / exposure-based)

• Strong problem-solving and analytical thinking skills
• Ability to design scalable, reusable, and resilient data solutions aligned with enterprise platforms
• Experience collaborating with cross-functional teams including AI/ML, product, and platform engineering
• Familiarity with AI/ML workflows, feature engineering, and model data pipelines
• Understanding of healthcare data domains such as claims, customer care, or clinical data
• Knowledge of data observability, lineage, and monitoring best practices
• Experience contributing to platform modernization initiatives (e.g., API gateway migration, data platform consolidation)
• Ability to mentor junior engineers and contribute to technical design reviews
• Strong communication skills with ability to translate business needs into technical solutions

Education

• Bachelor’s degree or equivalent work experience in Mathematics, Statistics, Computer Science, Engineering, or related discipline required
• Master’s degree preferred