Data Engineering Manager

The Data Engineering Manager (DEM) will lead Fred Hutch Cancer Centers centrally managed, cloud-based translational data infrastructure and user-facing applications as a hands-on manager. The person in this role will oversee a small team of data and platform engineers while actively contributing to development and design. The engineering team will ingest data from various sources, perform quality control, and operate secure, compliant services that power data science, precision oncology, and AI applications across research and clinical domains. The DEM will work closely with Product Management, IT, and business stakeholders to create practical roadmaps; partner with DaSL's Data Science & Software and Data Governance teams to scale governance alongside capabilities; and champion engineering excellence through clear standards, documentation, and reproducible practices in our hybrid, written-first culture.

The individual in this role will

• Ensure the translational data platform is secure, compliant, reliable, and cost-aware to support sustainable, performant, well-governed, analysis-ready data products.
• Turn ambiguity into clear interfaces and service expectations (SLAs/SLOs) and guide incremental delivery plans that achieve predictable outcomes.
• Raise the engineering bar through thoughtful code and design reviews, CI/CD and automated testing, infrastructure as code, and actionable observability.
• Lead blameless learning after incidents and drive follow-through that produces durable, organization-wide improvements.

• Contribute Python and SQL with Spark and Databricks to build and operate pipelines and data services for structured, semi-structured, and unstructured sources, landing in Delta Lake or appropriate storage layers with robust schema evolution and content-aware validation.
• Use Databricks tools such as Workflows and Delta Live Tables where appropriate; automate validations and backfills; tune for performance and cost on AWS across tabular datasets, JSON/event streams, documents, and large binary objects.
• Instrument workloads with actionable observability, maintain runbooks, and participate in on-call to improve mean time to detect and mean time to restore.
• Advance CI/CD for data and code, including test strategies for PySpark jobs, dbx/Repos workflows, and safe promotion between environments.
• Apply infrastructure as code for repeatable environments and permissions, coordinating with platform and security teams on identity and access management.

• Design data models, metadata strategies, and table or API contracts that work across structured and unstructured domains, including patterns for file stores, object storage, and discoverability via catalog and lineage.
• Lead architecture decision records, make pragmatic build-versus-buy calls, and provide written guidance that scales.
• Integrate governance and privacy by design with Unity Catalog policies, lineage, masking, and PHI workflows; partner with Data Governance on HIPAA-aligned practices that cover both tabular assets and unstructured content.
• Drive cost awareness through storage and compute posture, file layout and partitioning, caching strategies, and lifecycle policies tailored to differing access patterns across data types.

• Support and develop engineers through regular one-on-ones, feedback, growth planning, and performance reviews; build a diverse and inclusive team.
• Run predictable delivery with Product Management and stakeholders; decompose scope, surface risks and dependencies, and communicate progress and escalations with clarity.
• Foster written-first collaboration to ensure patterns, decisions, and interfaces are well-documented and accessible in our hybrid work environment.

• Integrate external consultants as force multipliers while maintaining clear ownership and quality standards.
• Establish expectations for code quality, documentation, security controls, and knowledge transfer; evaluate deliverables against requirements and internal standards.
• Coordinate with IT security and identity teams for onboarding, least-privilege access, and offboarding; ensure sustained maintainability after vendor engagement ends.

• BA/BS in Computer Science, Computer Engineering, Information Technology, Management Information Systems, Software Engineering, Software Development, Data Engineering or related field, or equivalent combination of education and experience.
• 7+ years of experience supporting or developing data infrastructure and technology systems with increasing technical and leadership responsibilities.

• Recent hands-on experience with Databricks, Spark, SQL, and Delta or Parquet at scale; strong Python skills for data engineering.
• Experience delivering reliable pipelines on AWS (S3, IAM, etc.) using CI/CD, automated testing, and infrastructure as code.
• Proven ability to lead people and projects while remaining technically hands-on, including hiring, performance management, and coaching.
• Strong written and verbal communication skills when working with clinical, research, and IT stakeholders in regulated environments
• Healthcare or research background with knowledge of HIPAA and PHI requirements
• Track record of establishing SLOs, observability practices, on-call rotations, and blameless post-mortems.
• Proficiency with Teams, Slack, Jira, and Confluence; strong documentation habits suited to a hybrid, written-first culture.