Data Engineer

The Main Event: What You ll Drive, Build, and Own

Design and Scale Data Pipelines: Build and maintain reliable batch and streaming pipelines that connect our applications, events, and external sources to Snowflake, ensuring data flows seamlessly across our platform.

Streaming Data at Scale: Work with Kafka, Schema Registry, and event-driven architectures to power real-time data products and ensure downstream systems get trustworthy, well-structured events.

Pipeline Orchestration: Use Airflow to orchestrate complex workflows, automate dependencies, and ensure data availability for analytics and business processes.

Data Infrastructure & Cloud Services: Leverage AWS tools (ECR, ECS, S3, etc.) to build scalable, containerized data services and optimize storage, compute, and costs.

Data Quality & Observability: Implement testing frameworks, monitoring, and alerting for data pipelines to ensure reliability and trust at scale.

Infrastructure as Code: Manage data infrastructure with tools like Terraform, Helmfile, or CloudFormation, automating deployments and ensuring reproducibility.

Optimize for Scale and Performance: Continuously improve the throughput, latency, and cost efficiency of pipelines, ensuring the platform scales with the business.

Modeling & Governance: Apply strong data modeling fundamentals (e.g., Kimball, Data Vault) and enforce governance around PII, compliance, and lifecycle management.

Collaboration for Impact: Partner with analysts, business stakeholders, and product teams to translate requirements into well-designed pipelines and models that unlock business value.

Automation & CI/CD: Implement testing, linting, and automated deployments for data pipelines to ensure high-quality, production-ready code.