Sr. Data Engineer - Monetization & Experimentation

RedCloud Consulting is a business and IT consulting company with local Puget Sound Enterprise and Mid-sized clients. RedCloud seeks a Senior Data Engineer to support immediate client operations. Seattle Business Magazine has recognized us, ranked #1 on their "Best Companies to Work for in Washington" for Mid-Sized Businesses list, awarded #1 Fastest Growing Company in Washington by Puget Sound Business Journal, and named on the Inc. 500/5000 list.

Overview:
This role supports large-scale, fast-evolving advertising systems where priorities change rapidly and perfect requirements are rare. The Senior Data Engineer will design, build, and operate massive-scale data pipelines that power bidding, targeting, experimentation, and measurement. Ideal candidates are deeply technical, strong owners, and able to deliver durable solutions under ambiguity and infrastructure constraints.
Build and operate high-reliability, high-scale data systems that enable real-time advertising decisioning and analytics. This engineer will own critical pipelines end-to-end, reduce recurring operational pain, increase efficiency, and drive innovation in an ads ecosystem where scale, speed, and correctness are essential.
Team works on Pacific Time. Preference for Seattle-local candidates.

Responsibilities include but are not limited to:

• Design, build, and maintain large-scale data pipelines supporting bidding, targeting, experimentation, and measurement.
• Optimize workflows under real-world constraints—sampling limits, narrow data windows, service ceilings, and high-throughput requirements.
• Ensure reliability, performance, and cost-efficiency across batch and streaming workloads.
• Build scalable solutions that deliver value today while creating a path for evolution over time.
• Own pipelines end-to-end, from design to deployment to maintenance.
• Diagnose, resolve, and eliminate recurring issues to strengthen system stability.
• Reduce duplicated workflows, wasted compute, and operational overhead.
• Establish engineering best practices around quality, testing, deployment, and observability.
• Partner closely with platform teams, infra teams, and other data/ML groups to unblock scaling constraints.
• Work directly with Data Science, ML Engineering, and Product to deliver features, unblock analysis, and support experimentation.
• Translate ambiguous requirements into well-structured engineering solutions.
• Operate with urgency in an environment where priorities shift quickly.
• Drive initiatives from zero to one with minimal guidance.
• Innovate within constraints and deliver solutions that work in the real world—not just on paper.

Required Knowledge, Skills, and Abilities:

• Deep expertise in at least one major big data ecosystem (e.g., Spark, Flink, EMR, Hadoop-style systems).
• Strong proficiency with distributed compute frameworks (Spark strongly preferred).
• Experience building and operating pipelines at massive scale, including challenges like:
  • Streaming ingestion
  • High-volume batch ETL
  • Strict SLAs or constrained windows
  • Sampling and stateful processing
• Hands-on experience with cloud-based data infrastructure (AWS or equivalent), including:
  • Object storage (e.g., S3)
  • Data lake formats (Iceberg/Hudi-style tables)
  • SQL engines (Redshift, Athena, Snowflake, etc.)
  • Streaming frameworks (Kinesis or Kafka-style systems)
• Strong coding skills in Python, Java, and/or Scala.
• Experience with infrastructure-as-code frameworks (e.g., Terraform).
• Strong engineering discipline with Git, CI/CD, testing frameworks, and automated deployment processes.
• Exposure to advertising, experimentation, or ML-driven systems is highly beneficial:
  • Real-time bidding / auction systems
  • Feature pipelines or feature stores
  • Measurement and attribution systems
  • Online/offline ML model pipelines
• High ownership mindset: “I will figure it out and ship it.”
• Bias for action and comfort moving quickly with incomplete requirements.
• Strong ability to operate independently and deliver end-to-end solutions.
• Adaptability in dynamic environments with fast-changing priorities.
• Curiosity, learning speed, and enjoyment of solving hard scaling problems.

Technology & Tooling Experience:

• Cloud & Compute: AWS, Kubernetes, Lambda, Step Functions
• Data Lake: S3, Iceberg/Hudi-like table formats
• Analytics: Redshift, Athena, EMR/Spark
• Batch & Orchestration: Spark, Airflow-style orchestration
• ML/Modeling: SageMaker or custom ML platforms
• Languages: Python, Java, Scala, SQL
• Infra & DevOps: Terraform, Git, CI/CD

Compensation range for position is $ 200,000 – 280,000 DOE.
Benefits and bonus information can be found at https://www.redcloudconsulting.com/careers
RedCloud requires employees have the legal ability to work in the United States during their employment period. During onboarding, proof of eligibility to work in the United States will be requested. RedCloud does not provide visa sponsorship.

About Us:
RedCloud is a boutique, business and technology consulting firm providing local companies with expert-level support for over two decades. Whether it’s to solve a specific business challenge or to provide additional support for an ambitious project, we can help bring even the most visionary endeavors to fruition.

Anchored by a foundation of "integrity-based consulting", the RedCloud team of subject matter experts collaborate closely with clients to develop and implement high-level solutions, bringing stability, growth, and innovation together for long-term success. We provide a broad array of business and technology consulting services through RedCloud’s core practices: Data & AI, Delivery Excellence, Digital Solutions, Marketing & Sales, Privacy, Security, and Strategy & Transformation.



Visit http://www.redcloudconsulting.com/ for more info.
#LI-Remote