Data Engineer – Financial Infrastructure & Analytics

About the Role

As a Quantitative Data Engineer , you will be the backbone of the data ecosystem powering our quantitative research, trading, and AI-driven strategies . You will design, build, and maintain the high-performance data infrastructure that enables low-latency, high-fidelity access to market, fundamental, and alternative data across multiple asset classes.

This role bridges quant engineering, data systems, and research enablement , ensuring that our researchers and traders have fast, reliable, and well-documented datasets for analysis and live trading. You’ll be part of a cross-functional team working at the intersection of finance, machine learning, and distributed systems .

Responsibilities

• Architect and maintain scalable ETL pipelines for ingesting and transforming terabytes of structured, semi-structured, and unstructured market and alternative data.
• Design time-series optimized data stores and streaming frameworks to support low-latency data access for both backtesting and live trading.
• Develop ingestion frameworks integrating vendor feeds (Bloomberg, Refinitiv, Polygon, Quandl, etc.), exchange data, and internal execution systems.
• Collaborate with quantitative researchers and ML teams to ensure data accuracy, feature availability, and schema evolution aligned with modeling needs.
• Implement data quality checks, validation pipelines, and version control mechanisms for all datasets.
• Monitor and optimize distributed compute environments (Spark, Flink, Ray, or Dask) for performance and cost efficiency.
• Automate workflows using orchestration tools (Airflow, Prefect, Dagster) for reliability and reproducibility.
• Establish best practices for metadata management, lineage tracking, and documentation.
• Contribute to internal libraries and SDKs for seamless data access by trading and research applications.

In Trading Firms, Data Engineers Typically:

• Build real-time data streaming systems to capture market ticks, order books, and execution signals.
• Manage versioned historical data lakes for backtesting and model training.
• Handle multi-venue data normalization (different exchanges and instruments).
• Integrate alternative datasets (satellite imagery, news sentiment, ESG, supply-chain data).
• Work closely with quant researchers to convert raw data into research-ready features .
• Optimize pipelines for ultra-low latency where milliseconds can impact P&L.
• Implement data observability frameworks to ensure uptime and quality.
• Collaborate with DevOps and infra engineers to scale storage, caching, and compute.

Tech Stack

• Languages: Python, SQL, Scala, Go, Rust (optional for HFT pipelines)
• Data Processing: Apache Spark, Flink, Ray, Dask, Pandas, Polars
• Workflow Orchestration: Apache Airflow, Prefect, Dagster
• Databases & Storage: PostgreSQL, ClickHouse, DuckDB, ElasticSearch, Redis
• Data Lakes: Delta Lake, Iceberg, Hudi, Parquet
• Streaming: Kafka, Redpanda, Pulsar
• Cloud & Infra: AWS (S3, EMR, Lambda), GCP, Azure, Kubernetes
• Version Control & Lineage: DVC, MLflow, Feast, Great Expectations
• Visualization / Monitoring: Grafana, Prometheus, Superset, DataDog
• Tools for Finance: kdb+/q (for tick data), InfluxDB, QuestDB

What You Will Gain

• End-to-end ownership of core data infrastructure in a high-impact, mission-critical domain.
• Deep exposure to quantitative research workflows , market microstructure , and real-time trading systems .
• Collaboration with elite quantitative researchers, traders, and ML scientists.
• Hands-on experience with cutting-edge distributed systems and time-series data technologies .
• A culture that emphasizes technical excellence, autonomy, and experimentation.

Qualifications

• Bachelor’s or Master’s in Computer Science, Data Engineering, or related field.
• 2+ years of experience building and maintaining production-grade data pipelines .
• Proficiency in Python , SQL , and frameworks like Airflow , Spark , or Flink .
• Familiarity with cloud storage and compute (S3, GCS, EMR, Dataproc) and versioned data lakes (Delta, Iceberg) .
• Experience with financial datasets , tick-level data , or high-frequency time series is a strong plus.
• Strong understanding of data modeling, schema design, and performance optimization .
• Excellent communication skills with an ability to support multidisciplinary teams .