Senior Data Engineer

QuantumGate is dedicated to developing and commercializing cutting-edge post-quantum cryptographic solutions. Our mission is to safeguard enterprise digital environments through innovative protocols and applications that address the evolving challenges of the post-quantum era.

We are seeking a talented and motivated Data Engineer to join our fast-growing ML-team. The ideal candidate has strong experience with distributed data processing, NoSQL databases, and modern data lake technologies. You will play a key role in designing and maintaining scalable data pipelines that power business intelligence, analytics, and machine learning initiatives.

Key Responsibilities

• Design, develop, and maintain scalable ETL/ELT pipelines using Apache Spark and/or similar solutions.
• Manage and optimize large datasets stored in Apache Iceberg tables.
• Design, build, and optimize data infrastructure.
• Integrate data from various sources including MongoDB, APIs, and flat files.
• Optimize data processing workflows for batch and real-time analytics.
• Implement and enforce data governance, data quality, and security best practices.
• Troubleshoot and resolve data-related issues in dev/production environments.
• Work closely with data scientists, analysts, and product teams to understand data needs and deliver efficient solutions.
• Monitor pipeline performance and troubleshoot data quality issues.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field.
• 3+ years of experience in data engineering or related roles.
• Strong hands-on experience with:
• o Apache Spark (PySpark, Scala, or Java)
• o MongoDB (data modeling, aggregation framework, replication)
• o Apache Iceberg or other data lake table formats (Delta Lake, Hudi)
• Proficient in SQL and at least one programming language (Python, Scala, or Java).
• Experience with data pipeline orchestration tools (e.g., Airflow).
• Understanding of data warehousing concepts (star schema, partitioning).
• Familiarity with cloud platforms such as AWS or Azure.

Nice to have qualifications

• Experience with stream processing (e.g., Kafka, Flink).
• Experience with Kubernetes for containerized data workloads.
• Familiarity with CI/CD practices and infrastructure as a code (e.g., Terraform)

Join us to drive innovation and shape the future of technology!