Data Engineer/Spark Engineer

Key Responsibilities

• Data Architecture & Design: Design and implement scalable data infrastructure including OLTP databases, data lakes, and cloud-based data warehouses.
• Spark-Based Data Engineering: Build and optimize data pipelines using Apache Spark (PySpark/Scala) for batch and real-time processing.
• Cloud Data Solutions: Leverage GCP services such as BigQuery, Dataflow, Dataproc, and Cloud Storage to build robust data solutions.
• ETL/ELT Development: Develop efficient ETL/ELT workflows to ingest, transform, and load data from diverse sources.
• Data Modeling & Governance: Define data models, enforce data quality standards, and implement governance practices across the data lifecycle.

Required Skills & Qualifications

• Experience: 2+ years in data engineering with proven experience in building data platforms from scratch.
• Spark Expertise: Hands-on experience with Apache Spark (PySpark or Scala) in production environments.
• Cloud Proficiency: Strong experience with Google Cloud Platform data services (BigQuery, Dataflow, Dataproc, Cloud Storage).
• Big Data Ecosystem: Solid understanding of Hadoop, Hive, and distributed data processing.
• Databases: Expertise in SQL and experience with NoSQL databases (e.g., MongoDB, Cassandra).
• Data Warehousing: Familiarity with data warehouse design principles (Kimball/Inmon) and tools like BigQuery, Snowflake, or Redshift.