Lead I - Data Engineering

5 - 7 Years
1 Opening
Bangalore, Hosur

Experienced Data Engineer with 5+ years of hands-on development using Python (preferred), along with strong proficiency in Scala, Java, or C#. Skilled in designing and building high-performance, scalable data pipelines and architectures. Specialized in working with PySpark on Databricks, and experienced across a range of data engineering tools including Snowflake, BigQuery, Apache Spark, Hive, Hadoop, Cloudera, and Redshift.

Key Competencies:

• Programming & Development: 5+ years of experience developing data-centric applications using Python (preferred), with additional expertise in Scala, Java, or C#.

• Big Data & Cloud Platforms: Deep experience with cloud-based and on-premise data platforms including Databricks (PySpark), Snowflake, BigQuery, Hive, and Hadoop ecosystems.

• Data Pipeline Architecture: Proven ability to design and implement high-performance ETL/ELT pipelines with strong guarantees around idempotency, exactly-once and at-least-once processing, fault tolerance, eventual and transactional consistency, and observability.

• Containerization & DevOps: Proficient in developing and deploying applications in containerized environments using Docker, Kubernetes, and Rancher.

• Workflow Orchestration: Experienced in orchestrating complex data workflows using Apache Airflow or similar tools.

• Education: Bachelor’s degree in Computer Science or a related field, or equivalent combination of education and professional experience

Python,Pyspark,Databricks