Lead Data Engineer (Remote)

This position is posted by Jobgether on behalf of a partner company. We are currently looking for a Lead Data Engineer - REMOTE. In this role, you will be pivotal in shaping the data infrastructure, enabling the seamless flow of information across various platforms. Your contribution will directly influence how data is utilized to drive business insights and decision-making. By architecting robust data pipelines and implementing cutting-edge data solutions, you will help elevate the company's analytical capabilities. Collaborating with cross-functional teams, you will ensure data integrity and security while setting the stage for innovative data-driven strategies.

Accountabilities

• Architect, build, and maintain complex data pipelines using Azure Data Factory, Python/PySpark, and Databricks.
• Design and implement modern data warehouse solutions, optimizing for performance and scalability.
• Ingest, cleanse, and transform data from various sources into usable structures for analytics.
• Implement security features, including role-based access control (RBAC) and data encryption.

Requirements

• Troubleshoot and optimize data systems and SQL queries for efficiency.
• Lead code reviews, mentor junior engineers, and define technical standards.
• Work collaboratively with data scientists, analysts, and stakeholders.
• Experience with Azure Data Factory, Databricks, Synapse Analytics, and Data Lake Storage.
• Proficiency in Python/PySpark, SQL, Scala, and CI/CD tools.
• Knowledge of ETL/ELT processes, data modeling, and data processing.

Benefits

• Commitment to building a diverse and inclusive workforce.
• Flexibility in work arrangements according to the needs of employees.
• Support programs for reintegrating employees after breaks due to health or family reasons.