Technology & Transformation - Engineering - Senior Consultant/Manager - Azure Data Bricks (ADB)

Senior Data Engineer

Total Experience - 7 to 11 Years

Relevant Experience - 5+

Keywords – Python, Databricks, Pyspark, SQL, Live / Streaming data, batch processing data

Roles and Responsibilities:

The Data Engineer will work on data engineering projects for various business units, focusing on delivery of complex data management solutions by leveraging industry best practices. They work with the project team to build the most efficient data pipelines and data management solutions that make data easily available for consuming applications and analytical solutions. A Data engineer is expected to possess strong technical skills.

Key Characteristics

Technology champion who constantly pursues skill enhancement and has inherent curiosity to understand work from multiple dimensions.

Interest and passion in Big Data technologies and appreciates the value that can be brought in with an effective data management solution.

Has worked on real data challenges and handled high volume, velocity, and variety of data.

Excellent analytical & problem-solving skills, willingness to take ownership and resolve technical challenges.

Contributes to community building initiatives like CoE, CoP.

Knowledge of and skills in various programming languages primarily Python

• Must to have knowledge on Back-end frameworks
• Thorough understanding of containers and functions. Deployment experience with Kubernetes (K8s) or Functions is highly desirable.
• Experience using Cloud Native CI/CD tools (Azure Pipelines/Circle CI/Jenkins X).

Mandatory skills:

Python - Master

Azure - Master

ELT - Skill

Data Modeling - Skill

Data Integration & Ingestion - Skill

Data Manipulation and Processing - Skill

GITHUB, Action, Azure DevOps - Skill

Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest - Skill

Optional skills:

Experience in project management, running a scrum team.

Experience working with BPC, Planning.

Exposure to working with external technical ecosystem.