Azure Databricks Data Engineer/Developer

Job Description:

• Looking for a skilled Data Engineer to join our team
• The ideal candidate should have experience in Databricks SQL PySpark Spark Python and Azure Data Factory ADF
• The candidate should be able to design develop and maintain data pipelines and data streams
• The candidate should also be able to move transform data across layers Bronze Silver Gold using ADF Python and PySpark
• Responsibilities
• Design and build modern data pipelines and data streams
• Move Transform data across layers Bronze Silver Gold using ADF Python and PySpark
• Develop and maintain data pipelines and data streams
• Work with stakeholders to understand their data needs and provide solutions
• Collaborate with other teams to ensure data quality and consistency
• Develop and maintain data models and data dictionaries
• Develop and maintain ETL processes
• Develop and maintain data quality checks
• Develop and maintain data governance policies and procedures
• Develop and maintain data security policies and procedures
• Provide technical guidance and mentorship to junior data engineers

Key Responsibilities:

• Requirements
• Experience in Databricks SQL PySpark Spark Python and Azure Data Factory ADF
• Experience in designing developing and maintaining data pipelines and data streams
• Experience in moving transforming data across layers Bronze Silver Gold using ADF Python and PySpark
• Experience in working with stakeholders to understand their data needs and provide solutions
• Experience in collaborating with other teams to ensure data quality and consistency
• Experience in developing and maintaining data models and data dictionaries
• Experience in developing and maintaining ETL processes
• Experience in developing and maintaining data quality checks
• Experience in developing and maintaining data governance policies and procedures
• Experience in developing and maintaining data security policies and procedures

Technical Requirements:

• Primary skills Technology Cloud Platform Azure Development Solution Architecting

Additional Responsibilities:

• Knowledge of more than one technology
• Basics of Architecture and Design fundamentals
• Knowledge of Testing tools
• Knowledge of agile methodologies
• Understanding of Project life cycle activities on development and maintenance projects
• Understanding of one or more Estimation methodologies Knowledge of Quality processes
• Basics of business domain to understand the business requirements
• Analytical abilities Strong Technical Skills Good communication skills
• Good understanding of the technology and domain
• Ability to demonstrate a sound understanding of software quality assurance principles SOLID design principles and modelling methods
• Awareness of latest technologies and trends
• Excellent problem solving analytical and debugging skills