Data Engineer (Databricks, Azure)

Client: Our client is an international investment company.
Project overview: Our team is currently developing a new Data Warehouse and maintain a several applications and services that are tightly integrated with each other and ETL processes support.
Position overview: Our client is looking for a skilled Data Engineer with strong experience in Databricks and Azure to join their team. This role focuses on both new development (70%) and maintenance of these developments (30%) within Azure. The position is fully remote, based in India, with work hours that overlap with U.S. Eastern Time, requiring availability until 2 PM EST.

• Responsibilities: Create and manage scalable data pipelines to collect, process, and store large volumes of data from various sources
• Integrate data from multiple sources, ensuring consistency, quality, and reliability
• Design, implement, and optimize database schemas and structures to support data storage and retrieval
• Develop and maintain ETL (Extract, Transform, Load) processes to accurately and efficiently move data between systems
• Build and maintain data warehouses to support business intelligence and analytics needs
• Optimize data processing and storage performance for efficient resource utilization and quick retrieval
• Create and maintain comprehensive documentation for data pipelines, ETL processes, and database schemas
• Monitor data pipelines and systems for performance and reliability, troubleshooting and resolving issues as they arise
• Stay up to date with emerging technologies and best practices in data engineering, evaluating and recommending new tools as appropriate

• Requirements: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field (Engineering or Math preferred)
• 5+ years of experience with SQL, Python, .NET, SSIS, and SSAS
• 2+ years of experience with Azure cloud services, particularly SQL Server, ADF, Azure Databricks, ADLS, Key Vault, Azure Functions, and Logic Apps, with an emphasis on Databricks
• 2+ years of experience using Git and deploying code using a CI/CD approach
• Strong analytical and problem-solving skills
• Excellent communication and interpersonal skills
• Ability to work independently and as part of a team
• Attention to detail and a commitment to quality