Senior Data Engineer

Senior Data Engineer, Azure Databricks

Job Summary: We are seeking a highly skilled Senior Data Engineer with 3-5 years of professional experience, specializing in the Azure Databricks platform. The ideal candidate will have a strong track record of designing, building, and optimizing scalable data pipelines. You will play a crucial role in enhancing our data infrastructure to support critical business intelligence and advanced analytics.

Key Responsibilities:

• Design, develop, and maintain robust data pipelines on Azure Databricks using PySpark.
• Implement and manage ETL/ELT processes for both batch and streaming data.
• Work with data scientists and analysts to understand their data requirements and deliver reliable data solutions.
• Optimize Databricks clusters and jobs for performance and cost efficiency.
• Ensure data quality, integrity, and security across all data pipelines.

Required Skills & Qualifications:

• Bachelor's degree in Computer Science, Engineering, or a related field.
• 3-5 years of demonstrable experience as a Data Engineer with a strong focus on Azure Databricks.
• Proficient in PySpark and the Spark ecosystem.
• Experience with Delta Lake for building data lakes and managing ACID transactions is a must.
• Strong command of SQL and solid understanding of data warehousing concepts.
• Hands-on experience with other Azure data services, including Azure Data Lake Storage (ADLS) and Azure Synapse Analytics.
• Excellent problem-solving and communication skills.

Job Types: Full-time, Permanent

Pay: ₹1,000,000.00 - ₹1,200,000.00 per year

Benefits:

• Paid sick time
• Paid time off

Work Location: In person