Senior Data Engineer

Job Title: Senior Data Engineer (Databricks Specialist) Role Overview We are seeking a seasoned Senior Data Engineer to lead the design, implementation, and optimization of our data platform using Databricks. You will be responsible for building scalable data pipelines while ensuring the highest standards of Data Governance and security. The ideal candidate isn't just a coder; they are an architect of data ecosystems who understands how to manage data at scale while maintaining strict Role-Based Access Control (RBAC). Key Responsibilities Pipeline Development: Design and build complex, high-volume ETL/ELT pipelines using Spark (PySpark/Scala) and Databricks Delta Live Tables (DLT). Lakehouse Architecture: Transition and maintain data structures within the Medallion Architecture (Bronze/Silver/Gold) to ensure data quality and reliability. Security & Compliance: Implement and manage Unity Catalog for fine-grained data governance, lineage, and discovery. Access Management: Define and enforce Role-Based Access Control (RBAC) and Attribute-Based Access Control (ABAC) to secure sensitive datasets. Performance Tuning: Optimize Spark jobs, cluster configurations, and SQL warehouses to reduce latency and cloud costs. Mentorship: Act as a technical lead, conducting code reviews and guiding junior engineers on Databricks best practices. Required Qualifications Category Requirement Total Experience 7–8+ years in Data Engineering or Big Data environments. Databricks Core 3–5 years of hands-on experience specifically with the Databricks platform. Technical Stack Proficiency in Python/PySpark, SQL, and Spark optimization techniques. Data Governance Expert knowledge of Unity Catalog, data masking, and row/column level security. Cloud Platforms Experience deploying Databricks on Azure, AWS, or GCP (please specify your preference). DevOps/MLOps Familiarity with Databricks Repos, Workflows, and CI/CD integration. Preferred Skills Databricks Certifications: (e.g., Data Engineer Professional or Cloud Provider specific certifications). Streaming: Experience with Structured Streaming or Event Hubs/Kafka. IaC: Knowledge of Terraform for managing Databricks workspaces and clusters.

Job Type: Full-time

Pay: Up to ₹3,500,000.00 per year

Benefits:

• Health insurance
• Provident Fund

Education:

• Bachelor's (Required)

Experience:

• Data engineer : 7 years (Required)
• Databricks: 3 years (Required)
• Python/PySpark: 3 years (Required)
• Unity catalog: 3 years (Required)
• Azure, AWS or GCP: 3 years (Required)

Language:

• Hindi (Required)

Work Location: In person