Data Architect

We are seeking a Databricks Architect with deep expertise in Azure Data Platform, Data Modelling, and Medallion (Lakehouse) Architecture . The ideal candidate will design and implement scalable, high-performance data solutions using Databricks, Delta Lake, and modern data engineering practices , enabling enterprise-wide analytics and business intelligence.

Location: Abu Dhabi

Experience: 10–15+ Years

Email : vijay.kajla@xebia.com

Key Responsibilities

Medallion Architecture & Lakehouse Design

• Design and implement Medallion Architecture (Bronze, Silver, Gold layers) on Azure Databricks
• Define data flow standards across ingestion, cleansing, transformation, and consumption layers
• Ensure data quality, lineage, and governance across all layers
• Optimize storage and compute using Delta Lake features (ACID transactions, time travel, schema evolution)

Architecture & Strategy

• Architect end-to-end Azure data platforms using:
• Azure Databricks
• ADLS Gen2
• Azure Data Factory
• Azure Synapse
• Define data platform standards, best practices, and reusable frameworks
• Lead data modernization initiatives (on-prem to cloud, legacy to lakehouse)

Data Engineering Leadership

• Build scalable pipelines using PySpark, Spark SQL, and Databricks workflows
• Implement batch + real-time ingestion patterns
• Drive performance tuning and cost optimization
• Mentor teams on Spark optimization, partitioning, and job orchestration

Data Modelling

• Design enterprise-grade data models :
• Dimensional (Star/Snowflake)
• Normalized and Data Vault (nice to have)
• Align data models with Gold layer consumption use-cases (BI/Reporting)
• Translate business needs into scalable analytical datasets

Integration & Consumption

• Build ingestion pipelines using:
• ADF, Event Hub, Kafka, APIs
• Enable reporting via:
• Power BI, Tableau, or similar BI tools
• Ensure seamless consumption from Gold datasets

Stakeholder Engagement

• Act as a data architecture SME in client discussions
• Drive solutioning, architecture reviews, and governance forums
• Collaborate with business teams for data-driven decision making

Required Skills

Core

• Strong hands-on experience with Azure Databricks & Delta Lake
• Proven expertise in Medallion Architecture implementation
• Advanced PySpark and Spark SQL
• Experience with ADF, ADLS Gen2, Synapse

Data Modelling

• Strong experience in:
• Dimensional modelling (must-have)
• Data modelling aligned to Gold layer analytics
• Understanding of data quality and governance frameworks

Nice-to-Have

• Real-time processing ( Kafka / Spark Streaming )
• Azure Purview or data governance tools
• ML pipelines on Databricks
• Domain: Airlines / BFSI / Retail

Leadership Skills

• Strong architectural thinking
• Client-facing experience
• Ability to lead and scale engineering teams
• Excellent communication and stakeholder management

We are an Equal Opportunity Employer committed to fostering a diverse, inclusive, and respectful workplace. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. We believe diversity drives innovation and success, and we welcome individuals from all backgrounds to join our team.