Senior Data Engineer

About the Role

We are seeking a highly skilled and experienced Senior Data Engineer to join our growing Data & Analytics team. This role is central to building, maintaining, and evolving our enterprise data platform using Microsoft Fabric and Azure Synapse Analytics. The ideal candidate will have deep expertise in data warehouse architecture, master data management (MDM), and modern cloud data ecosystems.

You will be responsible for integrating data from multiple enterprise sources including HubSpot (via API) and NetSuite (via ODBC), designing Lakehouse architectures, and developing Power BI dashboards that empower business decision-making.

Key Responsibilities

Microsoft Fabric & Lakehouse

▸ Design, build, and manage end-to-end data pipelines using Microsoft Fabric (Lakehouse, Pipelines, Datawarehouse).

▸ Implement API-based data ingestion from HubSpot into Fabric using REST connectors and custom pipeline logic.

▸ Configure and manage ODBC driver-based data imports from NetSuite into the Fabric ecosystem.

▸ Develop and maintain Notebook scripts (PySpark / Python) for Master Data Management (MDM) processes within Fabric.

▸ Maintain data quality, consistency, and governance standards across the Lakehouse layers (Bronze, Silver, Gold).

Azure Synapse Analytics

▸ Architect and implement data solutions using Azure Synapse Analytics in parallel with or migrating to Microsoft Fabric.

▸ Develop Synapse Pipelines, Spark Pools, and SQL Pools for large-scale data processing and analytics workloads.

▸ Integrate Synapse with other Azure services such as Azure Data Lake Storage, Azure Key Vault, and Azure Monitor.

Data Warehouse & MDM

▸ Apply industry-standard data warehouse design patterns including Star Schema, Snowflake Schema, and Data Vault 2.0.

▸ Lead Master Data Management initiatives — designing MDM workflows, golden record logic, and deduplication routines using Fabric notebooks.

▸ Ensure referential integrity, SCD (Slowly Changing Dimensions), and historical tracking across data warehouse layers.

SQL & Database Development

▸ Write complex Advanced SQL and Dynamic SQL queries for data transformation, reporting, and analytics.

▸ Develop Stored Procedures and Functions in both Microsoft SQL Server and PostgreSQL.

▸ Optimize query performance, indexing strategies, and execution plans for high-volume datasets.

Power BI

▸ Design and develop interactive, enterprise-grade Power BI Dashboards and Reports.

▸ Build and maintain robust semantic models (datasets), DAX measures, and calculated columns.

▸ Implement Row-Level Security (RLS) and ensure data governance in Power BI workspaces.

▸ Collaborate with business stakeholders to translate analytical requirements into visual insights.

Azure Analysis Services & Semantic Modelling

▸ Design, develop, and maintain enterprise-grade Tabular models using Azure Analysis Services (AAS).

▸ Build and manage Semantic Models with complex DAX logic, hierarchies, and perspectives aligned to business reporting needs.

▸ Define and implement measures, KPIs, calculated tables, and role-based security within Tabular models.

▸ Optimise model performance through partitioning strategies, aggregations, and incremental refresh policies.

▸ Migrate and modernise legacy SSAS Multidimensional (OLAP cube) models to modern Tabular / Fabric Semantic Model architecture.

▸ Integrate AAS models with Power BI, Excel, and third-party BI tools via XMLA endpoints.

Domain Expertise

▸ Apply data engineering and analytics capabilities across the following specialist business domains:

– Marketing & Customer Analytics — customer segmentation, campaign performance, lead funnel analysis, CRM data modelling (HubSpot), churn prediction, and customer lifetime value (CLV) metrics.

– Affordable Housing Projects — tenancy and property data management, housing allocation and waitlist reporting, compliance and regulatory dashboards, social impact KPI tracking.

– High Court & Legal Case Management — case lifecycle data modelling, court scheduling and outcomes reporting, legal document metadata management, SLA and compliance tracking.

– Probation & Justice Services — offender management data pipelines, risk assessment data integration, caseload reporting, and compliance with data sensitivity and access controls under justice sector regulations.

▸ Understand data sensitivity, privacy obligations (GDPR / sector-specific regulations), and access governance requirements specific to legal and social service domains.

Cross-Platform & Collaboration

▸ Work across multiple RDBMS and cloud data platforms including Databricks, Google BigQuery, and Snowflake as needed.

▸ Collaborate with data analysts, BI developers, and business stakeholders to deliver data solutions.

▸ Document architecture decisions, data models, pipeline logic, and MDM workflows.

Required Skills & Qualifications

Core Experience

▸ Minimum 5 years of hands-on experience in Data Engineering roles.

▸ Proven experience with Microsoft Fabric — Lakehouse, Pipelines, Datawarehouse, and Notebooks.

▸ Strong experience with Azure Synapse Analytics — Pipelines, Spark Pools, SQL Pools.

▸ Demonstrated expertise in Data Warehouse design and implementation.

▸ Experience with Master Data Management (MDM) using scripting/notebook-based approaches.

Integration & Connectivity

▸ Experience with API-based data integrations (REST/JSON), specifically with HubSpot or similar CRM platforms.

▸ Hands-on experience with ODBC driver-based integrations, specifically with NetSuite or similar ERP systems.

▸ Familiarity with authentication mechanisms: OAuth 2.0, API keys, and service accounts.

SQL & Database

▸ Expert-level SQL skills: Advanced SQL, Dynamic SQL, Stored Procedures, Functions, Views, and CTEs.

▸ Proficiency in Microsoft SQL Server (T-SQL) and PostgreSQL.

▸ Understanding of database performance tuning — indexing, query plans, partitioning.

Power BI

▸ Solid experience with Power BI Desktop and Power BI Service.

▸ Proficiency in DAX (Data Analysis Expressions) for complex measures and KPIs.

▸ Experience building enterprise semantic models and managing Power BI workspaces.

Azure Analysis Services & Semantic Models

▸ Hands-on experience designing and developing Tabular models in Azure Analysis Services (AAS) or SSAS Tabular.

▸ Proficiency with DAX for complex business logic: time intelligence, semi-additive measures, row context and filter context manipulation.

▸ Experience with Semantic Model design best practices — star schema, measure tables, calculation groups, and field parameters.

▸ Ability to manage AAS deployments, processing schedules, and role-based access via XMLA or REST API.

▸ Familiarity with Tabular Model Scripting Language (TMSL) and Tabular Editor for model development and automation.

Domain Knowledge

▸ Demonstrated experience working within one or more of the following specialist domains is highly valued:

– Marketing & Customer Analytics — working knowledge of CRM data structures (HubSpot), campaign attribution models, customer journey mapping, segmentation, churn prediction, and marketing funnel KPIs.

– Affordable Housing — understanding of tenancy management systems, social housing allocation, waitlist data modelling, regulatory compliance reporting, and community impact measurement.

– High Court & Legal Case Management — familiarity with case lifecycle data, court proceedings data models, legal SLA metrics, document metadata management, and sensitive legal data handling requirements.

– Probation & Criminal Justice Services — knowledge of offender case data pipelines, rehabilitation programme tracking, risk scoring systems, caseload analytics, and strict data access governance within justice sector frameworks.

▸ Ability to rapidly acquire domain knowledge in new sectors and translate complex business requirements into accurate data models and analytics solutions.

▸ Awareness of data sensitivity, privacy regulations (GDPR, sector-specific legislation), and access governance obligations within legal, justice, and social services domains.

Cloud & Other Platforms

▸ Awareness and working knowledge of other cloud data platforms such as:

– Databricks — Delta Lake, Spark, Unity Catalog

– Google BigQuery — SQL, partitioned tables, streaming

– Snowflake — Warehouses, stages, data sharing

Nice to Have

▸ Experience with dbt (data build tool) for transformation layer management.

▸ Familiarity with Medallion Architecture (Bronze / Silver / Gold layers).

▸ Knowledge of data governance frameworks and metadata management tools.

▸ Experience with CI/CD pipelines for data engineering (Azure DevOps, Git).

▸ Exposure to real-time streaming technologies (Azure Event Hubs, Kafka).

▸ Power BI certification (PL-300), Microsoft Fabric certification (DP-600), or Azure Analysis Services / SSAS certification is a plus.

Education

▸ Bachelor's degree or higher in Computer Science, Information Technology, Data Science, or a related field.

▸ Equivalent professional experience will be considered in lieu of formal education.

What We Offer

▸ Opportunity to work on a high-impact, modern data platform using cutting-edge Microsoft technologies.

▸ Collaborative environment with skilled data, analytics, and engineering teams.

▸ Competitive compensation package and career growth opportunities.

▸ Flexible work arrangements (on-site / hybrid / remote based on location).

▸ Professional development support including certifications and training programs.

Pay: $133,355.58 - $160,600.27 per year

Work Location: In person