Director, Data Engineering

Overview

We are a $5B national retailer headquartered in the Atlanta area, and we are seeking a seasoned and strategic Director of Data Engineering to lead our data engineering organization. This is a high-impact leadership role responsible for delivering reliable, high-quality data products that power the business — while simultaneously charting the course for a transformational migration from our current SQL Server-based data platform to Azure Databricks.

The ideal candidate is a hands-on technical leader who can operate at the intersection of engineering excellence, business partnership, and executive-level influence. You bring deep data engineering expertise, a track record of leading mid-to-large teams, and the ability to build and sell a compelling strategic vision to both technical and non-technical stakeholders.

Responsibilities

Team Leadership & Talent Development

• Lead, mentor, and grow a team of 8 data engineers, fostering a culture of accountability, technical excellence, continuous learning, and collaboration.

• Establish and model agile delivery practices, including sprint planning, backlog grooming, retrospectives, and iterative delivery.

• Build career development paths for team members and proactively recruit to scale the team as the platform evolves.

Business Partnership & Demand Management

• Serve as the primary IT interface for business stakeholders seeking data engineering capabilities, acting as both a trusted advisor and an accountable delivery partner.

• Own the intake, prioritization, and roadmap planning process for data engineering work requests, balancing business urgency with technical capacity.

• Partner with business leaders to identify and develop new data capabilities that drive competitive advantage — including reporting, analytics, and data product development.

Platform Delivery & Operations

• Oversee day-to-day delivery and operational support of the current SQL Server-based data platform, ensuring high availability, performance, and data integrity.

• Establish and enforce engineering standards for data quality, testing, CI/CD, documentation, and code review.

• Define and monitor SLAs/SLOs for data platform services and hold the team accountable to operational excellence.

Data Security & Governance

• Embed data security and governance principles into all aspects of platform design and delivery.

• Partner with Information Security, Compliance, and Architecture teams to ensure data assets are properly protected, cataloged, and governed.

• Champion data stewardship practices across both engineering and business teams.

Infrastructure Partnership

• Collaborate closely with infrastructure and cloud engineering teams to ensure underlying compute, storage, and networking resources are performant, well-maintained, and cost-optimized.

• Serve as a key stakeholder in infrastructure capacity planning and cloud cost management conversations.

Strategic Platform Modernization — SQL Server to Azure Databricks

• Define and own the enterprise vision and multi-year roadmap for migrating from SQL Server to Azure Databricks, including the Lakehouse architecture, Delta Lake design, and data pipeline modernization strategy.

• Develop a compelling, business-aligned business case — quantifying value, risk, and ROI — and present it to IT and company leadership to secure executive sponsorship and funding.

• Build and maintain alignment across business, IT, Finance, and Infrastructure stakeholders throughout the migration lifecycle.

• Lead phased execution of the migration, ensuring continuity of existing operations while progressively transitioning workloads to the new platform.

Minimum Eligibility Requirements:

• Bachelor’s degree in Business Administration, Computer Science, Information Systems or equivalent combination of education and experience.

• A solid understanding of key Business Intelligence trends

• 5-10+ years of IT Management experience

• Ability to lead multiple projects in a large complex organization.

• Ability to work closely in a collaborative team environment, and lead teams and project deliverables.

• Excellent oral and written communication skills, including comfort with communication with senior leadership.

• Excels working in an innovative, entrepreneurial, and growing organization.

Preferred Requirements

Experience

• 10+ years of progressive experience in data engineering, with at least 4–5 years in a senior leadership role.

• Demonstrated experience leading teams of 10 to 50 engineers, including performance management, hiring, and organizational development.

• Proven track record of delivering large-scale, complex data platform initiatives on time and within budget.

• Hands-on expertise with SQL Server and deep experience in cloud-based data platform technologies, particularly Azure Databricks, Azure Data Factory, Azure Data Lake Storage (ADLS), and Delta Lake.

• Experience defining and executing cloud migration strategies — ideally SQL Server to Azure — including business case development and executive stakeholder management.

• Strong grounding in agile delivery methodologies and experience instilling these practices within data engineering teams.

• (Strongly Preferred) Retail industry experience, with an understanding of retail data domains such as merchandising, supply chain, store operations, loyalty, and omnichannel analytics.

Technical Skills

• Expert-level proficiency in SQL, Python, and/or Scala for data engineering workloads.

• Strong knowledge of Azure Databricks, including Unity Catalog, Delta Live Tables, Workflows, and Spark optimization.

• Familiarity with modern data stack tooling: Azure Data Factory, dbt, Apache Airflow, or equivalent orchestration tools.

• Experience with data modeling for both OLTP and analytical (dimensional/lakehouse) patterns.

• Working knowledge of data governance frameworks, data cataloging (e.g., Microsoft Purview), and data quality tooling.

• Familiarity with DevOps/DataOps practices including CI/CD pipelines, version control (Git), and infrastructure-as-code.

Leadership & Business Skills

• Exceptional communication and executive presence — able to translate complex technical concepts into clear business value for non-technical audiences.

• Strong strategic thinking with the ability to develop multi-year roadmaps and build the business cases to fund them.

• Demonstrated ability to build cross-functional alignment and navigate organizational dynamics to drive large-scale change.

• Financially literate, with experience managing team budgets and articulating technology investment trade-offs.

Why Join Us

• Lead a consequential, once-in-a-generation platform transformation at a well-capitalized, growing retailer.

• High-visibility role with direct access to senior IT and business leadership.

• Collaborative, entrepreneurial technology culture that values innovation and continuous improvement.

• Competitive compensation, bonus, and benefits package.

• Headquarters in the Atlanta metro area with flexible hybrid work arrangement.