Lead Data Engineer

Senior / Lead / Data Engineer – Comprehensive Job Description

About the Role

We are seeking a highly skilled and experienced Senior/Lead/Staff Data Engineer to

design, build, and maintain scalable, reliable, and high-performance data systems. This

role requires deep technical expertise, strong ownership of data platforms, and the ability

to lead cross-functional initiatives that power analytics, machine learning, business

intelligence, and mission-critical applications.

You will play a key role in shaping data architecture, ensuring data quality and governance,

and enabling teams across the organization to make data-driven decisions with

confidence.

Key Responsibilities

Data Architecture & Modeling

• Design and implement scalable data models, including:

o Star schema (fact and dimension tables)

o Conceptual, logical, and physical data models

• Develop enterprise data warehouse solutions ensuring:

o Flexibility, scalability, and performance optimization

• Define and enforce data modeling standards and best practices

Data Engineering & Pipeline Development

• Build and maintain end-to-end data pipelines:

o Data ingestion → staging → transformation → serving layers

• Develop robust ETL/ELT processes using modern tools and frameworks
• Implement:

o Typed staging layers

o Data validation pipelines

o Safe backfill and replay mechanisms

• Optimize SQL queries, indexing strategies, and database performanceData Platform Reliability & Observability
• Own data pipeline reliability and integrity
• Build observability systems, including:

o Monitoring dashboards

o Alerting frameworks (actionable, low-noise)

o Data freshness tracking

• Implement:

o Schema contracts and enforcement

o Data quality checks and validation rules

o Failure detection in CI pipelines (not post-deployment)

• Lead incident management and root cause analysis

Data Governance & Quality

• Establish data governance frameworks:

o Standards, policies, and best practices

• Enforce:

o Data integrity

o Data lineage and traceability

o Access controls and compliance

• Separate and maintain clear boundaries between:

o Core governed datasets

o Downstream analytics/BI layers

Cloud & Big Data Integration

• Integrate systems with:

o Data lakes and large-scale data platforms

• Work with modern cloud ecosystems (AWS preferred):

oo Data warehouse platforms (e.g., Redshift, Snowflake)

• Design cost-efficient and scalable data solutions

Analytics Engineering & Data Products

• Design and deliver data products used by:

o Analysts

o Data scientists

o Product teams

• Build reusable frameworks for:

o Data transformation

o Orchestration

o Feature engineering (ML readiness)

• Support advanced analytics and reporting use cases

CI/CD & DevOps for Data

• Implement CI/CD pipelines for data workflows
• Automate:

o Testing

o Deployment

o Validation

• Use version control systems (Git/GitHub)
• Ensure:

o High availability

o Backup and recovery strategies

Programming & Tooling

• Develop solutions using:

oo Python (automation, validation, tooling)

• Work with tools such as:

o dbt (deep experience required)

o DBeaver, Visual Studio

o Orchestration tools (e.g., Dagster or similar)

Leadership & Collaboration

• Lead cross-functional technical initiatives
• Mentor and guide junior to mid-level data engineers
• Conduct:

o Code reviews

o Architecture discussions

o Knowledge sharing sessions

• Collaborate with:

o Engineering teams

o Product managers

o Business stakeholders

• Translate business requirements into scalable data solutions

Strategic Impact

• Drive platform evolution and innovation
• Identify opportunities for:

o Performance improvements

o Cost optimization (FinOps mindset)

o Automation and scalability

• Influence organizational data strategy and roadmapRequired Qualifications
• Bachelor’s degree in computer science, Engineering, IT, or related field
• 4–5+ years of experience in data engineering or analytics engineering
• Strong expertise in:

o Data modeling (star schema, dimensional modeling)

o SQL and performance tuning

o ETL/ELT pipeline development

• Hands-on experience with:

o Data warehousing and data lakes

o Big data integrations

• Experience with:

o Cloud platforms (AWS preferred)

o Data quality, validation, and governance frameworks

• Strong understanding of:

o System design

o Scalable architecture

• Excellent communication and stakeholder management skills
• Ability to work in agile, fast-paced environments
• Eligibility to obtain security clearance (if required)

Preferred Qualifications

• Experience with:

o AWS services (Glue, Lambda, etc.)

o Snowflake, Redshift

o dbt (production-level deployments)

• Knowledge of:

o Machine learning data pipelines

o Certifications:

o SQL Server, CDMP, MCSA, or similar

• Experience in:

o Healthcare, defense, or regulated industries

• Familiarity with:

o CI/CD for data systems

o Data governance and FinOps practices

What Makes a Strong Candidate

• Thinks of systems, not just pipelines
• Focuses on root-cause solutions, not temporary fixes
• Balances:

o Business value

o Scalability

o Maintainability

• Strong ownership mindset with ability to lead initiatives end-to-end
• Passion for building resilient, high-quality data platforms

Job Type: Full-time

Pay: Rs250,000.00 - Rs350,000.00 per month

Work Location: In person