Data Engineer

Job Description

Must Have Technical/Functional Skills

We are seeking a skilled Data Engineer to design, build, and maintain high-quality, scalable data pipelines with a strong focus on Silver and Gold layer transformations, data quality automation, and CI/CD-driven deployments. The ideal candidate will have hands-on experience with Python, PySpark, Git-based CI/CD pipelines, ARM templates, and data quality libraries such as Great Expectations.

Roles & Responsibilities

• Design and implement Silver and Gold layer data models following medallion architecture best practices.
• Perform data cleansing, standardization, enrichment, and aggregation to support analytics and reporting.
• Build optimized PySpark-based transformations for large-scale data processing.
• Ensure data consistency, performance, and scalability across datasets.
• Build and maintain CI/CD pipelines using Git-based workflows (Azure DevOps / GitHub).
• Use ARM templates (or IaC equivalents) for automated infrastructure provisioning.
• Enable automated deployment of data pipelines, notebooks, and configurations.
• Follow DevOps best practices for version control, branching, and release management.
• Develop reusable Python-based automation scripts for pipeline orchestration, validation, and monitoring.
• Create modular, maintainable, and testable Python code.
• Support automation of metadata, logging, alerting, and operational tasks.
• Implement data quality libraries such as Great Expectations.
• Define and automate data quality rules (completeness, accuracy, freshness, consistency).
• Integrate data quality checks into data pipelines and CI/CD workflows.
• Monitor, log, and troubleshoot data quality issues proactively.
• Work closely with data architects, analysts, QA, and business stakeholders.
• Translate business and analytical requirements into robust data engineering solutions.
• Participate in Agile ceremonies and support sprint-based delivery.
• Provide production support, performance tuning, and troubleshooting
  
  

Salary Range $120,000-$130,000 year

TCS Employee Benefits Summary

Discretionary Annual Incentive.

Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.

Family Support: Maternal & Parental Leaves.

Insurance Options: Auto & Home Insurance, Identity Theft Protection.

Convenience & Professional Growth: Commuter Benefits & Certification & amp; Training Reimbursement.

Time Off: Vacation, Time Off, Sick Leave & Holidays.

Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Qualifications: BACHELOR OF COMPUTER SCIENCE