AWS Data Engineer (Standard)

Primary Responsibilities

• Develop and maintain data processing scripts using Python (Pandas, NumPy)

• Build and manage ETL workflows using AWS Glue

• Implement event-driven and batch processing using AWS Lambda

• Ingest data from SFTP locations, email deliveries, and flat files (CSV)

• Handle complex CSV structures, including:

  • Non-standard or multi-row headers

  • Dynamic column positioning

  • Reading data using column index-based logic

• Integrate and consume REST APIs for data ingestion

• Perform data validation, cleansing, transformation, and enrichment

• Collaborate with data engineering, analytics, and business teams

• Support production deployments, monitoring, and issue resolution

?? Required Skills

• Strong proficiency in Python, especially Pandas and NumPy

• Hands-on experience with AWS Glue and AWS Lambda

• Experience working with CSV files having complex or irregular headers

• Strong understanding of ETL concepts and data transformation

• Experience in ingesting data from SFTP and file-based sources

• Basic to intermediate knowledge of SQL and relational databases

• Good problem-solving, debugging, and data analysis skills

?? Good to Have

• Exposure to cloud data architectures (AWS preferred)

• Experience with email-based data ingestion or automated file handling

• Understanding of data security, access controls, and PII handling

• Familiarity with Agile delivery environments