Lead II - Software Engineering - ETL, AWS

7 - 9 Years
1 Opening
Bangalore, Hyderabad

Role Proficiency:

• AWS ETL Developer: Must have skills: experience in the development of Data Lakes with Data Ingestion from disparate data sources, including relational databases, flat files, APIs, and streaming data. Proficiency in programming languages like Python, PySpark, Spark, to ensure efficient data processing. preferably Python. Architect and implement robust ETL pipelines using AWS Glue, defining data extraction methods, transformation logic, and data loading procedures across different data sources Solid understanding of NoSQL databases and their architecture, particularly in the context of Amazon Document DB. Strong experience with Amazon Web Services (AWS), specifically Amazon Document DB. Expertise in core AWS Services including AWS IAM, VPC, EC2, S3, RDS, Lambda, CloudWatch, CloudFormation, CloudTrail. Experience in designing and developing Event-driven Data Pipelines in AWS Cloud using Glue. Ability to design, orchestrate and schedule jobs using Airflow Experience in the development of Event-Driven Distributed Systems in the Cloud using Serverless Architecture. Experience in development of CI/CD pipelines (GitHub Actions, Jenkins). Experience with AWS Cloud Data Ingestion Patterns and Practices, using S3 as the storage backbone. Experience in using IaC tools like Terraform. Experience in migrating data from on-prem data sources to the AWS Cloud using AWS Database Migration Service – DMS

Etl,Aws Services,Data Lake,Python