Data Engineer- AWS

Title: Data Engineer - AWS

Purpose of the Position:

You will be a critical member of the InfoCepts Cloud Data Architect Team. This position requires a sound knowledge on Big Data using Cloud technologies, EMR, Athena, Spark-Scala, PySpark, S3, AWS Lambda etc. Strong foundation on database concepts and SQL is also required.

Key Result Areas and Activities:

• Study existing technology landscape and understand current data integration framework and do impact assessment for the requirements.

• Develop spark jobs using Scala for new project requirements.

• Enhance existing spark jobs for any ongoing product enhancement.

• Performance tuning of spark jobs, stress testing etc.

• Create new data pipelines for developed/enhanced spark jobs using AWS Lambda or Apache Airflow

• Responsible for database design process, logical design, physical design, star schema, snowflake schema etc.

• Analyze data processing, integration, modelling and reporting requirements & Define data loading strategies considering volume, data types, frequency and analytics specifications.

• Ensure optimal balance between cost and performance.

• Project documentation, Adheres to Quality guidelines & Schedules.

• Works hand in hand with PM for successful delivery of project and provide Estimation, scoping, scheduling assistance.

• Manage build phase and quality assure code to ensure fulfilling requirements and adhering to Cloud architecture. Resolve difficult design and develop issues.

  
  

5 to 7 years

India