Senior Consultant

Skill Set:

AWS, Snowflake, Kafka, Airflow, GitHub, PySpark

Key Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines
• Ingest data from various sources (APIs, databases, files, etc.)
• Implement both real-time and batch processing solutions based on use case requirements
• Ensure data quality through validation and cleansing processes
• Collaborate with Product Managers and Business Stakeholders to gather and understand data requirements
• Translate business needs into technical specifications
• Ensure data security, access control, and compliance with relevant policies
• Maintain documentation and follow best practices for data engineering

Ideal candidate profile: 4-8 years of hands-on experience with Snowflake, Airflow and Pyspark in any cloud platform (AWS/Azure/GCP). The candidate should be serving notice with LWD in the next 3 weeks.