Lead Data Engineer - AWS

Valleysoft | Center of Excellence is a regional IT services provider based in Egypt, serving clients globally since 2006. The company collaborates with global partners like Amazon Web Services (AWS) to offer comprehensive solutions that address diverse business and technical challenges. Valleysoft's commitment to excellence ensures that clients receive high-quality, innovative, and cost-effective services, leveraging the power of cloud technologies.

Valleysoft is seeking a highly-skilled Lead Data Engineer specializing in AWS to join our team. In this role, you will lead the design and implementation of cutting-edge data solutions, unlocking the potential of big data and machine learning for our clients. You will drive data strategy, manage data architecture, and oversee a talented team of data engineers. If you are passionate about cloud-based data engineering and want to make an impact in a fast-paced environment, we want to hear from you!

• • Lead design and implementation of batch and real-time data pipelines on AWS
  • Architect and implement ETL/ELT solutions using AWS Glue, Spark on EMR, and other AWS-native services
  • Lead workflow orchestration using Apache Airflow (MWAA)
  • Architect serverless data processing solutions with AWS Lambda
  • Design enterprise-grade data warehousing solutions using Amazon Redshift
  • Develop and enforce data storage strategies using Amazon S3 and DynamoDB
  • Implement and govern data cataloging, governance, and access control using AWS DataZone
  • Establish monitoring, logging, and observability using Amazon CloudWatch
  • Drive performance optimization, scalability, and cost efficiency
  • Troubleshoot complex data processing and infrastructure issues
  • Collaborate with cross-functional teams including Analytics, BI, Data Science, and Business teams
  • Provide technical leadership, conduct code reviews, and mentor junior engineers
  • Ensure adherence to security, compliance, and data governance standards
  • Participate in architecture discussions and contribute to long-term data strategy

Required Skills & Experience

• • 10–12 years of experience in Data Engineering or related roles
  • 5+ years of hands-on experience designing enterprise-grade AWS solutions
  • Expertise with: AWS Glue, Amazon EMR (Spark), AWS Lambda, Apache Airflow (MWAA), Amazon EC2, Amazon CloudWatch, Amazon Redshift, Amazon DynamoDB, AWS DataZone, Amazon S3
  • Strong programming skills in Python
  • Advanced SQL skills with performance tuning expertise
  • Strong understanding of data warehousing, data modeling (Star/Snowflake), ETL/ELT best practices
  • Experience designing scalable, distributed data processing systems
  • Proven experience in leading teams, mentoring engineers, and making architectural decisions
  • Experience working in Agile/Scrum delivery environments

Nice to Have.

• AWS Professional Certification (Solutions Architect / Data Analytics Specialty).
• Experience with real-time streaming (Kinesis, Kafka).
• Exposure to metadata management, data lineage, and governance tools.
• Expertise in cost optimization strategies in AWS.
• Experience working with large-scale enterprise data platforms.

• Private Health Insurance
• Training & Development