Pyspark

Skill: Pyspark

Experience: 6 to 13 years

Location: Bhubaneswar

Job description :

• Develop and maintain scalable data pipelines using Python and PySpark.
• Collaborate with data engineers and data scientists to understand and fulfill data processing needs.
• Optimize and troubleshoot existing PySpark applications for performance improvements.
• Write clean, efficient, and well-documented code following best practices.
• Participate in design and code reviews.
• Develop and implement ETL processes to extract, transform, and load data.
• Ensure data integrity and quality throughout the data lifecycle.
• Stay current with the latest industry trends and technologies in big data and cloud computing.

Qualifications

• Proven experience as a Python Developer with expertise in PySpark.
• Strong understanding of big data technologies and frameworks.
• Experience with distributed computing and parallel processing.
• Proficiency in SQL and experience with database systems.
• Solid understanding of data engineering concepts and best practices.
• Ability to work in a fast-paced environment and handle multiple projects simultaneously.
• Excellent problem-solving and debugging skills.
• Strong communication and collaboration abilities.