Senior Data Engineer - PySpark & Python (Banking Project)

Senior Data Engineer - PySpark & Python

About the Role

We are hiring a Senior Data Engineer - PySpark & Python for one of our Dubai-based banking clients. The role involves designing, building, and optimizing high-performance, scalable data processing and integration solutions within a distributed banking ecosystem. Candidates based in India or Dubai are encouraged to apply.

Key Responsibilities

• Develop and maintain high-quality, scalable Python applications using Python 3.6+.
• Build and optimize large-scale data processing pipelines using PySpark for enterprise banking systems.
• Work with databases such as PostgreSQL, MongoDB, and other relational and NoSQL technologies.
• Design and develop RESTful APIs using frameworks like Flask and Django.
• Integrate and deploy AI/ML models into production-grade data and application workflows.
• Implement solutions involving multiprocessing, concurrency, and load balancing for high-throughput systems.
• Apply strong understanding of modern Python frameworks, asynchronous programming, and concurrency patterns.
• Work closely with DevOps teams to implement and maintain CI/CD pipelines.
• Actively troubleshoot, debug, and resolve issues within distributed services environments.
• Ensure resiliency, scalability, observability, and monitoring of production systems.
• Follow best practices for secure coding, unit testing, documentation, and code reviews.

Required Skills & Qualifications

• Strong hands on coding experience with Python (3.6 or higher).
• Solid experience with PySpark and distributed data processing frameworks.
• Experience working with PostgreSQL, MongoDB, or similar databases.
• Hands on experience developing REST APIs using Flask and/or Django.
• Good exposure to AI/ML model integration in real world applications.
• Strong understanding of multiprocessing, concurrency, and load balancing concepts.
• Experience working in distributed systems and production environments.
• Familiarity with CI/CD principles, automation, and DevOps practices.
• Experience with monitoring, logging, and troubleshooting large scale systems.

Good to Have

• Prior experience in banking or BFSI projects.
• Exposure to cloud platforms (AWS, Azure, GCP).
• Experience with Docker, Kubernetes, or containerized deployments.
• Knowledge of data security, governance, and compliance standards.

Seniority level

Mid Senior level

Employment type

Full time

Job function

Information Technology

Industries

Banking

Referrals increase your chances of interviewing at The Glove by 2x