Lead Engineer (AI/ML - MDS)

Overview:

Join the Prodapt team in building a unified, enterprise-scale solution for model, data, and service management across the AI/ML lifecycle. You will help design, develop, and optimize robust workflows that empower data scientists, engineers, and business teams to efficiently prepare datasets, manage models, and orchestrate services at scale.

Responsibilities:

• Design, develop, and maintain scalable data integration and ETL pipelines using Informatica, PySpark, and Apache Kafka.
• Build and optimize model development workflows leveraging Python, TensorFlow, PyTorch, scikit-learn, and MLflow.
• Develop and manage cloud-native infrastructure using Google Cloud Platform (GCP), Kubernetes, Docker, and multi-cloud environments (OCI, AWS, Azure).
• Implement and support data storage solutions including Oracle Exadata, HDFS, Hive, Google Cloud Storage, and key-value stores (Juno, Aerospike).
• Integrate CI/CD pipelines using Google Cloud Build, Jenkins, Ansible, and Harness for automated deployment and testing.
• Ensure robust security and compliance through role-based access controls, encryption, and integration with IAM tools (Hashicorp Key Vault, Sailpoint).
• Collaborate with cross-functional teams to deliver reliable, production-grade solutions for model/data/service workflows.
• Monitor, troubleshoot, and optimize platform performance using DataDog, Oracle Enterprise Manager, and other monitoring tools.
• Contribute to documentation, onboarding guides, and best practices for users and developers.

Requirements:

Required Technical Skills

• Proficiency in Python and experience with ML/AI model development (TensorFlow, PyTorch, scikit-learn, MLflow).
• Experience with data integration and ETL tools (Informatica, PySpark, Apache Kafka).
• Hands-on experience with cloud platforms (GCP, OCI, AWS, Azure), Kubernetes, and Docker.
• Strong understanding of data storage and management (Oracle Exadata, HDFS, Hive, GCS, key-value stores).
• Experience with CI/CD tools and automation for ML/data workflows.
• Knowledge of security best practices, IAM, and compliance requirements.
• Excellent troubleshooting, debugging, and communication skills.

Preferred Qualifications

• Experience with large-scale financial/ML platforms.
• Familiarity with agentic frameworks, GPT-powered APIs, and orchestration SDKs.
• Exposure to workflow automation (Control-M, Airflow) and enterprise scheduling.
• Experience with monitoring, dashboarding, and operational health tools.
• Knowledge of data privacy, vulnerability scanning, and compliance automation.