Lead SW Architect

We are seeking a highly skilled Lead Software Engineer to drive our machine learning operations (ML Ops), large language model (LLM) and agentic AI integration, Databricks administration and governance, data lake management, and cloud platform initiatives. The ideal candidate will architect, implement, and optimize scalable ML pipelines, manage cloud infrastructure (GCP & Azure), lead prompt engineering for generative and agentic AI, establish robust CI/CD practices, ensure Databricks platform governance and compliance, and oversee data lake architecture and operations. You will mentor a team of engineers and collaborate with data scientists, product managers, and stakeholders to deliver innovative AI solutions

Essential

· 10 plus years of software engineering experience, with 3 and above years in ML Ops, agentic AI, Databricks administration, and cloud platforms.

· Experience in improving developer experience on ML OPS and DEV OPS practices.

· Knowledge of observability platform like Dynatrace for API monitoring, Lang smith for model monitoring

· Deep expertise in ML Ops tools (MLflow, Kubeflow, Vertex AI, Azure ML, etc.).

· Hands-on experience with LLMs (OpenAI, Google Gemini, Azure OpenAI, etc.), agentic AI systems, and prompt engineering.

· Proven experience architecting and managing data lake solutions (Azure Data Lake, GCP BigLake, etc.). · Advanced knowledge of Databricks platform administration and governance.

· Strong proficiency in Python, and experience with other languages (Java, Go, etc.) is a plus.

· Advanced knowledge of GCP and Azure cloud services.

· Proven experience designing and managing CI/CD pipelines (GitHub Actions, Azure DevOps, Jenkins, etc.).

· Solid understanding of containerization (Docker, Kubernetes).

· Experience with security, compliance, and governance in cloud ML and agentic AI environments.

· Familiarity with MLOps for LLMs, agentic AI, and generative AI.

· Contributions to open-source ML Ops, agentic AI, Databricks, or cloud projects.

· Certifications in GCP, Azure, Databricks, or ML Ops.

· Experience with data engineering, ETL, and big data technologies.

· Excellent communication, leadership, and problem-solving skills.

· Knowledge of data governance, lineage, and compliance in cloud environments.

· Strong problem-solving, communication, and stakeholder management skills.

· Ability to lead cross-functional teams and mentor junior ML engineers.

· Experience integrating third-party APIs, managing secrets, and optimizing cloud costs.

Essential

· Bachelor’s or Master’s degree in Computer Science, Engineering, or related field.