ML Engineer(Databricks MLOps)

ML Engineer within the Data Science and Machine Learning team leverages and third-party software to create solutions to business problems defined by specific business requirements. In this position, you will draw upon technical, AI/ML engineering, Data and MLOps experience to solve complex marketing analytics problems on very large volumes of data.

As a Machine Learning (ML) Engineer You will build and operationalize machine learning pipelines involving terabytes of data. You will have the responsibility to help define requirements, create software designs, implement code to these specifications, As an ML Engineer, you will be expected to:

Develop and maintain a comprehensive enterprise architecture for AI/ML/GenAI initiatives, ensuring alignment with overall business strategy and technology roadmap.

Architect hyperscale MLOps solutions and pipelines

Work with Applied Scientists, Data Scientists, Product owners, ML Engineers, and Software Engineers to design and deliver ML solutions in production at scale.

Develop automated AI and ML workflows and end-to-end pipelines for data preparation, training, deployment, monitoring, etc., and ensure the quality of architecture and design of our ML systems and data infrastructure.

Leverage AI to develop GenAI powered solutions to complement our data science and product build capabilities

Assess current state AI/ML/GenAI capabilities across various business domains, identify gaps, and design target state architectures to drive innovation, revenue growth and operational excellence.

Lead transformational initiatives to bridge the gap between current and desired AI/ML capabilities, collaborating with cross-functional teams to ensure successful implementation.

Establish governance frameworks and decision criteria for AI/ML and GenAI projects, ensuring adherence to industry standards, regulatory requirements, Responsible AI and architectural principles.

Create and maintain reference architectures, patterns, and best practices for AI/ML lifecycle and integration within enterprise ecosystem.

Lead the technology evaluation and process improvements to drive Experimentation, Model Development and ML Ops at scale

Lead and drive standardization of LLM onboarding process, RAG pipelines and application development

Conduct architecture reviews and risk assessments for proposed AI/ML solutions, ensuring they meet security, scalability, and interoperability requirements.

Utilize advanced data science techniques (e.g., Natural Language Processing, clustering, predictive analytics, regression analyses, survival analysis, segmentation, and experimentation) to propose enhancements and innovations to business processes.

Conduct sophisticated statistical analyses and maintain high reliability of machine learning pipelines in production environments, ensuring minimal downtime and optimal performance.

Collaborate with business leaders and product to identify opportunities for AI/ML-driven innovation and guide the development of use cases that deliver tangible business value.

Foster a culture of continuous learning and innovation in AI/ML practices across the enterprise architecture team and broader organization.

Bachelor’s Degree in Computer Science or Associate Degree & 3+ years of development experience or equivalent experience

8+ years of experience architecting scalable ML infrastructure and big data systems.

Databricks Architect Certification is required

6+ years of architecting solutions using Databricks. Strong experience using Mosaic AI, Unity Catalogue, mlflow, workflow orchestration and other databricks native MLOps capabilities.

At least 1+ year experience in GenAI (Technical familiarity with 2 or more) - OpenAI API, Bedrock API, Vertex API, LangGraph, other agentic frameworks

High attention to detail and proven ability to manage multiple, competing priorities simultaneously.

Experience MLOps and orchestration tools such as Airflow, Kubeflow, DAGster, Optuna, Mlflow or other similar MLOps tools.

Experience with operationalizing and migrating ML models into production at scale.

Developing Large scale model inference solutions using parallel execution framework using spark, EMR, databricks

Experience developing complex orchestration and MLOps pipelines stitching together large volumes of data for training and scoring

Experience with Large Language Models, fine tuning and deployment frameworks using HuggingFace capabilities or cloud provider solutions such as Amazon Bedrock, Vertex AI model garden etc.

Familiarity with Vector databases such as Pinecone, ChromaDB or similar tools.

Experience in CI/CD/DevOps, Deployment and Automation Tool – CI/CD, Jenkins, Terraform, Cloud Formation Template or similar

Proficiency with Apache Spark, EMR/DataProc and Cloud based tools (Snowflake, Redshift, EMR, Glue, Step Functions, Lambda, Step functions, AWS Batch, or similar etc.).

Experience with ML libraries like H20, scikit learn and deep learning frameworks (PyTorch, TensorFlow, etc.).

Experience with end-to-end software development and life cycle of ML solutions.

Excellence in technical communication with scientists and engineers.

At least 2years Database (SQL) experience, Linux

At least 6+ years of AWS infrastructure experience - Cloud run, App server, RDS, S3, EC2, EMR or equivalent GCP experience

Databricks Certification

Langgraph, Databricks MLFlow experience, Docker experience, Kubernetes experience

Knowledge of LLM observability platforms

Good communication skills: communicate ideas clearly and effectively to other members of the analytics team and to the client at multiple levels (both technical and business)

Analytic problem-solving skills with the ability to think outside-the-box

Analytical thinker that excels at analyzing and understanding data to answer questions

Excellent understanding of data concepts, data architecture, data manipulation/engineering, and data engineering design

Passion for considering how projects fit into the wider business picture

An understanding in multiple types of programming languages in order to be adaptable (statically typed vs. dynamically typed and object-oriented vs. procedural)

Self-Starter – Able to work independently with little guidance

Adaptable - Able to adapt to diverse technical challenges and systems

Ability to formulate and present insights with gathered data to both technical and non-technical peers, leaders, and clients