ML Engineer + Data Scientist

Key Responsibilities

1. Machine Learning Development

• Design, develop, and optimize ML models for predictive analytics, classification, regression, NLP, or other use cases.

• Perform exploratory data analysis (EDA), feature engineering, data preprocessing, model selection, tuning, and evaluation.

• Implement responsible AI practices including model performance monitoring, drift detection, and interpretability.

2. Databricks Platform Expertise

• Develop and maintain Databricks notebooks, jobs, Delta Lake pipelines, and MLflow tracking workflows.

• Optimize large-scale data workloads using Spark, Delta Live Tables, and Databricks clusters.

• Manage data access, lineage, and governance through Databricks Unity Catalog.

3. MLOps & Productionization

• Build and maintain end-to-end ML pipelines using Databricks, MLflow, and CI/CD tools (Azure DevOps / GitHub Actions / Jenkins).

• Deploy models to production using MLflow Models, Databricks Model Serving, or containerized microservices.

• Implement automated monitoring for model drift, data quality, and inference performance.

• Support continuous model retraining strategies and versioning of datasets, features, and models.

4. Data Engineering Collaboration

• Work closely with Data Engineering to design scalable ETL/ELT pipelines on Delta Lake.

• Ensure high availability of feature pipelines and support/maintenance via the feature store (Databricks Feature Store).

5. API Integrations

• Develop RESTful APIs for real-time model inference and analytics workflows.

• Integrate with internal and external systems using API gateways, event-driven architectures, or message queues.

• Ensure security, observability, and performance of deployed endpoints.

6. Governance, Security & Compliance

• Apply data governance best practices across Unity Catalog, including permissions, lineage tracking, and data auditing.

• Comply with enterprise security controls, secrets management, and model governance frameworks.

Required Skills & Experience

• 3-8+ years of experience in Data Science / ML Engineering (adjust as needed).

• Strong hands-on experience with Databricks, Spark, Delta Lake, and MLflow.

• Proficiency in Python, SQL, and common ML libraries (scikit-learn, PySpark MLlib, TensorFlow/PyTorch optional).

• Solid understanding of MLOps concepts: CI/CD, feature stores, monitoring, model deployment, pipelines.

• Experience integrating ML systems via REST APIs or event-driven services.

• Deep understanding of ML lifecycle: data ingestion training evaluation deployment monitoring.

• Familiarity with cloud platforms (Azure, AWS, or GCP, preferably Azure Databricks).

• Experience with Unity Catalog data governance and access control.