Specialist I - Software Engineering-Devops Engineer with Databricks

9 - 12 Years
2 Openings
Bangalore, Chennai, Hyderabad, Kochi, Pune, Trivandrum

Role-Senior DevOps Engineer with strong expertise in Databricks platform administration

• Databricks Platform Administration:

  • Manage Databricks workspaces, clusters, jobs, and user access.

  • Handle upgrades, patching, and platform performance tuning.

  • Implement security controls and integrate with enterprise identity providers (e.g., SSO, SCIM, role-based access).

• DevOps Automation:

  • Design and implement CI/CD pipelines for data and ML workflows using tools like Azure DevOps, GitHub Actions, Jenkins, etc.

  • Automate infrastructure provisioning using Terraform, ARM, or similar IaC tools.

• Monitoring & Reliability:

  • Implement monitoring, ing, and logging solutions for Databricks and related services using tools such as Azure Monitor, Prometheus, Grafana, or Datadog.

  • Ensure platform reliability, scalability, and cost optimization.

• Collaboration & Support:

  • Work closely with data engineers, data scientists, and platform teams to support project delivery and operational needs.

  • Provide troubleshooting and root cause analysis for platform issues and job failures.

• Security & Compliance:

  • Ensure the Databricks environment complies with organizational and regulatory standards (e.g., data sovereignty, GDPR).

  • Manage data access, encryption, and secure networking configurations.

• Documentation & Best Practices:

  • Maintain documentation for architecture, configurations, operational procedures, and support guides.

  • Define and enforce platform best practices and standards.

Required Qualifications

• Experience:

  • 8+ years in DevOps, cloud infrastructure, or platform engineering roles.

  • 2+ years hands-on experience managing Databricks on AWS, Azure, or GCP.

• Technical Skills:

  • Proficient in scripting (Bash, Python, PowerShell).

  • Experience with Infrastructure-as-Code (Terraform, ARM, CloudFormation).

  • Strong understanding of CI/CD tools and pipeline development.

  • Deep knowledge of Linux systems, networking, and security.

• Cloud Platforms:

  • Strong experience with cloud environments (Azure preferred; AWS/GCP also valuable).

  • Familiarity with cloud-native services such as Key Vaults, Storage, Networking, IAM, etc.

• Soft Skills:

  • Strong problem-solving skills and ability to work independently.

  • Excellent communication and collaboration skills across cross-functional teams.

Terraform,Git,Python,Aws, Databricks platform admin