Databricks Platform Administrator/10+yrs/Remote/Contract

Job Title - Databricks Platform Administrator

Location – Remote

Duration – 12-24 Months

Job Summary: We are seeking a skilled Databricks Platform Administrator / Engineer with proven experience supporting modern data platforms in a Property & Casualty (P&C) insurance setting. The ideal candidate will be responsible for managing, configuring, optimizing, and maintaining the Databricks environment to support data engineering, analytics, and machine learning workloads in a secure and compliant manner.

Education: Bachelor’s or Master’s degree in Computer Science, Engineering, Information Systems, or related field.

Required Qualifications:

10+ years of experience in data platform engineering or system administration.

2+ years of hands-on experience with Databricks administration on Azure

Solid understanding of Apache Spark, Delta Lake, and Lakehouse architecture.

Proficiency in scripting and automation using Python, Bash, PowerShell, or Terraform.

Familiarity with data security, audit, and compliance frameworks relevant to P&C insurance (e.g., GDPR, NYDFS, SOC 2).

Experience with Azure Data Services, ADLS, Key Vault, and Azure DevOps

Strong understanding of the P&C insurance data ecosystem (policy, claims, underwriting, actuarial, etc.).

Preferred Qualifications:

Databricks Certified Associate / Professional (Administrator / Data Engineer) certification.

Experience integrating Databricks with core systems like Guidewire, Duck Creek, or custom P&C platforms.

Knowledge of industry standards like ACORD data models and ISO data feeds.

Exposure to tools like Unity Catalog, Purview, Collibra, Informatica, or similar.

Soft Skills:

Strong problem-solving and analytical skills.

Ability to work cross-functionally in an agile team environment.

Excellent communication and stakeholder management skills.

Experience working in regulated enterprise environments with strong data governance standards.

Key Responsibilities:

Platform Administration & Management:

Manage and administer Databricks workspaces, clusters, jobs, libraries, and notebooks.

Implement access controls using Unity Catalog, SCIM, and workspace-level RBAC.

Configure and monitor autoscaling, cluster policies, and compute usage to optimize cost and performance.

Support platform upgrades, patching, and operational automation.

Security, Compliance & Governance:

Enforce security policies aligned with P&C insurance industry regulations.

Integrate with enterprise identity providers (Azure AD, Okta) and manage SSO.

Support data governance using tools like Unity Catalog, Immuta, or Collibra.

DevOps & Automation:

Build CI/CD pipelines for notebooks, jobs, and ML workflows using tools like Azure DevOps, GitHub Actions, or Jenkins.

Automate platform provisioning and configurations using Infrastructure-as-Code (Terraform, ARM, etc.).

Monitoring & Troubleshooting:

Set up logging, monitoring, and alerting using tools like Azure Monitor, Datadog, or Splunk.

Troubleshoot performance issues, job failures, and cluster errors.

Collaboration & Enablement:

Work closely with Data Engineers, Data Scientists, and Business Analysts to ensure optimal platform usage.

Develop platform usage guidelines, onboarding documentation, and best practices.

Provide technical support and training to users.

Job Types: Full-time, Contractual / Temporary
Contract length: 24 months

Pay: ₹100,000.00 - ₹120,000.00 per month

Benefits:

• Work from home