Solutions Architect

Key Skills: Databricks, PySpark, and architecture solutions.
Secondary Skills: Unity Catalog, StarBurst, Azure DevOps, CICD, and orchestration layers.
Job Location: Hyderabad, Pune, Coimbatore, and Bengaluru
Mode of Work: Work From Office

Experience: 12+ Years

About the job:
We are seeking an experienced Databricks Architect with strong expertise in PySpark and architecture solutions. This role involves leading migration projects, designing cost-effective data platforms, and ensuring smooth data governance and orchestration in Databricks environments. The Architect will collaborate with various teams to build secure, scalable, and high-performance data pipelines and analytical layers.

Know your team:

At ValueMomentum Technology Solution Centers, we are a team of passionate software engineering teams who thrive on tackling complex business challenges with innovative solutions catering to the P&C insurance value chain. We achieve this through a strong software engineering foundation and by continuously refining our processes, methodologies, tools, agile delivery teams, and core engineering archetypes. Our core expertise lies in six key areas: Platforms, Infra/Cloud, Application, Data, Core, and Quality Assurance.

Join our team that invests in your growth. Our Infinity Program empowers you to build your career with role-specific skill development, leveraging immersive learning platforms. You'll have the opportunity to work with some of the best minds serving insurance customers in the US, UK and Canadian markets.

Responsibilities:

• Lead migration from Native Spark to Databricks.
• Architect Hadoop migration to Databricks.
• Build Data Governance solutions using tools like Unity Catalog and StarBurst.
• Develop orchestration workflows using Databricks and ADF.
• Implement CICD pipelines for Databricks in Azure DevOps.
• Process near-real-time data through Auto Loader and DLT pipelines.
• Design secure, scalable infrastructure for Delta Lake and Spark SQL.
• Optimize Databricks environments for cost-effectiveness.
• Extract complex business logic from on-prem solutions like SSIS, Informatica, Vertica, etc., into PySpark.
• Build analytical dashboards on top of the Databricks Lakehouse.

Requirements:

• 12+ years of overall experience, with at least 4+ years as a solution architect.
• Experience with Databricks, PySpark, and modern data platforms.
• Expertise in cloud-native data platform architecture and solution design.
• Familiarity with Agile/Scrum methodologies.
• Experience in pre-sales and in the P&C insurance domain is an advantage.