Senior Quality & AI Reliability Engineer

Industry: Technology, Information and Internet

Seniority level: Mid-Senior level

Min Experience: 8 years

Location: Bengaluru

JobType: full-time

We are seeking a highly experienced Senior Quality & AI Reliability Engineer to drive end-to-end testing, automation strategy, and reliability validation across advanced AI-powered systems and distributed software platforms. In this role, you will architect scalable automation frameworks, enhance coverage for complex workflows, and ensure delivery of secure, deterministic, and high-performing releases. You will work closely with engineering, product, and data science teams to shape quality strategy and safeguard system integrity as the product evolves rapidly.

This role demands deep technical expertise in automation, API and UI testing, AI/ML system validation, and modern DevOps practices, coupled with the ability to analyze failure patterns, implement preventive improvements, and ensure seamless continuous delivery cycles.

Architect, implement, and maintain large-scale automation frameworks covering web interfaces, APIs, microservices, and distributed workloads.

Build and optimize automated test suites leveraging Python, Selenium, Rest Assured, Pytest/JUnit, and modern testing tools.

Develop structured testing methodologies for AI/ML systems, including validation for model outputs, LLM evaluation, deterministic outcomes, and RAG-based workflows.

Design automated checks for model drift, hallucination control, safety guardrails, performance stability, and end-to-end workflow reliability.

Partner with engineering and data science teams to define testability standards, acceptance criteria, and automation-first development strategies.

Lead functional, integration, performance, resilience, and scalability testing across cloud-based environments and CI/CD pipelines.

Troubleshoot system-level failures, conduct RCA, and drive quality improvements to ensure customer trust and product reliability.

Monitor automated execution dashboards, ensure test coverage metrics, and improve execution efficiency and stability.

Advocate for a quality-first culture, mentoring junior team members and enabling intelligent automation practices.

Create and maintain well-structured documentation and testing guidelines for scalable knowledge sharing.

8–12 years of strong automation and software testing experience, preferably in product-based or large-scale distributed system environments.

Advanced proficiency with Python, Selenium, Rest Assured, and modern automation frameworks.

Practical experience validating AI & LLM-driven features, RAG pipelines, agentic automation workflows, and familiarity with PyTorch.

Deep understanding of automated CI/CD pipelines, release processes, Git workflows, cloud-based environments, and Agile development.

Strong systems thinking and debugging skills with the ability to analyze sophisticated architectures.

Excellent communication, documentation, and cross-functional collaboration abilities.

Experience with high-scale performance testing environments.

Knowledge of observability tools, telemetry analysis, and production reliability strategies.