QA Consultant

Take your career to the next level with the only consulting firm born in AI and delivering with AI.

At Atrium, we’re not simply adapting to an AI-driven world — we’ve helped define it since we were founded. Our clients partner with us because we turn potential into measurable impact, reshaping industries, realizing exponential value, and empowering organizations to thrive in an era of unprecedented technological advancement.

As pioneers in AI-assisted delivery, we’re constantly optimizing how we deliver services for greater speed, accuracy, and efficiency. This commitment allows us to repeatedly deliver outcomes that other Salesforce and Snowflake partners merely promise. Care to join us?

Who are you?

You’re a smart collaborator who likes solving complex problems and takes ownership to get things done. You stay up to date with the latest and greatest in business and technology tools, platforms, and languages — and want to ensure your clients do too. You love working across teams and are enthusiastic about doing your part to ensure everyone succeeds.

What will you be doing at Atrium?

We are seeking a highly skilled AI QA Consultant to ensure the quality, functionality, and performance of our AI solutions. This role is responsible for testing across the Salesforce platform with a specialised focus on validating AI-powered features, agentic workflows, and integrations with Large Language Models (LLMs). This position bridges traditional Salesforce QA with modern AI testing principles, requiring a deep understanding of Salesforce best practices, a keen eye for detail, and the technical acumen to test complex, non-deterministic systems. You will work closely with AI developers, Salesforce administrators, and business stakeholders to deliver robust and reliable AI-driven user experiences.

Responsibilities include:

• Conduct thorough manual and exploratory testing of AI features on the Salesforce platform. You will identify, document, and track defects in Jira, clearly distinguishing between standard configuration bugs and nuanced, probabilistic issues inherent in AI model outputs
• Take ownership of data quality for the entire AI fine-tuning lifecycle. You will be responsible for validating and helping prepare the high-quality, unbiased datasets that are essential for building reliable and effective enterprise AI
• Develop detailed test plans, test cases, and test scripts specifically designed to evaluate the accuracy, relevance, safety, and performance of LLM responses and AI-driven logic
• Collaborate with product owners and AI developers to understand requirements and design comprehensive test strategies for Salesforce solutions that incorporate AI components like intelligent chatbots, RAG-based knowledge retrieval, automated case summarization, and agentic process automation
• Create scenarios to test for prompt injection vulnerabilities, model hallucinations, data grounding issues, and bias in AI outputs
• Assess the performance, latency, and resource consumption of AI features to ensure they meet user experience benchmarks
• Facilitate User Acceptance Testing (UAT) for AI-powered features. Guides business users on how to evaluate and provide feedback on intelligent systems
• Stay updated with the latest advancements in LLMs, prompt engineering techniques, and AI testing methodologies, and champion the adoption of best practices

Requirements for this role:

• Strong foundation with 2-3 years of dedicated QA experience on the Salesforce platform. It is crucial for validating how AI features interact with core Salesforce objects, data models, and business workflows
• A solid understanding of modern AI/NLP concepts, including Retrieval-Augmented Generation (RAG), model fine-tuning, and the principles of creating feedback loops for reinforcement learning (e.g., RLHF)
• Proficiency in Python, used for writing test scripts, data manipulation, and interacting with AI APIs
• Hands-on experience testing applications built with LLM APIs (like OpenAI, Gemini), frameworks such as LangChain/LlamaIndex, and various Vector Databases
• Working knowledge of SQL, NoSQL, and vector databases
• Excellent analytical and communication skills, with experience thriving in an Agile/Scrum environment

Next Steps

Recruiting at Atrium is highly personalized. While some candidates may complete the hiring process quickly, others may take a bit longer, depending on the role and its requirements. We’re excited to get to know you and ensure you get to know our team along the way.

At Atrium, we believe a diverse workforce allows us to match our growth ambitions and drive inclusion across the business. We are an equal opportunity employer and all qualified applicants will receive consideration for employment.