QA Engineer – AI Automation Validation

Position Summary

IEM is seeking a detail-oriented and technically skilled QA Engineer / Automation Specialist to ensure the quality, reliability, and safety of our AI agents and automation workflows. In this role, you will develop comprehensive test strategies, build automated testing frameworks, and validate AI outputs for accuracy and consistency before production release. You will work closely with developers, business analysts, and project managers to identify edge cases, track defects, and provide the final quality sign-off for enterprise AI and automation solutions deployed across our Salesforce, MuleSoft, and AI platforms.

Ideal Candidate Profile

The ideal candidate is a detail-oriented QA engineer with strong analytical and critical-thinking skills, able to design effective test scenarios, identify edge cases and failure modes, and diagnose complex issues through clear reproduction steps and root-cause analysis. They are a proactive quality advocate who takes ownership of end-to-end validation for AI agents and automation workflows, ensuring reliability, consistency, and safety prior to release. In addition, the ideal candidate communicates clearly with both technical and non-technical partners and produces thorough, well-organized documentation (test plans, test cases, and quality reports). They collaborate effectively across teams, adapt quickly to new tools and platforms, and bring a continuous-learning mindset to stay current with evolving testing approaches and technologies.

Key Responsibilities

• Develop comprehensive test strategies: Create and maintain test plans covering functional, performance, and regression testing for AI agents and automation workflows.
• Build and maintain automated test frameworks: Develop and maintain automated test scripts/frameworks to ensure bot responses and workflow outputs remain accurate, consistent, and reliable over time.
• Build and maintain test assets: Create/curate golden conversation sets, synthetic test data, evaluation rubrics, and regression suites to measure quality over time.
• Validate AI agent safety and reliability: Test tool-calling end-to-end (tool selection, parameter mapping, sequencing, retries/timeouts, error handling, and recovery), including negative/edge cases; confirm outputs align to business rules and document risk mitigations.
• Test integration/workflow integrity: Validate API contracts, data transformations, and downstream impacts (idempotency, duplicate prevention, partial failures, and latency) across environments.
• Manage defects through resolution: Identify, log, prioritize, and track defects; partner with developers to reproduce issues and validate fixes.
• Provide release certification: Deliver final quality sign-off prior to production release, ensuring defined acceptance criteria and quality standards are met.
• Align on acceptance criteria: Partner with Business Analysts to ensure user stories include clear, testable acceptance criteria and that test coverage maps to requirements.
• Document testing and metrics: Maintain test cases, data, results, and quality metrics; communicate outcomes and risks.
• Participate in Agile ceremonies: Engage in planning, grooming, standups, and retrospectives to embed quality throughout delivery.
• Drive continuous improvement: Identify opportunities to improve automation coverage, testing effectiveness, and quality practices.

Qualifications

• 3+ years in a QA, test automation, or software testing role with demonstrated experience developing and executing test strategies.
• Test automation expertise: Strong hands-on experience building automated test frameworks using tools such as Selenium, Playwright, Cypress, Jest, Pytest, or similar frameworks.
• Scripting and programming skills: Proficiency in at least one programming or scripting language (Python, JavaScript, Java, C#) for test automation development.
• Agile methodology: Direct experience working in an Agile/Scrum environment with hands-on involvement in sprint cycles and continuous integration.
• Defect management tools: Experience using bug tracking and test management tools such as Jira, Azure DevOps, TestRail, or similar platforms.
• Bachelors level degree is preferred, but not required.

Location

• The position is fully remote but may require up to 15% travel to IEM facilities, supplier sites, compliance audits or conferences. Candidate may be required to work from the Austin office in the future as business needs evolve.