AI Tester with Automation Testing

About the role
As an AI Tester with Automation expertise, you will make an impact by validating the quality, performance, and reliability of next generation AI and LLM powered applications. You will be a valued member of the Quality Engineering and Assurance team and work collaboratively with AI engineers, automation architects, and cross functional stakeholders to ensure robust, scalable, and compliant AI solutions.
________________________________________
In this role, you will:

• Design and execute automated test scripts using Python and Playwright for AI, LLM, and ML workloads.
• Validate LLM based applications, including prompt flows, RAG pipelines, vector search, and retrieval logic.
• Test multi agent orchestration workflows, including MCP/A2A protocol–based interactions.
• Conduct functional, integration, API, and data validation testing across complex ML pipelines.
• Collaborate with engineering teams to troubleshoot defects, optimize model behavior, and ensure compliance and accuracy of AI outputs.

________________________________________
Work model
At Cognizant, we strive to provide flexibility wherever possible, and we are here to support a healthy work life balance through our various wellbeing programs. Based on this role’s business requirements, this is an onsite position requiring presence in Weehawken, New Jersey.
The working arrangements for this role are accurate as of the date of posting and may change based on project engagement, business needs, or client requirements.
________________________________________
What you need to have to be considered

5 to 7 yrs. of QA experience with

• Strong programming proficiency in Python and hands on automation experience with Playwright.
• Working knowledge of AI/ML algorithms, including NLP, classification, clustering, and LLM evaluation.
• Experience with LLM development frameworks such as LangChain, LangGraph, vector databases, RAG architectures, and prompt engineering.
• Familiarity with multi agent AI architectures, including multi agent orchestration using MCP and A2A protocols.
• Strong analytical skills with a solid understanding of data modeling and data transformation using SQL.
• Experience testing or consuming APIs, RESTful services, and data connectors.
• Awareness of source code control systems such as GitLab, along with CI/CD build and deployment pipelines.

________________________________________
These will help you stand out

• Hands on experience with Generative AI, GPT models, or enterprise LLM deployments.
• Exposure to cloud platforms—especially Microsoft Azure—for AI, ML, or infrastructure workloads.
• Good understanding of Neo4j, knowledge graphs, or graph based reasoning for LLM workflows.
• Experience testing end to end AI pipelines, including data ingestion, evaluation, and observability.
• Ability to work in fast paced, innovative environments delivering scalable AI solutions
• Cognizant will only consider applicants for this position who are legally authorized to work in the United States without company sponsorship. *Please note, this role is not able to offer visa transfer or sponsorship now or in the future*

We're excited to meet people who share our mission and can make an impact in a variety of ways. Don't hesitate to apply, even if you only meet the minimum requirements listed. Think about your transferable experiences and unique skills that make you stand out as someone who can bring new and exciting things to this role.

Benefits:
Cognizant offers the following benefits for this position, subject to applicable eligibility requirements:

• Medical/Dental/Vision/Life Insurance
• Paid holidays plus Paid Time Off
• 401(k) plan and contributions
• Long-term/Short-term Disability
• Paid Parental Leave
• Employee Stock Purchase Plan