Software QA Engineer

Role Overview:

We seek a Senior QA Engineer to lead the design, testing, validation, and automation of QA strategies across our Data & AI programs. The role spans testing data pipelines, AI/ML models, Generative AI (LLM) applications, and multi-agent (Agentic AI) workflows, ensuring solutions meet the highest standards of accuracy, reliability, fairness, security, and compliance. The ideal candidate will bring hands-on expertise in testing LLM-based applications and Agentic AI systems, along with strong foundations in data quality assurance and AI/ML model validation.

Key Responsibilities:

• Define
  and enforce QA frameworks, processes, and standards for data pipelines,
  AI/ML models, GenAI/LLM applications, and agent-based AI systems.
• Develop and automate data validation tests (integrity, consistency, lineage, schema drift).
• Design and execute ML model validation tests, including performance, fairness, bias, and reproducibility.
• Implement
  automated prompt-response validation for GenAI/LLM applications
  (semantic similarity, factuality, hallucination detection).
• Integrate QA processes into CI/CD, MLOps, and LLMOps workflows.
• Develop monitoring and alerting systems for data drift, model decay, and pipeline failures.
• Define, track, and report on AI evaluation metrics.
• Ensure compliance with AI governance, ethics, security, and privacy requirements.
• Document QA processes, maintain reusable test cases, and support audits.

Skills & Qualifications:

• 5+ years of experience in Quality Assurance, with at least 2+ years focused on Data, AI/ML, GenAI, or Agentic AI testing.
• Proven experience testing LLM/GenAI applications (prompt testing, RAG pipelines, grounding validation, hallucination detection).
• Experience testing Agentic AI workflows (multi-agent orchestration, decision-making validation, safety guardrails).
• Strong Python skills and proficiency with testing frameworks (PyTest, unittest).
• Knowledge of ML evaluation metrics (precision, recall, F1, ROC, fairness, bias testing).
• Familiarity with AI governance, compliance, and privacy testing.
• Strong analytical skills, with attention to detail and quality.
• Excellent communication and collaboration skills for cross-functional work.
• Excellent writing/ verbal communication in Arabic/ English