FIND_THE_RIGHTJOB.
QA Data Science Engineer
JOB_REQUIREMENTS
Hires in
Not specified
Employment Type
Not specified
Company Location
Not specified
Salary
Not specified
Come work at a place where innovation and teamwork come together to support the most exciting missions in the world!
Job Description
We are seeking a Data Science focused QA engineer to develop next-generation Security Analytics products. You will work closely with Data scientists, engineers and product managers to design and optimize AI driven security solutions.
As QA engineer, the ideal candidate has a strong background in Backend engineering, system integrations, ML,AI and data pipelines.
Responsibilities (QA Engineer – Data Science / ML)
- Establish QA best practices for Traditional ML and Generative AI workflows, including:
- Functional and regression testing of ML pipelines using pytest and Airflow/Dagster test utilities and API testing tools (e.g., Postman, pytest-httpx).
- Validate data contracts, schemas, and API compatibility across services using Pandera, and custom validation rules.
- Model behavior validation (input/output ranges, invariants, edge cases) using NumPy, SciPy, and statistical assertions
- Runtime and performance testing for inference latency, throughput, and resource usage using Locust, k6, or custom load tests.
- Integrate ML-specific tests into CI/CD pipelines using GitHub Actions, GitLab CI, or Jenkins, alongside containerized workflows (Docker, Kubernetes).
Implement LLM-specific testing, including:
- Prompt and response validation, determinism checks, and regression testing using LangSmith.
- Evaluation of hallucinations, toxicity, and policy adherence using LLM-as-a-judge and/or rule-based checks.
- Cost, token usage, and timeout monitoring for GenAI workflows
Verify logging, monitoring, and alerting for ML services using Prometheus, Grafana, and cloud-native observability tools.
Requirements:
- BS or MS in Computer Science or a related field.
- 2-5 years of experience in Data or Machine Learning projects.
- Familiarity and experience of GenAI applications and tools -PyTorch, LangChain, vLLM etc.
- Demonstrates a commitment to continuous learning in this rapidly evolving field.
- Tools listed in the responsibilities section.
Similar jobs
No similar jobs found
© 2026 Qureos. All rights reserved.