ai model evaluator

MERCOR

<div>About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Language Model Evaluator 
Type: Full-time or Part-time Contract Work 
Compensation: $23/hour 
Location: Geography restricted to Egypt, Saudi Arabia, UAE, USA
Role Responsibilities<ul><li>
Evaluate LLM-generated responses on their ability to effectively answer user queries.</li><li>
Conduct fact-checking using trusted public sources and external tools.</li><li>
Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies.</li><li>
Assess reasoning quality, clarity, tone, and completeness of responses.</li><li>
Ensure model responses align with expected conversational behavior and system guidelines.</li><li>
Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines.</li></ul>
Qualifications
Must-Have<ul><li>
Bachelor&rsquo;s degree</li><li>
Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in Arabic</li><li>
Significant experience using large language models (LLMs)</li><li>
Excellent writing skills</li><li>
Strong attention to detail</li><li>
Adaptable and comfortable moving across topics, domains, and customer requirements</li><li>
Background or experience in domains requiring structured analytical thinking</li><li>
Excellent college-level mathematics skills</li></ul>
Preferred<ul><li>
Prior experience with RLHF, model evaluation, or data annotation work</li><li>
Experience writing or editing high-quality written content</li><li>
Experience comparing multiple outputs and making fine-grained qualitative judgments</li><li>
Familiarity with evaluation rubrics, benchmarks, or quality scoring systems</li></ul>
Application Process (Takes 20&ndash;30 mins to complete)<ul><li>
Upload resume</li><li>
AI interview based on your resume</li><li>
Submit form</li></ul>
Resources &amp; Support<ul><li>
For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome</li><li>
For any help or support, reach out to: support@mercor.com</li></ul>
PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.</div>

Data Annotator

Senior Content Reviewer

Bilingual LLM Evaluator - Arabic Expert

Mercor

<div>
 
 About The Job
 
 
 
 
 Mercor
 
 connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include
 
 Benchmark
 
 ,
 
 General Catalyst
 
 ,
 
 Peter Thiel
 
 ,
 
 Adam D'Angelo
 
 ,
 
 Larry Summers
 
 , and
 
 Jack Dorsey
 
 .
 
 
 
 Position:
 
 Language Model Evaluator
 
 
 
 Type:
 
 
 Full-time or Part-time Contract Work
 
 
 
 
 Compensation:
 
 
 $23/hour
 
 
 
 
 Location:
 
 
 Geography restricted to Egypt, Saudi Arabia, UAE, USA
 
 
 
 
 Role Responsibilities
 
 
 
 <ul>
 <li>
 Evaluate LLM-generated responses on their ability to effectively answer user queries.
 </li>
 <li>
 Conduct fact-checking using trusted public sources and external tools.
 </li>
 <li>
 Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies.
 </li>
 <li>
 Assess reasoning quality, clarity, tone, and completeness of responses.
 </li>
 <li>
 Ensure model responses align with expected conversational behavior and system guidelines.
 </li>
 <li>
 Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines.
 
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 
 Must-Have
 
 
 
 <ul>
 <li>
 Bachelor&rsquo;s degree
 </li>
 <li>
 Native speaker or ILR 5/primary fluency (C2 on the CEFR scale) in Arabic
 </li>
 <li>
 Significant experience using large language models (LLMs)
 </li>
 <li>
 Excellent writing skills
 </li>
 <li>
 Strong attention to detail
 </li>
 <li>
 Adaptable and comfortable moving across topics, domains, and customer requirements
 </li>
 <li>
 Background or experience in domains requiring structured analytical thinking
 </li>
 <li>
 Excellent college-level mathematics skills
 
 
 
 </li>
 </ul>
 
 Preferred
 
 
 
 <ul>
 <li>
 Prior experience with RLHF, model evaluation, or data annotation work
 </li>
 <li>
 Experience writing or editing high-quality written content
 </li>
 <li>
 Experience comparing multiple outputs and making fine-grained qualitative judgments
 </li>
 <li>
 Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
 
 
 
 </li>
 </ul>
 
 Application Process (Takes 20&ndash;30 mins to complete)
 
 
 
 <ul>
 <li>
 Upload resume
 </li>
 <li>
 AI interview based on your resume
 </li>
 <li>
 Submit form
 
 
 
 </li>
 </ul>
 
 Resources &amp; Support
 
 
 
 <ul>
 <li>
 For details about the interview process and platform information, please check: https://talent.docs.mercor.com/welcome/welcome
 </li>
 <li>
 For any help or support, reach out to: support@mercor.com
 
 
 
 </li>
 </ul>
 
 PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
 
</div>

Conversational AI Analyst - Fully Remote

Talent Ledger

<div>
 
 
 Company Description
 
 
 
 Talent Ledger provides technical delivery services and access to top-tier, vetted engineering talent to help businesses achieve their goals with speed and precision. Specializing in areas such as AI, automation, fintech, blockchain, protocol security, data engineering, and cloud infrastructure, Talent Ledger supports a wide range of projects. From assembling small and efficient strike teams to rapidly deploying specialists for critical tasks, the company ensures high-priority work gets done efficiently without the need for extensive hiring cycles. Built on trust and reliability, Talent Ledger is the ideal partner for organizations that value agility and top-tier execution.
 
 
 
 
 
 
 Role Description
 
 
 
 The Multilingual AI Evaluator will assess and refine AI model outputs to ensure accuracy, relevance, and alignment with intended outcomes. Daily responsibilities include evaluating multilingual content, providing data-driven feedback, running quality assurance checks, and collaborating with teams to enhance performance. This is a full-time, on-site role based in Multan, requiring excellent language skills and familiarity with AI technologies and evaluation protocols.
 
 
 
 
 
 
 Qualifications
 
 
 <ul>
 <li>
 Proficiency in multiple languages, with strong written and verbal communication skills in at least two or more languages.
 </li>
 <li>
 Analytical and problem-solving skills, perception in evaluating AI-generated content, and attention to detail.
 </li>
 <li>
 Experience or familiarity with AI tools, machine learning concepts, and data evaluation workflows.
 </li>
 <li>
 Ability to work collaboratively in an on-site environment, meeting deadlines and producing high-quality evaluations.
 </li>
 <li>
 Understanding of cultural nuances and context in different languages to ensure relevancy and accuracy.
 </li>
 <li>
 Self-motivated with excellent time management and organizational abilities.
 </li>
 <li>
 Bachelor&rsquo;s degree in linguistics, computer science, or related field is preferred, though relevant experience will also be considered.
 </li>
 </ul>
 
 
 
 
 
 How to apply:
 
 
 
 Click the link below to get forwarded to our partner screening website:
 
 
 
 
 
 https://refer.micro1.ai/referral/jobs?referralCode=e4967b43-e0bc-457d-bea5-f38735a3f512&amp;linkId=f6ec34a0-c27b-49e1-9f81-a8f255471b89&amp;utm_source=referral&amp;utm_medium=share&amp;utm_campaign=job_referral
 
</div>

Multilingual AI Evaluator

DataAnnotation

<div>
 We are looking for a
 
 Pharmacy Technician
 
 to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 In this role, you will need to be an expert in healthcare. We are interested in a wide range of expertise, so relevant backgrounds include Revenue Cycle Management &amp; Patient Finance, Pharmacy Operations (Pharm Tech), Laboratory Services and Operations (Lab Tech), and Clinical Support &amp; Ambulatory Operations.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 Full-time or part-time remote position
 </li>
 <li>
 Choose which projects you want to work on
 </li>
 <li>
 Flexible schedule
 </li>
 <li>
 Projects are paid hourly starting at $20+ per hour
 </li>
 <li>
 Bonuses available for high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Provide AI chatbots with diverse and complex healthcare-related problems
 </li>
 <li>
 Evaluate AI outputs for logic, accuracy, and performance
 </li>
 <li>
 Ensure the medical accuracy and overall quality of model responses
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 A current or in-progress medical or healthcare-related degree
 
 
 </li>
 </ul>
 
 Notes:
 
 Payment is made via PayPal. We will never ask you for any money. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contractor position.
 
 
 
 #generalist
 
</div>

Pharmacy Technician

Search 10000+ Ai model evaluator jobs near you this March. New roles added regularly across leading industries. Apply today.

10000 ai model evaluator jobs Near Me

Data Annotator

Senior Content Reviewer

Bilingual LLM Evaluator - Arabic Expert

Conversational AI Analyst - Fully Remote

Conversational AI Analyst - Fully Remote

Multilingual AI Evaluator

Pharmacy Technician