model evaluation specialist

YO IT CONSULTING

<div>Work Mode: Remote 
Engagement Type: Independent Contractor 
Schedule: Full-Time or Part-Time Contract
Role:
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
This project focuses on evaluating and improving general chat behavior in large language models (LLMs). You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations.
What You&rsquo;ll Do
<ul><li>Evaluate LLM-generated responses on their ability to effectively answer user queries
</li><li>Conduct fact-checking using trusted public sources and external tools
</li><li>Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
</li><li>Assess reasoning quality, clarity, tone, and completeness of responses
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a Bachelor&rsquo;s degree
</li><li>You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Arabic
</li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly articulate nuanced feedback
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>You are adaptable and comfortable moving across topics, domains, and customer requirements
</li><li>You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
</li><li>You have excellent college-level mathematics skills
</li></ul>Nice-to-Have Specialties
<ul><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience writing or editing high-quality written content
</li><li>Experience comparing multiple outputs and making fine-grained qualitative judgments
</li><li>Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
</li></ul>What Success Looks Like
<ul><li>You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
</li><li>You produce clear, consistent, and reproducible evaluation artifacts
</li><li>Your feedback leads to measurable improvements in response quality and user experience
</li></ul><h2 class="jobSectionHeader">Contract and Payment Terms</h2>
<ul><li>You will be engaged as an independent contractor.</li>
<li>This is a fully remote role that can be completed on your own schedule.</li>
<li>Projects can be extended, shortened, or concluded early depending on needs and performance.</li>
<li>Payments are weekly on Stripe or Wise based on services rendered.</li></ul></div>

Arabic & English Ai Trainer - Remote

<div>Location: Remote
Type: Full-time or Part-time Contract Work
 Fluent Language Skills Required: English &amp; Brazilian Portuguese
Why This Role Exists :
This organization works with AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
This project focuses on evaluating and improving general chat behavior in large language models (LLMs). You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations.
What You&rsquo;ll Do :
<ul><li><ul><li>Evaluate LLM-generated responses on their ability to effectively answer user queries
</li><li>Conduct fact-checking using trusted public sources and external tools
</li><li>Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
</li><li>Assess reasoning quality, clarity, tone, and completeness of responses
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul></li></ul>Who You Are :
<ul><li><ul><li>You hold a Bachelor&rsquo;s degree
</li><li>You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Brazilian Portuguese
</li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly articulate nuanced feedback
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>You are adaptable and comfortable moving across topics, domains, and customer requirements
</li><li>You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
</li><li>You have excellent college-level mathematics skills
</li></ul></li></ul>Nice-to-Have Specialties :
<ul><li><ul><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience writing or editing high-quality written content
</li><li>Experience comparing multiple outputs and making fine-grained qualitative judgments
</li><li>Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
</li></ul></li></ul>What Success Looks Like :
<ul><li><ul><li>You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
</li><li>You produce clear, consistent, and reproducible evaluation artifacts
</li><li>Your feedback leads to measurable improvements in response quality and user experience</li></ul></li></ul></div>

Brazilian Portuguese Expert - Remote

<div>Work Mode: Remote 
Engagement Type: Independent Contractor 
Schedule: Full-Time or Part-Time Contract
Role:
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
This project focuses on evaluating and improving general chat behavior in large language models (LLMs). You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations.
What You&rsquo;ll Do
<ul><li>Evaluate LLM-generated responses on their ability to effectively answer user queries
</li><li>Conduct fact-checking using trusted public sources and external tools
</li><li>Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
</li><li>Assess reasoning quality, clarity, tone, and completeness of responses
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a Bachelor&rsquo;s degree
</li><li>You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Italian
</li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly articulate nuanced feedback
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>You are adaptable and comfortable moving across topics, domains, and customer requirements
</li><li>You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
</li><li>You have excellent college-level mathematics skills
</li></ul>Nice-to-Have Specialties
<ul><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience writing or editing high-quality written content
</li><li>Experience comparing multiple outputs and making fine-grained qualitative judgments
</li><li>Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
</li></ul>What Success Looks Like
<ul><li>You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
</li><li>You produce clear, consistent, and reproducible evaluation artifacts
</li><li>Your feedback leads to measurable improvements in response quality and user experience</li></ul></div>

English & Italian Trainer - Remote

<div>Seeking experienced researchers and technical experts to support a frontier-model evaluation project focused on agentic workflows. You will design and validate challenging benchmark tasks in data science, machine learning, finance, and coding to help identify reasoning and problem-solving gaps in advanced STEM models. The role involves building real-world tasks with executable tests and analyzing model or agent behavior.
<h2 class="jobSectionHeader">Key Responsibilities</h2>
<ul><li>Design challenging, real-world STEM problems
</li><li>Implement each task within an agentic development environment using Python
<h2 class="jobSectionHeader">Contract and Payment Terms</h2>
<ul><li>You will be engaged as an independent contractor.</li>
<li>This is a fully remote role that can be completed on your own schedule.</li>
<li>Projects can be extended, shortened, or concluded early depending on needs and performance.</li>
<li>Payments are weekly on Stripe or Wise based on services rendered.</li></ul></li></ul></div>

PhD Rater - Remote

Search 10000+ Model evaluation specialist jobs near you this March. New roles added regularly across leading industries. Apply today.

10000 model evaluation specialist jobs Near Me

Arabic & English Ai Trainer - Remote

Arabic & English Ai Trainer - Remote

Arabic & English Ai Trainer - Remote

Brazilian Portuguese Expert - Remote

English & Italian Trainer - Remote

PhD Rater - Remote

PhD Rater - Remote