conversational ai evaluator

YO IT CONSULTING

<div>Work Mode: Remote 
Engagement Type: Independent Contractor 
Schedule: Full-Time or Part-Time Contract
Role:
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
This project focuses on evaluating and improving general chat behavior in large language models (LLMs). You will assess model-generated responses across diverse topics, provide high-quality human feedback, and help ensure AI systems communicate in ways that are accurate, well-reasoned, and aligned with human expectations.
What You&rsquo;ll Do
<ul><li>Evaluate LLM-generated responses on their ability to effectively answer user queries
</li><li>Conduct fact-checking using trusted public sources and external tools
</li><li>Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuracies
</li><li>Assess reasoning quality, clarity, tone, and completeness of responses
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent annotations by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a Bachelor&rsquo;s degree
</li><li>You are a native speaker or have ILR 5/primary fluency (C2 on the CEFR scale) in Italian
</li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly articulate nuanced feedback
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>You are adaptable and comfortable moving across topics, domains, and customer requirements
</li><li>You have a background or experience in domains requiring structured analytical thinking (e.g., research, policy, analytics, linguistics, engineering)
</li><li>You have excellent college-level mathematics skills
</li></ul>Nice-to-Have Specialties
<ul><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience writing or editing high-quality written content
</li><li>Experience comparing multiple outputs and making fine-grained qualitative judgments
</li><li>Familiarity with evaluation rubrics, benchmarks, or quality scoring systems
</li></ul>What Success Looks Like
<ul><li>You identify factual inaccuracies, reasoning errors, and communication gaps in model responses
</li><li>You produce clear, consistent, and reproducible evaluation artifacts
</li><li>Your feedback leads to measurable improvements in response quality and user experience</li></ul></div>

English & Italian Trainer - Remote

<div>Work Mode: Remote 
Engagement Type: Independent Contractor 
Schedule: Full-Time or Part-Time Contract
Language Requirement: Fluent English
Role :
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
In engineering-related contexts, conversational AI systems must demonstrate accurate applied reasoning, quantitative precision, and practical problem-solving aligned with real-world systems. This project focuses on evaluating and improving how models reason about and explain engineering concepts across multiple disciplines.
What You&rsquo;ll Do
<ul><li>Write and refine prompts to guide model behavior in engineering scenarios
</li><li>Evaluate LLM-generated responses to engineering-related queries for technical accuracy, applied reasoning, and completeness
</li><li>Conduct fact-checking and verify any technical claims using authoritative public sources and domain knowledge
</li><li>Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
</li><li>Assess clarity, structure, and appropriateness of explanations for different audiences
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a PhD in Engineering or a closely related field
</li><li>You have deep expertise in one or more of the following sub-domains:
<ul><li>Mechanical &amp; Physical Systems Engineering
</li><li>Electrical, Electronic &amp; Computer Engineering
</li><li>Chemical, Materials &amp; Process Engineering
</li><li>Civil, Environmental &amp; Infrastructure Engineering
</li></ul></li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly explain complex engineering concepts
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>Experience reviewing or editing technical or academic writing
</li></ul>Nice-to-Have Specialties
<ul><li>Experience with applied research, industry engineering workflows, or systems design
</li><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience teaching, mentoring, or explaining engineering concepts to non-expert audiences
</li><li>Familiarity with evaluation rubrics, benchmarks, or structured review frameworks
</li></ul>What Success Looks Like
<ul><li>You identify technical inaccuracies, flawed assumptions, or incomplete reasoning in engineering-related model outputs
</li><li>Your feedback improves the rigor, clarity, and correctness of AI explanations
</li><li>You deliver consistent, reproducible evaluation artifacts that strengthen model performance</li></ul></div>

Civil Infrastructure Engineer (PhD) - Remote

<div>Engagement Type: Independent Contractor
Work Mode: Remote 
Schedule: (15&ndash;25 hrs/week, flexible up to 40 hrs) 
 Duration: 1&ndash;3 Months | Immediate
<div><div>Location: Geography restricted to the USA, UK, Canada, EU
Type: Full-time or Part-time Contract Work
Fluent Language Skills Required: English
Role
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
In mathematics-related contexts, conversational AI systems must demonstrate precise formal reasoning, mathematical rigor, and conceptual clarity. This project focuses on evaluating and improving how models reason about mathematical problems, explanations, and proofs across both foundational and advanced areas of mathematics.
What You&rsquo;ll Do
<ul><li>Write and refine prompts to guide model behavior in mathematical contexts
</li><li>Evaluate LLM-generated responses to mathematics-related queries for correctness, rigor, and logical coherence
</li><li>Verify mathematical claims, derivations, and proofs using domain expertise
</li><li>Conduct fact-checking using authoritative public sources and domain knowledge
</li><li>Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
</li><li>Assess clarity, structure, and appropriateness of explanations for different audiences
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a PhD in Mathematics or a closely related field
</li><li>You have demonstrated experience in Probability &amp; Statistics, and may also have experience in one or more of the following areas:
<ul><li>Algebra &amp; Number Theory
</li><li>Calculus &amp; Analysis
</li><li>Geometry &amp; Topology
</li><li>Discrete Mathematics, Logic &amp; Computation
</li></ul></li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly explain complex mathematical concepts
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>Experience reviewing or editing technical or academic writing
</li></ul>Nice-to-Have Specialties
<ul><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience teaching, mentoring, or explaining mathematical concepts to non-expert audiences
</li><li>Familiarity with evaluation rubrics, benchmarks, or structured review frameworks
</li></ul>What Success Looks Like
<ul><li>You identify inaccuracies or weak reasoning in mathematical-related model outputs
</li><li>Your feedback improves the rigor, clarity, and correctness of AI explanations
</li><li>You deliver consistent, reproducible evaluation artifacts that strengthen model performance
</li><li>Customers trust their AI systems in mathematical contexts because you&rsquo;ve rigorously evaluated them
</li></ul></div></div><div><div><h2 class="jobSectionHeader">Contract and Payment Terms</h2>
<ul><li>You will be engaged as an independent contractor.</li>
<li>This is a fully remote role that can be completed on your own schedule.</li>
<li>Projects can be extended, shortened, or concluded early depending on needs and performance.</li>
<li>Payments are weekly on Stripe or Wise based on services rendered.</li></ul></div></div></div>

Math Teacher for Ai Research - Remote

Mechanical & Physical Engineer (PhD) - Remote

Computer Science Engineer (PhD) - Remote

Search 7291+ Conversational ai evaluator jobs near you this March. New roles added regularly across leading industries. Apply today.

7291 conversational ai evaluator jobs Near Me

English & Italian Trainer - Remote

Civil Infrastructure Engineer (PhD) - Remote

Math Teacher for Ai Research - Remote

Math Teacher for Ai Research - Remote

Math Teacher for Ai Research - Remote

Mechanical & Physical Engineer (PhD) - Remote

Computer Science Engineer (PhD) - Remote