ai evaluation specialist

YO IT CONSULTING

<div>Work Mode: Remote 
Engagement Type: Independent Contractor 
Schedule: Full-Time or Part-Time Contract
Language Requirement: Fluent English
Role :
Partners with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems. These systems are used across a wide range of everyday and professional scenarios, and their effectiveness depends on how clearly, accurately, and helpfully they respond to real user questions.
In engineering-related contexts, conversational AI systems must demonstrate accurate applied reasoning, quantitative precision, and practical problem-solving aligned with real-world systems. This project focuses on evaluating and improving how models reason about and explain engineering concepts across multiple disciplines.
What You&rsquo;ll Do
<ul><li>Write and refine prompts to guide model behavior in engineering scenarios
</li><li>Evaluate LLM-generated responses to engineering-related queries for technical accuracy, applied reasoning, and completeness
</li><li>Conduct fact-checking and verify any technical claims using authoritative public sources and domain knowledge
</li><li>Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
</li><li>Assess clarity, structure, and appropriateness of explanations for different audiences
</li><li>Ensure model responses align with expected conversational behavior and system guidelines
</li><li>Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines
</li></ul>Who You Are
<ul><li>You hold a PhD in Engineering or a closely related field
</li><li>You have deep expertise in one or more of the following sub-domains:
<ul><li>Mechanical &amp; Physical Systems Engineering
</li><li>Electrical, Electronic &amp; Computer Engineering
</li><li>Chemical, Materials &amp; Process Engineering
</li><li>Civil, Environmental &amp; Infrastructure Engineering
</li></ul></li><li>You have significant experience using large language models (LLMs) and understand how and why people use them
</li><li>You have excellent writing skills and can clearly explain complex engineering concepts
</li><li>You have strong attention to detail and consistently notice subtle issues others may overlook
</li><li>Experience reviewing or editing technical or academic writing
</li></ul>Nice-to-Have Specialties
<ul><li>Experience with applied research, industry engineering workflows, or systems design
</li><li>Prior experience with RLHF, model evaluation, or data annotation work
</li><li>Experience teaching, mentoring, or explaining engineering concepts to non-expert audiences
</li><li>Familiarity with evaluation rubrics, benchmarks, or structured review frameworks
</li></ul>What Success Looks Like
<ul><li>You identify technical inaccuracies, flawed assumptions, or incomplete reasoning in engineering-related model outputs
</li><li>Your feedback improves the rigor, clarity, and correctness of AI explanations
</li><li>You deliver consistent, reproducible evaluation artifacts that strengthen model performance</li></ul></div>

Civil Infrastructure Engineer (PhD) - Remote

Electrical Electronic Engineer (PhD) - Remote

Mechanical & Physical Engineer (PhD) - Remote

Search 10000+ Ai evaluation specialist jobs near you this March. New roles added regularly across leading industries. Apply today.

10000 ai evaluation specialist jobs Near Me

Civil Infrastructure Engineer (PhD) - Remote

Electrical Electronic Engineer (PhD) - Remote

Mechanical & Physical Engineer (PhD) - Remote

Electrical Electronic Engineer (PhD) - Remote

Civil Infrastructure Engineer (PhD) - Remote

Mechanical & Physical Engineer (PhD) - Remote

Electrical Electronic Engineer (PhD) - Remote