ai model evaluator

YO IT Consulting

<div>
 
 Work Mode:
 
 Remote
 
 
 
 Engagement Type:
 
 Independent Contractor
 
 
 
 Schedule:
 
 Full-Time or Part-Time Contract
 
 
 
 Language Requirement:
 
 Fluent English
 
 
 
 Role Overview
 
 
 
 We partner with leading AI teams to improve the quality, usefulness, and reliability of general-purpose conversational AI systems.
 
 
 This project focuses specifically on evaluating and improving how AI systems reason about code, generate programming solutions, and explain technical concepts across various complexity levels.
 
 
 The role involves rigorous technical evaluation of AI-generated responses in coding and software engineering contexts.
 
 
 
 What You&rsquo;ll Do
 
 
 
 Evaluate LLM-generated responses to coding and software engineering queries for accuracy, reasoning, clarity, and completeness
 
 
 Conduct fact-checking using trusted public sources and authoritative references
 
 
 Conduct accuracy testing by executing code and validating outputs using appropriate tools
 
 
 Annotate model responses by identifying strengths, areas of improvement, and factual or conceptual inaccuracies
 
 
 Assess code quality, readability, algorithmic soundness, and explanation quality
 
 
 Ensure model responses align with expected conversational behavior and system guidelines
 
 
 Apply consistent evaluation standards by following clear taxonomies, benchmarks, and detailed evaluation guidelines
 
 
 
 Who You Are
 
 
 
 You hold a BS, MS, or PhD in Computer Science or a closely related field
 
 
 You have significant real-world experience in software engineering or related technical roles
 
 
 You are an expert in at least one relevant programming language (e.g., Python, Java, C++, JavaScript, Go, Rust)
 
 
 You are able to solve HackerRank or LeetCode Medium and Hard-level problems independently
 
 
 You have experience contributing to well-known open-source projects, including merged pull requests
 
 
 You have significant experience using LLMs while coding and understand their strengths and failure modes
 
 
 You have strong attention to detail and are comfortable evaluating complex technical reasoning, identifying subtle bugs or logical flaws
 
 
 
 Nice-to-Have Specialties
 
 
 
 Prior experience with RLHF, model evaluation, or data annotation work
 
 
 Track record in competitive programming
 
 
 Experience reviewing code in production environments
 
 
 Familiarity with multiple programming paradigms or ecosystems
 
 
 Experience explaining complex technical concepts to non-expert audiences
 
 
 
 What Success Looks Like
 
 
 
 You identify incorrect logic, inefficiencies, edge cases, or misleading explanations in model-generated code, technical concepts, and system design discussions
 
 
 Your feedback improves the correctness, robustness, and clarity of AI coding outputs
 
 
 You deliver reproducible evaluation artifacts that strengthen model performance
</div>

Python Developer - Ai Training

DataAnnotation

<div>
 We are looking for a
 
 Pharmacy Technician
 
 to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 In this role, you will need to be an expert in healthcare. We are interested in a wide range of expertise, so relevant backgrounds include Revenue Cycle Management &amp; Patient Finance, Pharmacy Operations (Pharm Tech), Laboratory Services and Operations (Lab Tech), and Clinical Support &amp; Ambulatory Operations.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 Full-time or part-time remote position
 </li>
 <li>
 Choose which projects you want to work on
 </li>
 <li>
 Flexible schedule
 </li>
 <li>
 Projects are paid hourly starting at $20+ per hour
 </li>
 <li>
 Bonuses available for high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Provide AI chatbots with diverse and complex healthcare-related problems
 </li>
 <li>
 Evaluate AI outputs for logic, accuracy, and performance
 </li>
 <li>
 Ensure the medical accuracy and overall quality of model responses
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 A current or in-progress medical or healthcare-related degree
 
 
 </li>
 </ul>
 
 Notes:
 
 Payment is made via PayPal. We will never ask you for any money. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contractor position.
 
 
 
 #generalist
 
</div>

Pharmacy Technician

SME Careers

<div>
 As a remote, hourly paid R Engineer, you will review AI-generated responses and generate high-quality R and data-analysis-focused content, evaluating the reasoning quality and step-by-step problem-solving behind each solution. You will assess solutions for accuracy, clarity, and adherence to the prompt; identify errors in statistical methodology or data-wrangling workflows; fact-check analytical results; write expert-level explanations and model solutions that demonstrate correct use of R; and rate and compare multiple AI responses based on correctness and reasoning quality. This contract role is with SME Careers, a fast-growing AI Data Services company and subsidiary of SuperAnnotate that provides AI training data for many of the world&rsquo;s largest AI companies and foundation model labs, directly helping to improve the world&rsquo;s premier AI models.
 
 
 Key Responsibilities
 
 
 <ul>
 <li>
 Develop AI Training Content: Create detailed prompts in various topics and responses to guide AI learning, ensuring the models reflect a comprehensive understanding of diverse subjects.
 </li>
 <li>
 Optimize AI Performance: Evaluate and rank AI responses to enhance the model's accuracy, fluency, and contextual relevance.
 </li>
 <li>
 Ensure Model Integrity: Test AI models for potential inaccuracies or biases, validating their reliability across use cases.
 
 </li>
 </ul>
 
 Your Profile
 
 
 <ul>
 <li>
 2+ years of hands-on experience using R for data analysis, statistics, or data science work.
 </li>
 <li>
 Strong proficiency in R programming, including data wrangling, functional programming patterns, and writing reusable functions or packages.
 </li>
 <li>
 Solid grounding in applied statistics, including regression, inference, and model validation, with practical experience implementing these methods in R.
 </li>
 <li>
 Experience building end-to-end analyses in R that include data cleaning, exploratory analysis, modeling, and visualization.
 </li>
 <li>
 Familiarity with common R ecosystems such as tidyverse, data.table, and ggplot2, and the ability to choose appropriate tools for a given task.
 </li>
 <li>
 Professional experience in a data-focused role such as data scientist, statistician, quantitative analyst, or similar.
 </li>
 <li>
 Minimum Bachelor&rsquo;s degree in Statistics, Mathematics, Computer Science, or a closely related quantitative field.
 </li>
 <li>
 Significant experience using large language models (LLMs) to assist with coding, analysis design, and code review in R.
 </li>
 <li>
 Excellent English writing skills with the ability to document analyses and explain complex statistical ideas clearly to non-experts.
 </li>
 <li>
 Previous experience with AI data training or model evaluation is strongly preferred, and Minimum C1 English proficiency is required.
 </li>
 </ul>
</div>

R Engineer

<div>
 We are looking for a Math Teacher to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 In this role you will need to hold an expert level of mathematical reasoning. Other related fields include, but are not limited to: Applied Mathematics and/or Computer Science.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex mathematics problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 Detail-oriented
 </li>
 <li>
 Proficient in arithmetic, algebra, geometry, calculus, probability, statistics, and inductive/deductive reasoning
 </li>
 <li>
 A current, in progress, or completed Master's and/or PhD is preferred but not required
 
 
 </li>
 </ul>
 
 Notes
 
 
 
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 
 
 #math
</div>

Math Teacher

<div>
 We are looking for an Entertainment Lawyer to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model. In this role you will need to hold an expert level of legal reasoning &mdash; a completed or in-progress law degree is required.
 
 
 We are interested in a wide range of expertise, so relevant backgrounds include: corporate law, criminal law, civil litigation, constitutional law, administrative law, environmental law, family law, intellectual property law, tax law, labor and employment law, immigration law, real estate law, health law, entertainment law, human rights law, international law, cybersecurity and privacy law, contract law, securities law, bankruptcy law, and alternative dispute resolution.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at $50-$60 USD per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex legal problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 Must have at least 5+ years of law experience
 </li>
 <li>
 Must be a law school graduate
 
 
 </li>
 </ul>
 
 Note
 
 
 
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 
 
 #law
</div>

Entertainment Lawyer

Frontend JavaScript Developer- Ai Training

Search 10000+ Ai model evaluator jobs near you this March. New roles added regularly across leading industries. Apply today.

10000 ai model evaluator jobs Near Me

Python Developer - Ai Training

Pharmacy Technician

R Engineer

Math Teacher

Entertainment Lawyer

Frontend JavaScript Developer- Ai Training

Math Teacher