General Multilingual Rubrics – Data Annotator / AI Trainer

Remote | Freelance / Independent Contractor | Flexible Hours Company: Agents Only LLCAbout Agents Only LLC

LLC is a global AI solutions company supporting the development of advanced artificial intelligence systems through high-quality data annotation, localization, evaluation, and model training services. We work with multilingual talent across multiple regions to improve how AI systems understand language, reasoning, cultural nuance, and domain-specific knowledge.

We are currently expanding our General Multilingual Rubrics program and are looking for experienced Data Annotators / AI Trainers with strong analytical and language skills. We are specifically seeking candidates with academic or professional backgrounds in Law, Legal Studies, Political Science, International Relations, Public Policy, Governance, or related fields.

Position Overview

As a General Multilingual Rubrics – Data Annotator / AI Trainer, you will help train and evaluate AI systems by reviewing multilingual content, assessing reasoning quality, validating factual consistency, and applying structured evaluation rubrics.

This role requires strong critical thinking, reading comprehension, research capability, and attention to linguistic and contextual accuracy. Candidates with legal or political science expertise are highly preferred due to the complexity of policy, governance, legal reasoning, and structured argumentation tasks involved in the project.

Key Responsibilities

• Review, evaluate, and annotate multilingual AI-generated responses
• Apply detailed rubrics and scoring guidelines consistently
• Assess content for:
• Accuracy
• Logical reasoning
• Localization quality
• Tone and clarity
• Policy and factual consistency
• Perform comparative ranking and evaluation tasks
• Conduct light research and verification when necessary
• Identify hallucinations, misinformation, bias, or unsupported claims
• Provide structured feedback to improve AI model performance
• Work on tasks related to:
• Legal reasoning
• Public policy
• Governance
• Current affairs
• Multilingual localization
• General knowledge evaluation

Preferred Qualifications

Required

• Fluent or native-level proficiency in at least one target language
• Strong English comprehension and written communication skills
• Excellent analytical and critical thinking ability
• Ability to follow complex instructions and annotation guidelines
• Reliable internet connection and computer/laptop access

Strongly Preferred

• Background in:
• Law
• Legal Studies
• Political Science
• International Relations
• Public Administration
• Public Policy
• Governance
• Humanities or Social Sciences
• Experience in:
• Data annotation
• AI training
• Content evaluation
• Research
• Academic writing
• Legal analysis
• Policy analysis
• Localization projects
• Familiarity with Large Language Models (LLMs), AI evaluation, or prompt-response assessment is an advantage

Ideal Candidate Profile

We are looking for annotators who can:

• Analyze nuanced or sensitive content objectively
• Understand structured argumentation and policy language
• Detect inconsistencies and reasoning gaps
• Evaluate multilingual responses with cultural awareness
• Maintain high attention to detail while working independently

Candidates with legal or political science expertise are especially valuable for tasks involving:

• Regulatory interpretation
• Argument evaluation
• Policy alignment
• Civic and governance-related content
• Fact-sensitive assessments

Compensation

• Payment structure: Per task / project-based
• Competitive rates based on task complexity, language, and expertise level
• Flexible workload depending on project availability and performance

Work Setup

• Fully remote
• Flexible schedule
• Freelance / Independent Contractor arrangement
• Global applicants are welcome

Application Process

If you have a strong background in language, legal reasoning, policy analysis, or multilingual evaluation and want to contribute to the future of AI systems, we encourage you to apply.

Qualified candidates may be asked to:

• Complete a language or annotation assessment
• Attend a short onboarding or calibration session
• Sign project documentation prior to production work

موقع العمل: عن بُعد