evaluation specialist

Invisible Agency

<div><h3 class="jobSectionHeader">Project Overview</h3>

We are sourcing independent Search Engine Evaluation Specialists to provide their expertise for an AI benchmark evaluation project. As AI models increasingly interpret search intent, analyze indexing protocols, and evaluate search rank responses, their accuracy relies entirely on robust, expert-crafted training data. The objective of this project is to autonomously produce high-quality evaluation tasks, strong prompts, and clear, well-structured rubrics that generate clean, reliable data for model training.
<h3 class="jobSectionHeader">
Project Deliverables &amp; Scope</h3>

Operate autonomously to design complex evaluation frameworks and provide structured training data. Expected deliverables include:
<ul>
<li>Task &amp; Prompt Creation: Generating realistic, high-quality prompts that compel the AI model to produce complex, professional-grade deliverables specific to search rank response analysis, indexing logic, and query relevance.</li>
<li>Benchmark Evaluation Data Generation: Producing clean, reliable training data that directly aids in the evaluation and refinement of AI models handling complex SEO and search engine ranking tasks.</li>
<li>Quality Assurance &amp; Fact-Checking: Ensuring all generated tasks and scoring criteria reflect strict, real-world search quality rating standards, technical indexing mechanics, and user intent accuracy.</li>
</ul><h3 class="jobSectionHeader">Required Expertise</h3>

To successfully fulfill the deliverables of this project, Contractors must possess deep industry knowledge to craft realistic professional scenarios.

Core skillset includes:
<ul>
<li>Demonstrable professional expertise within the Search Engine Optimization (SEO), search quality rating, or digital marketing sectors, with a deep understanding of indexing, ranking factors, and search response evaluation.</li>
<li>Strong writing and prompt generation skills, with the ability to design highly realistic, complex search evaluation task scenarios for AI training.</li>
<li>Proficiency in rubric generation, specifically the ability to create objective, non-ambiguous scoring criteria that leave no room for subjective interpretation.</li>
<li>A meticulous, detail-oriented approach to fact-checking search rankings, evaluating content quality against official rating guidelines, and generating reliable data for system benchmarking.</li>
</ul>
We offer a pay rate of $20 per hour, with the exact rate determined after evaluating your experience, expertise, and geographic location. Final offer amounts may vary from the pay range listed above. As a contractor you'll supply a secure computer and high‑speed internet; company‑sponsored benefits such as health insurance and PTO do not apply.

Engagement Type: Freelance / Independent Contractor

Workplace Type: Remote</div>

Search Engine Evaluation Specialist - Freelance AI Trainer Project

YO IT CONSULTING

<div>Job Title: AI Evaluation Specialist
Job Type: Contractor
Location: Remote
Job Summary:
As a AI Evaluation Specialist, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required , your domain knowledge is what matters.
Key Responsibilities:
<div><ul><li>Review and critically assess AI-generated outputs for quality, clarity, usability, and overall user experience.</li></ul>
<ul><li>Identify inconsistencies, weaknesses, and improvement opportunities across diverse content types and visual experiences.</li></ul>
<ul><li>Apply structured evaluation guidelines and provide insightful, nuanced feedback to inform model improvements.</li></ul>
<ul><li>Collaborate with cross-functional teams to define and refine quality standards, scoring criteria, and review processes.</li></ul>
<ul><li>Maintain high accuracy and consistency across evaluations, ensuring reliable and actionable results.</li></ul>
<ul><li>Contribute to the continuous enhancement of evaluation workflows and best practices.</li></ul>
<ul><li>Support the development of training data that enhances AI system performance and reliability.</li></ul></div> 
Required Skills and Qualifications:
<div><ul><li>Exceptional reading comprehension and acute attention to detail.</li></ul>
<ul><li>Strong observational skills with excellent aesthetic or editorial judgment.</li></ul>
<ul><li>Ability to think critically and deliver consistent, high-quality evaluations in ambiguous scenarios.</li></ul>
<ul><li>Excellent written and verbal communication skills.</li></ul>
<ul><li>Comfort working across various content types and evaluation guidelines.</li></ul>
<ul><li>Strong sense of ownership, reliability, and commitment to delivering high-quality work for the customer's team.</li></ul>
<ul><li>Open-minded approach to learning and quickly adapting to new workflows.</li></ul>
</div>Preferred Qualifications:
<div><ul><li>Background in design, UX/UI, creative direction, editing, or content strategy.</li></ul>
<ul><li>Experience reviewing creative, visual, or AI-generated content.</li></ul>
<ul><li>Familiarity with annotation, QA, moderation, or evaluation workflows, as well as working with AI tools or LLMs.</li></ul>
</div>We&rsquo;re seeking highly perceptive and detail-oriented trainers passionate about shaping the future of AI. Bring your refined judgment, critical thinking skills, and commitment to excellence to our customer's team and make a meaningful impact on emerging AI technologies.</div>

Ai Evaluation Expert - Remote

YO IT Consulting

<div>
 
 Job Title:
 
 AI Evaluation Specialist
 
 
 
 Job Type:
 
 Contractor
 
 
 
 Location:
 
 Remote
 
 
 
 Job Summary
 
 
 
 As a AI Evaluation Specialist, you'll apply your expertise to help train next-generation AI systems. Your work will shape how models learn, reason, and perform through high-quality, real-world input. No prior experience in AI is required &mdash; your domain knowledge is what matters.
 
 
 
 Key Responsibilities
 
 
 
 <ul>
 <li>
 Review and critically assess AI-generated outputs for quality, clarity, usability, and overall user experience.
 </li>
 <li>
 Identify inconsistencies, weaknesses, and improvement opportunities across diverse content types and visual experiences.
 </li>
 <li>
 Apply structured evaluation guidelines and provide insightful, nuanced feedback to inform model improvements.
 </li>
 <li>
 Collaborate with cross-functional teams to define and refine quality standards, scoring criteria, and review processes.
 </li>
 <li>
 Maintain high accuracy and consistency across evaluations, ensuring reliable and actionable results.
 </li>
 <li>
 Contribute to the continuous enhancement of evaluation workflows and best practices.
 </li>
 <li>
 Support the development of training data that enhances AI system performance and reliability.
 
 
 </li>
 </ul>
 
 Required Skills And Qualifications
 
 
 
 <ul>
 <li>
 Exceptional reading comprehension and acute attention to detail.
 </li>
 <li>
 Strong observational skills with excellent aesthetic or editorial judgment.
 </li>
 <li>
 Ability to think critically and deliver consistent, high-quality evaluations in ambiguous scenarios.
 </li>
 <li>
 Excellent written and verbal communication skills.
 </li>
 <li>
 Comfort working across various content types and evaluation guidelines.
 </li>
 <li>
 Strong sense of ownership, reliability, and commitment to delivering high-quality work for the customer's team.
 </li>
 <li>
 Open-minded approach to learning and quickly adapting to new workflows.
 
 
 </li>
 </ul>
 
 Preferred Qualifications
 
 
 
 <ul>
 <li>
 Background in design, UX/UI, creative direction, editing, or content strategy.
 </li>
 <li>
 Experience reviewing creative, visual, or AI-generated content.
 </li>
 <li>
 Familiarity with annotation, QA, moderation, or evaluation workflows, as well as working with AI tools or LLMs.
 
 
 </li>
 </ul>
 We&rsquo;re seeking highly perceptive and detail-oriented trainers passionate about shaping the future of AI. Bring your refined judgment, critical thinking skills, and commitment to excellence to our customer's team and make a meaningful impact on emerging AI technologies.
</div>

Ai Trainer/Data Annotator - Remote

Ai Trainer - Remote

Cleveland Museum of Art

<div>
 Job Details
 
 
 
 Description
 
 
 
 The Audience Impact Specialist plays a key role in fostering an audience-centered and data-informed approach across the museum. The Audience Impact Specialist supports the Assistant Director of Audience Research and Evaluation with the planning, execution, and analysis of audience research and evaluation studies and with the gathering, visualization, and reporting on audience, attendance, and marketing data in reliable, insight, approachable, and actionable ways. This role also is responsible for using project management software to organize, oversee, and communicate timelines, tasks, deliverables, and budget alignment for evaluation studies and marketing campaigns; this position therefore works closely with the entire Communications and External Relations division.
 
 
 
 Requirements And Key Competencies
 
 
 
 Bachelor&rsquo;s Degree, 3-5 years related experience and/or training, or equivalent combination of education/training and experience.
 
 
 Excellent writing skills and computer proficiency, especially in Microsoft Word, Excel, and PowerPoint. Previous experience with word-processing software, spreadsheet software, and computer-database management preferred. Previous experience with project management software preferred. Data analysis experience in SPSS, R, or similar software preferred.
 
 
 Prior experience in evaluation, research, visitor studies, or market research, preferably for an informal learning institution such as a nonprofit, museum, library, cultural organization, or foundation.
 
 
 Demonstrated experience contributing to evaluation projects, including knowledge of qualitative and quantitative evaluation methods, analysis and coding techniques, data visualization, and sharing results clearly and effectively for a variety of stakeholders.
 
 
 Must be professional and have excellent interpersonal skills and the ability to interact diplomatically with staff members and other customers. Must be able to work creatively, effectively, and independently on multiple projects simultaneously, and to be detail-oriented in a fast-paced, team environment.
 
 
 Must possess excellent organizational ability and time management skills to establish timetables, meet strict and sometimes concurrent deadlines, multi-task and prioritize workloads. Ability to work with a wide variety of people in a variety of social settings required.
 
 
 
 Key Responsibilities
 
 
 
 Audience Research and Evaluation
 
 
 <ul>
 <li>
 Assists the Assistant Director of Audience Research and Evaluation in all stages of evaluation studies, including design of logic models, articulation of outcomes and research questions, evaluation planning, creation of data collection instruments, data entry and cleaning, data analysis and coding, data visualization, and reporting.
 </li>
 <li>
 Foster an audience-centered, data-informed approach by developing and delivering audience research projects that are reliable, insightful, approachable, and accessible.
 
 
 </li>
 </ul>
 Audience, Attendance, and Marketing Reporting
 
 
 <ul>
 <li>
 Alongside the Assistant Director of Audience Research and Evaluation, gathers, visualizes, and reports evaluation insights, audience metrics, attendance data, and marketing performance metrics such as earned media and social media impressions for grants, institutional dashboards, presentations, and other needs.
 
 
 </li>
 </ul>
 Project Management and Marketing Support
 
 
 <ul>
 <li>
 Organize and oversee evaluation projects and marketing campaigns in project management software to create and manage a project plan for the tasks, timelines, and resources required. Communicate with the project team to monitor progress to achieve timeline and budget goals.
 </li>
 <li>
 Maintain and communicate processes for tracking progress against milestones, deadlines and budget scope of marketing campaigns and evaluation projects.
 </li>
 <li>
 Attend Marketing events such as exhibition media previews as required to assist the Chief Marketing Officer, Director of Communications, and Communications Manager in coordination and implementation, occasionally outside of regular business hours.
 </li>
 <li>
 Attend meetings and participate on committees/workgroups as required.
 </li>
 <li>
 Performs special projects and other duties as assigned.
 
 
 </li>
 </ul>
 
 Other Responsibilities
 
 
 
 <ul>
 <li>
 Upholds the professional standards of his/her field and always acts in a manner that is consistent with the best interests of the museum and will protect and enhance its reputation and standing within the community of museums.
 </li>
 <li>
 Shows respect for co-workers and visitors and an understanding of and appreciation for the diversity of the museum&rsquo;s staff, volunteers, and audiences.
 </li>
 <li>
 Aspires to excellence in all aspects of his/her work and serves as a model for others.
 </li>
 <li>
 Maintains confidentiality.
 </li>
 <li>
 Suggests methods for improvement for job responsibilities and professional growth.
 </li>
 <li>
 Adheres to all CMA protocols, procedures, rules, and policies.
 
 
 </li>
 </ul>
 
 Application Materials:
 
 Please submit a cover letter and resume/CV. References may be requested of top candidates after an interview.
 
 
 
 Salary range: $45,000-$48,000
 
 
 
 
 Full-time Benefits Include
 
 
 
 <ul>
 <li>
 Partner level membership to CMA
 </li>
 <ul>
 <li>
 Free, unlimited admission to select Cleveland Museum of Art ticketed exhibitions (two adult member tickets per visit, subject to availability)
 </li>
 <li>
 50% off admission to select ticketed exhibitions for members' guests
 </li>
 <li>
 Free admission to select ticketed exhibitions for unlimited children, 17 and younger, when accompanied by a member
 </li>
 <li>
 Priority registration and discounts for museum art classes for adults and children
 </li>
 <li>
 20% discount in the museum store
 </li>
 <li>
 10% discount in the museum restaurant and caf&eacute;
 </li>
 <li>
 Annual subscription to Cleveland Art members magazine
 </li>
 </ul>
 <li>
 Free Garage Parking
 </li>
 <li>
 Your employment relationship with the museum qualifies you for free or discounted admissions to other cultural institutions such as the Natural History Museum, Botanical Gardens, The Cleveland Zoo, etc.
 </li>
 <li>
 Medical
 </li>
 <li>
 Dental
 </li>
 <li>
 Vision
 </li>
 <li>
 Life and Accidental Death and Dismemberment Insurance
 </li>
 <li>
 Voluntary Life
 </li>
 <li>
 Short Term Disability
 </li>
 <li>
 Long Term Disability
 </li>
 <li>
 HSA
 </li>
 </ul>
</div>

Audience Impact Specialist

Search 7034+ Evaluation specialist jobs near you this May. New roles added regularly across leading industries. Apply today.

7034 evaluation specialist jobs Near Me

Search Engine Evaluation Specialist - Freelance AI Trainer Project

Ai Evaluation Expert - Remote

Ai Evaluation Expert - Remote

Ai Trainer/Data Annotator - Remote

Ai Trainer/Data Annotator - Remote

Ai Trainer - Remote

Audience Impact Specialist