agentic ai engineer

ai chatbot evaluator

DataAnnotation

<div>
 We are looking for a Data Scientist to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 <br/>
 <br/>
 In this role you will need to hold an expert understanding of data analysis. Other related fields include, but are not limited to: Applied skills in Statistics and/or Computer Science.
 <br/>
 <br/>
 <strong>
  Benefits
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   This is a full-time or part-time REMOTE position
  </li>
  <li>
   You&rsquo;ll be able to choose which projects you want to work on
  </li>
  <li>
   You can work on your own schedule
  </li>
  <li>
   Projects are paid hourly starting at $40+ USD per hour, with bonuses on high-quality and high-volume work
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Give AI chatbots writing and editing tasks and evaluate their outputs
  </li>
  <li>
   Evaluate the quality produced by AI models for correctness and performance
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Fluency in English (native or bilingual level)
  </li>
  <li>
   Detail-oriented
  </li>
  <li>
   Demonstrated proficiency in related technical, statistical, and mathematical skills
  </li>
  <li>
   A current, in progress, or completed Masters and/or PhD is preferred but not required
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Notes
  <br/>
  <br/>
 </strong>
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 <br/>
 <br/>
 #math
</div>


Data Scientist

<div>
 We are looking for an Independent Contractor to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 <br/>
 <br/>
 In this role you will need to hold an expert level of expertise in writing, editing, and language use. Other related fields include, but are not limited to: applied skills in research and fact-checking.
 <br/>
 <br/>
 <strong>
  Benefits
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   This is a full-time or part-time REMOTE position
  </li>
  <li>
   You&rsquo;ll be able to choose which projects you want to work on
  </li>
  <li>
   You can work on your own schedule
  </li>
  <li>
   Projects are paid hourly starting at $25-$30+/hr, with bonuses on high-quality and high-volume work
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Give AI chatbots tasks and evaluate their outputs
  </li>
  <li>
   Evaluate the quality produced by AI models for correctness and performance
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Fluency in English (native or bilingual level)
  </li>
  <li>
   Detail-oriented
  </li>
  <li>
   Excellent command of grammar, style, and brand voice
  </li>
  <li>
   A current, in progress, or completed Bachelor&rsquo;s degree is preferred but not required
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Notes
  <br/>
  <br/>
 </strong>
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 <br/>
 <br/>
 #generalist
</div>


Independent Contractor

<div>
 We are looking for an AI Content Writing Specialist to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 <br/>
 <br/>
 In this role you will need to hold an expert level of writing, editing, and content development. Other related fields include, but are not limited to: Communications and/or Journalism.
 <br/>
 <br/>
 <strong>
  Benefits
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   This is a full-time or part-time REMOTE position
  </li>
  <li>
   You&rsquo;ll be able to choose which projects you want to work on
  </li>
  <li>
   You can work on your own schedule
  </li>
  <li>
   Projects are paid hourly starting at $25-$30+/hr, with bonuses on high-quality and high-volume work
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Give AI chatbots writing and editing tasks and evaluate their outputs
  </li>
  <li>
   Evaluate the quality produced by AI models for correctness and performance
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Fluency in English (native or bilingual level)
  </li>
  <li>
   Detail-oriented
  </li>
  <li>
   Excellent command of grammar, style, and brand voice
  </li>
  <li>
   A current, in progress, or completed Bachelor's is preferred but not required
  </li>
  <li>
   Detail-oriented Excellent command of grammar, style, and brand voice.
  </li>
  <li>
   A current, in progress, or completed Bachelor's is preferred but not required.
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Notes
  <br/>
  <br/>
 </strong>
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 <br/>
 <br/>
 #generalist
</div>


AI Content Writing Specialist

<div>
 We are looking for a Social Media Content Manager to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 <br/>
 <br/>
 In this role you will need to hold an expert level of expertise in writing, editing, and language use. Other related fields include, but are not limited to: applied skills in research and fact-checking.
 <br/>
 <br/>
 <strong>
  Benefits
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   This is a full-time or part-time REMOTE position
  </li>
  <li>
   You&rsquo;ll be able to choose which projects you want to work on
  </li>
  <li>
   You can work on your own schedule
  </li>
  <li>
   Projects are paid hourly starting at $25-$30+/hr, with bonuses on high-quality and high-volume work
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Responsibilities
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Give AI chatbots complex tasks and evaluate their outputs
  </li>
  <li>
   Evaluate the quality produced by AI models for correctness and performance
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Qualifications
  <br/>
  <br/>
 </strong>
 <ul>
  <li>
   Fluency in English (native or bilingual level)
  </li>
  <li>
   Detail-oriented
  </li>
  <li>
   Excellent command of grammar, style, and brand voice
  </li>
  <li>
   A current, in progress, or completed Bachelor&rsquo;s degree is preferred but not required
   <br/>
   <br/>
  </li>
 </ul>
 <strong>
  Notes
  <br/>
  <br/>
 </strong>
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 <br/>
 <br/>
 #generalist
</div>


Social Media Content Manager

Autodesk

<div><div><div><div><div><div><div><div><div><div><div><div><div><div><div><p><b>Job Requisition ID #</b></p></div></div></div></div></div></div></div></div></div></div></div></div></div></div><div></div>
26WD97432<p></p><p><b>
Position Overview</b></p><p>
At Autodesk, we do what no other company can: we help our customers design and make anything. The Experience Foundations team at Autodesk plays a critical role in designing the experiences that make that mission a reality, especially in this transformative moment where seamless digital experiences and AI-powered innovation will empower customers and teams to achieve meaningful outcomes faster.</p><p></p><p>
Within the Experience Foundations team for AEC (Architecture, Engineering, and Construction), we are seeking a Senior Manager of Content Strategy to lead content strategy for our Design Solutions team. Design Solutions consists of several pivotal products and platforms for Autodesk including AutoCAD, Revit, Civil 3D, and Forma. Reporting to the Senior Director of Content Strategy, this role will lead and grow a team of content strategists to strategically shape best-in-class product design and deliver high-quality user experiences that are intuitive, personalized, contextual, and spark joy. They'll also create and foster a tight-knit team culture marked by close collaboration and shared approaches, craft development, innovation and experimentation, and empowerment and AI-enablement. They'll partner closely with Design, Product, Engineering, Research, and Operations, and other cross-functional partners, to ensure content strategy is engaging on the top priorities in ways that meet top customer needs and drive business outcomes.</p><p></p><p>
This moment requires us to work in new ways&mdash;using AI internally to improve how we operate, make decisions, and deliver experiences, while also building trusted AI capabilities that help our customers automate repetitive work, accelerate creativity, and unlock smarter, more connected workflows. To deliver on our vision, we need exceptional contributors and leaders who are curious, adaptable, customer-focused, and excited to help shape the future of work at Autodesk.</p><p></p><p><b>
Responsibilities</b></p><ul><li><p>
Manage a team of full-time content strategists and contractors to set and execute the content strategy across the Design Solutions portfolio ensuring strategic impact on the top priorities and driving coherence across interconnected journeys</p></li><li><p>
Lead, coach, inspire, and grow the Design Solutions Content Strategy team, fostering a tight-knit culture, high standards, innovation, experimentation, career development, and strong collaboration</p></li><li><p>
As part of the Senior Director of Content Strategy's leadership team, shape and support the AEC Content Strategy team by contributing to strategic initiatives, nurturing team culture, fostering learning opportunities, and mentoring members</p></li><li><p>
Oversee the delivery and management of clear, concise user interface text, in-product messaging, and product documentation and lead by example on key projects</p></li><li><p>
Guide the design of learning and onboarding experiences that are intuitive, approachable, personalized, contextual, and aligned with Autodesk&rsquo;s AI-forward product direction</p></li><li><p>
Define a clear vision for how content meets user needs, supports their business goals, reduces friction, and improves collaboration, comprehension, confidence, and task success</p></li><li><p>
Build and evolve scalable frameworks, standards, evaluation methods, and governance for UX content, terminology, voice, tone, and content quality</p></li><li><p>
Partner closely with Design, Product, Engineering, Research, Analytics, and Operations to ensure content is integrated early and effectively into the product development process on top priorities</p></li><li><p>
Develop scaling strategies including empowering partners, evolving approaches to support documentation, and leveraging AI</p></li><li><p>
Use customer insights, research findings, data, and business context to prioritize content opportunities and inform strategic decisions</p></li><li><p>
Drive alignment with senior stakeholders by translating ambiguous needs into clear plans, priorities, and measurable outcomes</p></li><li><p>
Champion accessibility, inclusivity, and plain-language principles across all content experiences</p></li><li><p>
Continuously explore and improve tools, methods, and AI-enabled ways of working that strengthen content quality, speed, innovation, and impact at scale</p></li></ul><p></p><p><b>
Minimum Qualifications</b></p><ul><li><p>
12+ years of experience in content strategy, UX writing, content design, conversation design, or related experience design disciplines</p></li><li><p>
5+ years of people management experience leading and developing high-performing content strategy teams</p></li><li><p>
Demonstrated experience designing or leading agentic and conversational experiences, including AI-assisted, chatbot, assistant, or guided interaction models</p></li><li><p>
Proficiency with AI tools and workflows, and sound judgment in applying them to content creation, iteration, and governance</p></li><li><p>
Proven experience shaping content strategy for complex product ecosystems, digital experiences, or enterprise software environments</p></li><li><p>
Strong expertise in content design principles, information clarity, content systems, and customer-centered writing for product experiences</p></li><li><p>
Track record of influencing cross-functional partners and driving alignment across Design, Product, Engineering, and Research</p></li><li><p>
Strong systems thinking and the ability to connect content decisions across end-to-end journeys, platforms, and customer touchpoints</p></li><li><p>
Excellent communication and storytelling skills, with the ability to present strategy, rationale, and recommendations to senior stakeholders</p></li><li><p>
Experience using research, metrics, and qualitative insights to evaluate content effectiveness and improve customer outcomes</p></li><li><p>
Commitment to accessibility, inclusive language, and building trusted customer experiences</p></li><li><p>
Bachelor&rsquo;s degree in English, Communications, Human-Computer Interaction, Design, Journalism, or a related field, or equivalent practical experience</p></li></ul><p></p><p><b>
Preferred Qualifications</b></p><ul><li><p>
Experience in the AEC industry</p></li><li><p>
Passion for applying product design, AI, and content strategy to empower designers, makers, and builders</p></li></ul><p></p><div><div><div><div><div><div><div><p><b>
Learn More</b></p></div></div></div></div></div></div></div><p></p><p><b>
About Autodesk</b></p><p>
Welcome to Autodesk! Amazing things are created every day with our software &ndash; from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.</p><p></p><p>
We take great pride in our culture here at Autodesk &ndash; it&rsquo;s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.</p><p></p><p>
When you&rsquo;re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!</p><p></p><p><b>
Benefits</b></p><p>
From health and financial benefits to time away and everyday wellness, we give Autodeskers the best, so they can do their best work. Learn more about our benefits in the U.S. by visiting https://benefits.autodesk.com/</p><p></p><div><div><div><div><div><div><div><p><b>
Salary transparency</b></p></div></div></div></div></div></div></div>
Salary is one part of Autodesk&rsquo;s competitive compensation package. For U.S.-based roles, we expect a starting base salary between $157,000 and $281,930. Offers are based on the candidate&rsquo;s experience and geographic location, and may exceed this range. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.<p></p><p><b>
Equal Employment Opportunity</b><b>
</b></p><p>At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world. Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic. We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.</p><p></p><p><b>
Diversity &amp; Belonging</b></p><p>
We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here: https://www.autodesk.com/company/diversity-and-belonging</p><p></p><p><b>
Are you an existing contractor or consultant with Autodesk?
</b></p><p>Please search for open jobs and apply internally (not on this external site).</p></div>

Senior Manager, Content Strategy, AEC, Design Solutions

Prolific Academic Ltd

<div><h2 class="jobSectionHeader"><b>Computer Sciences - Graduates - AI Training</b></h2>
<h4 class="jobSectionHeader"><b><i>About Prolific</i></b></h4>
<p>
Prolific is not just another player in the AI space &ndash; we are building the biggest pool of quality human data in the world.</p>
<p>
Over 35,000 AI developers, researchers, and organizations use Prolific to gather data from paid study participants with a wide variety of experiences, knowledge, and skills.</p>
<h4 class="jobSectionHeader"><b><i>
The role</i></b></h4>
<p>
We're looking for <b>Computer Science Specialists</b> to join our Expert Network to help train and evaluate cutting-edge AI models. If you have a background in CS research or technical analysis, we'll send you a quick 10- to 15-minute test to assess your skills. If successful, you'll be invited to join Prolific as a participant, where you'll get paid to help AI understand and summarize complex scientific data.</p>
<p>
Researchers looking for your skills tend to pay up to <b>$60/hr, depending on skills and experience level</b>. You must be prepared to complete paid tasks that <b>require one hour of uninterrupted work</b>, though many are shorter.</p>
<h4 class="jobSectionHeader"><b><i>
What you'll bring</i></b></h4>
<ul><li><b>Educational Background:</b> at minimum, a <b>BSc (Bachelor of Science) in Computer Science</b> or a closely related technical field.</li>
<li><b>Technical Literacy:</b> ability to interpret research papers, understand complex algorithms, and review code logic.</li>
<li><b>Analytical Mindset:</b> high level of cognitive competency with a sharp eye for technical hallucinations or logical flaws.</li>
<li><b>Professional Verification:</b> a valid LinkedIn profile to verify your degree and background during the screening process.</li>
<li><b>A PayPal account to receive payment from our clients</b></li>
</ul><h4 class="jobSectionHeader"><b><i>What you'll be doing in the role</i></b></h4>
<ul><li><b>AI Evaluation &amp; Ranking:</b> comparing multiple AI-generated responses to technical prompts and ranking them based on accuracy, logic, and safety.</li>
<li><b>Scientific Review:</b> reviewing CS research papers alongside AI-generated summaries and graphical abstracts to ensure scientific integrity.</li>
<li><b>Fact-Checking:</b> identifying inaccuracies where the AI has misinterpreted technical data, formulas, or research findings.</li>
<li><b>RLHF (Reinforcement Learning from Human Feedback):</b> providing the human &quot;ground truth&quot; to help models align with professional standards in software engineering and data science.</li>
<li><b>Code &amp; Logic Verification:</b> auditing AI-generated code snippets or architectural diagrams for structural and functional correctness.</li>
</ul><h4 class="jobSectionHeader"><b><i>Key Technologies</i></b></h4>
<ul><li><b>Generative AI &amp; LLMs:</b> training the next generation of technical and reasoning models.</li>
<li><b>Technical Documentation:</b> working with research papers, code repositories, and data visualizations.</li>
<li><b>Verification Frameworks:</b> using structured evaluation rubrics to audit AI performance.</li>
</ul><h4 class="jobSectionHeader"><b><i>Why Prolific is a great platform to join as a Participant</i></b></h4>
<p>
Joining our Expert Network will give you the chance to influence the AI models of the future using professional legal expertise. Once you pass our assessment, you can join Prolific in just 15 minutes, and start enjoying competitive pay rates, flexible hours, and the ability to work from home.</p>
<p>
We've built a unique platform that connects researchers and companies with a global pool of participants, enabling the collection of high-quality, ethically sourced human behavioural data and feedback. This data is the cornerstone of developing more accurate, nuanced, and aligned AI systems.</p>
<div>
We believe that the next leap in AI capabilities won't come solely from scaling existing models, but from integrating diverse human perspectives and behaviours into AI development. By providing this crucial human data infrastructure, Prolific is positioning itself at the forefront of the next wave of AI innovation &ndash; one that reflects the breadth and the best of humanity.</div>
<h4 class="jobSectionHeader"><b><i>Links to more information on Prolific</i></b></h4>
<p>
Website</p>
<p>
Youtube</p>
<h4 class="jobSectionHeader"><b><i>
Privacy Statement</i></b></h4>
<p>
By submitting your application, you agree that Prolific may collect your personal data for recruiting and global organisation planning. Prolific's Candidate Privacy Notice explains what personal information Prolific may process, where Prolific may process your personal information, its purposes for processing your personal information, and the rights you can exercise over Prolific use of your personal personal information.</p></div><p></p>

Computer Sciences - Graduates - AI Training - Washington DC, US

Explore 125+ Ai chatbot evaluator in Washington this May. Find roles across top companies and apply now to boost your career.

125 ai chatbot evaluator jobs In Washington

Data Scientist

Data Scientist

Independent Contractor

AI Content Writing Specialist

Social Media Content Manager

Senior Manager, Content Strategy, AEC, Design Solutions

Computer Sciences - Graduates - AI Training - Washington DC, US