code reviewer

YO IT CONSULTING

<div><div><div><p>Partnering with a top AI research organization to evaluate and improve how coding assistants reason, act, and communicate during development workflows. We&rsquo;re seeking technically sharp experts (especially those with experience in code review, testing, or documentation) to assess full transcripts of user&ndash;AI coding conversations. This short-term engagement helps shape the future of developer-assisting AI systems.</p>
<p>Key Responsibilities</p>
<ul><li>Review long-form transcripts between users and AI coding assistants</li></ul>
<ul><li>Analyze the AI&rsquo;s logic, execution, and stated actions in detail</li></ul>
<ul><li>Score each transcript using a 10-point rubric across multiple criteria</li></ul>
<ul><li>Optionally write brief justifications citing examples from the dialogue</li></ul>
<ul><li>Detect mismatches between claims and actions (e.g., saying &ldquo;I&rsquo;ll run tests&rdquo; but not doing so)</li></ul>
<p>Ideal Qualifications</p>
<p>Top choices:</p>
<ul><li>Senior or Staff Engineers with deep code review experience and execution insight</li></ul>
<ul><li>QA Engineers with strong verification and consistency-checking habits</li></ul>
<ul><li>Technical Writers or Documentation Specialists skilled at comparing instructions vs. implementation</li></ul>
<p>Also a strong fit:</p>
<ul><li>Backend or Full-Stack Developers comfortable with function calls, APIs, and test workflows</li></ul>
<ul><li>DevOps or SRE professionals familiar with tool orchestration and system behavior analysis</li></ul>
<p>Languages and Tools:</p>
<ul><li>Proficiency in Python is helpful (most transcripts are Python-based)</li></ul>
<ul><li>Familiarity with other languages like JavaScript, TypeScript, Java, C++, Go, Ruby, Rust, or Bash is a plus</li></ul>
<ul><li>Comfort with Git workflows, testing frameworks, and debugging tools is valuable</li></ul>
<p>More About the Opportunity</p>
<ul><li>Must complete each transcript batch within 5 hours of starting (unlimited tasks to be done)</li></ul>
<ul><li>Flexible, task-based engagement with potential for recurring batches</li></ul>
<p>Application Process</p>
<ul><li>Submit your resume to begin</li></ul>
<ul><li>If selected, you&rsquo;ll receive rubric documentation and access to the evaluation platform</li></ul>
<ul><li>Most applicants hear back within a few business days</li></ul>
<p>We consider all qualified applicants without regard to legally protected characteristics and provide reasonable accommodations upon request.</p>
</div></div><div><div><h2 class="jobSectionHeader"><b>Contract and Payment Terms</b></h2>
<ul><li>You will be engaged as an independent contractor.</li>
<li>This is a fully remote role that can be completed on your own schedule.</li>
<li>Projects can be extended, shortened, or concluded early depending on needs and performance.</li>
<li>Your work at will not involve access to confidential or proprietary information from any employer, client, or institution.</li>
<li>Payments are weekly on Stripe or Wise based on services rendered.</li></ul></div></div></div>

Software Test Engineer QA

QA Software Engineer - Code Review Expert

Search 8109+ Code reviewer jobs near you this March. New roles added regularly across leading industries. Apply today.

8109 code reviewer jobs Near Me

Software Test Engineer QA

QA Software Engineer - Code Review Expert

Software Test Engineer QA

Software Test Engineer QA

QA Software Engineer - Code Review Expert

QA Software Engineer - Code Review Expert

Software Test Engineer QA