evaluation scenario writer - ai agent testing specialist

claims specialist

evaluation scenario writer - ai agent testing specialist Jobs in Arizona

evaluation scenario writer ai agent testing specialist

Mindrift

<div>
 
 
 Please submit your CV in English and indicate your level of English proficiency.
 
 
 
 
 Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
 
 Participation isproject-based, not permanent employment.
 
 
 
 
 What This Opportunity Involves
 
 
 
 You'll create challenging coding test cases that push AI coding systems to their limits:
 
 
 <ul>
 <li>
 Review and refine realistic coding tasks based on provided production codebases with realistic scope, requirements and information sources
 </li>
 <li>
 Write comprehensive functional tests that validate actual end-to-end behavior and edge-cases, not just superficial checks
 </li>
 <li>
 Craft "fair but hard" challenges where the AI has all the context it needs, but has to work for it (information scattered across files and external sources, complex reasoning required)
 </li>
 <li>
 Analyze AI failures to understand what the model struggles with vs. what it masters
 </li>
 <li>
 Iterate based on feedback from expert QA reviewers who score your work on 7 quality criteria
 
 
 </li>
 </ul>
 
 What We Look For
 
 
 
 This opportunity is a good fit for experienced developers, software engineers, and/or test automation specialists open to part-time, non-permanent projects. Ideally, contributors will have:
 
 
 <ul>
 <li>
 Degree in Computer Science, Software Engineering or related fields
 </li>
 <li>
 5+ years in software development, primarily Python (pytest, async/await, subprocess, file operations)
 </li>
 <li>
 Background in Full-Stack development, with an equal focus on building React-based interfaces and robust Back-end systems
 </li>
 <li>
 Experience writing tests (functional, integration - not just running them)
 </li>
 <li>
 Docker containers (running evaluations locally in containers)
 </li>
 <li>
 CI/CD understanding (GitHub Actions as a user: triggers, labels, reading results)
 </li>
 <li>
 English proficiency - B2
 
 
 </li>
 </ul>
 
 How It Works
 
 
 
 Apply &rarr; Pass qualification(s) &rarr; Join a project &rarr; Complete tasks &rarr; Get paid
 
 
 
 Effort estimate
 
 
 
 Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
 
 
 
 Payment
 
 
 
 <ul>
 <li>
 Paid contributions, with rates up to $80/hour*
 </li>
 <li>
 Fixed project rate or individual rates, depending on the project
 </li>
 <li>
 Some projects include incentive payments
 </li>
 <li>
 Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project
 </li>
 </ul>
</div>

Evaluation Scenario Writer - AI Agent Testing Specialist

<div>
 
 
 This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.
 
 
 
 
 Mindrift is looking for highly skilled Vibecode specialists to join the Tendem project (https://tendem.ai/) and drive specialized data scraping workflows within our hybrid AI + human system.
 
 
 In this role, as an
 
 AI Pilot
 
 - that's how we refer to this role at Mindrift - you'll collaborate with Tendem Agents that handle repetitive tasks, while you provide critical thinking, domain expertise, and quality control to deliver accurate and actionable results.
 
 
 This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction and processing.
 
 
 
 What We Do
 
 
 
 The Mindrift platform connects specialists with AI projects from major tech innovators. Our mission is to unlock the potential of Generative AI by tapping into real-world expertise from across the globe.
 
 
 
 About The Role
 
 
 
 This is a freelance role for a Tendem project. As a Vibe Code specialist, you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing various tools such as our provided Apify and OpenRouter alongside your own resourceful approaches.
 
 
 
 Key Responsibilities
 
 
 
 <ul>
 <li>
 Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets.
 </li>
 <li>
 Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements.
 </li>
 <li>
 Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior.
 </li>
 <li>
 Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery.
 </li>
 <li>
 Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes
 
 
 </li>
 </ul>
 
 How To Get Started
 
 
 
 Simply apply to this post, qualify, and get the chance to contribute to projects that match your technical skills, on your own schedule. From coding and automation to fine-tuning AI outputs, you'll play a key role in advancing AI capabilities and real-world applications.
 
 
 
 Requirements
 
 
 
 <ul>
 <li>
 At least 1 year of relevant experience in data analysis, AI automation, data engineering, or software development is desirable.
 </li>
 <li>
 Bachelor's or Master's Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
 </li>
 <li>
 Python web scraping: Build reliable scraping scripts using BeautifulSoup, Selenium (or equivalents) for multi-level sites, dynamic JS content (infinite scroll, AJAX), and API endpoints via provided proxy
 </li>
 <li>
 Data extraction expertise: Navigate complex hierarchical structures (regions &rarr; companies &rarr; details), handling archived pages and varied HTML formats
 </li>
 <li>
 Data processing: Clean, normalize, and validate scraped data; deliver high-quality datasets in well-structured formats (CSV, JSON, Google Sheets) with clear, consistent presentation
 </li>
 <li>
 Hands-on experience with LLMs and AI frameworks to enhance automation and problem-solving
 </li>
 <li>
 Strong attention to detail and commitment to data accuracy
 </li>
 <li>
 Self-directed work ethic with ability to troubleshoot independently
 </li>
 <li>
 English proficiency: Upper-intermediate (B2) or above (required)
 
 
 </li>
 </ul>
 
 Benefits
 
 
 
 
 Why this freelance opportunity might be a great fit for you?
 
 
 
 <ul>
 <li>
 Get paid for your expertise, with rates that can go up to $32/hour depending on your skills, experience, and project needs
 </li>
 <li>
 Work fully remote on your own schedule with just a laptop and stable internet connection.
 </li>
 <li>
 Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly &mdash; a distinctive skill set in a rapidly growing field.
 </li>
 <li>
 Participate in performance-based bonus programs that reward high-quality work and consistent delivery
 </li>
 </ul>
</div>

Vibe Coding Web Scraping Expert (Freelance)

DataAnnotation

<div>
 DataAnnotation is committed to creating high-quality AI. We are looking for a Finance Manager to join our team to help train the next generation of AI while enjoying the flexibility of remote work and the freedom to set your own schedule. This role is designed to fit a variety of lifestyles &mdash; whether you&rsquo;re looking to contribute part-time alongside a current position, pursue it full-time, or engage periodically as a flexible professional opportunity.
 
 
 We're currently expanding into an exciting new area &ndash; teaching AI Assistant models to be a more useful tool for finance professionals. We're seeking experienced finance professionals with advanced degrees (MBA+) and professional experience to use their expertise to help shape how AI understands financial principles and decision-making.
 
 
 
 Role
 
 
 
 We&rsquo;re growing a team of finance experts, and as the team grows, so will your opportunities. In this role, you might:
 
 
 <ul>
 <li>
 Review and improve AI Assistant answers to questions about macro trends, corporate finance, and capital markets
 </li>
 <li>
 Leverage your education and work experience to check the reasoning and accuracy of an AI Assistant's work
 </li>
 <li>
 Push the models with complex, real-world scenarios and edge cases to see where their reasoning holds up &ndash; and where it doesn&rsquo;t.
 </li>
 <li>
 Share clear, structured feedback to help make each new version of the AI smarter and more reliable.
 
 
 </li>
 </ul>
 To succeed in this position, you should have expert-level financial reasoning and formal training in a finance-related discipline. A Master&rsquo;s or PhD (completed or in progress) is strongly preferred. Relevant backgrounds include Financial Accounting, Investment Banking, Corporate Development, Wealth Management, and Insurance Planning.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at USD $50-$60 per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 Detail-oriented
 </li>
 <li>
 Proficient in financial analysis, financial modeling, data analysis, and other reasoning exercises related to finance management
 </li>
 <li>
 A current, in progress, or completed Masters and/or PhD is is preferred but not required
 
 
 </li>
 </ul>
 Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in United States will be considered for this role. This is an independent contract position.
 
 
 #finance
</div>

Finance Manager

<div>
 We are looking for a Registered Nurse to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 
 Physicians &amp; Advanced Practice Clinicians
 
 
 
 In this role you will need to be an expert in healthcare. We are interested in a wide range of expertise, so relevant backgrounds include:
 
 
 Physicians of all specialties (e.g., Internists, Cardiologists, Oncologists), Physician Assistants, Nurse Practitioners, Certified Nurse-Midwives, Certified Registered Nurse Anesthetists, Clinical Nurse Specialists, Registered Nurses.
 
 
 
 Therapists
 
 
 
 Physical Therapists, Occupational Therapists, Speech-Language Pathologists, Respiratory Therapists, Athletic Trainers, Massage/Recreational Therapists.
 
 
 
 Diagnostic &amp; Laboratory Professionals
 
 
 
 Radiologic Technologists, Sonographers, MRI &amp; Nuclear Medicine Technologists, Medical Laboratory Scientists, Phlebotomists, Histology &amp; Genetics Technicians.
 
 
 
 Public Health &amp; Specialized Roles
 
 
 
 Dietitians/Nutritionists, Genetic Counselors, Epidemiologists, Public Health Nurses.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at $50-$60 USD per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex healthcare related problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 </li>
 <li>
 Ensure the medical accuracy of model performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 A current or in progress medical degree
 
 
 </li>
 </ul>
 Notes: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 
 
 #medicine
</div>

Registered Nurse

Piping Estimator

Turo

<div>
 
 About The Team
 
 
 
 The Claims Specialist team handles claims of higher complexity, requiring significant levels of understanding and interpretation of Turo&rsquo;s operating policies and procedures. They investigate causation, determine whether coverage applies, and exercise judgment on matters of financial impact to Turo and its hosts.
 
 
 
 What You Will Do
 
 
 
 <ul>
 <li>
 Investigate high-complexity claims with significant financial impact by interpreting policies, determining methods of investigation, interviewing customers, assessing credibility, analyzing damages, and determining eligibility for protection on behalf of Turo.
 </li>
 <li>
 Communicate with customers to gather information, explain protection procedures, educate hosts, clarify coverage decisions, and negotiate claim resolutions.
 </li>
 <li>
 Review escalations or reconsiderations from standard/fast track claims teams, including Fair Claims reviews and arbitration presentations, and make recommendations on behalf of Turo management.
 </li>
 <li>
 Exercise discretion and judgment to waive or make exceptions to processes and policies to achieve fair outcomes for hosts, when warranted.
 </li>
 <li>
 Participate in special projects and business changes, providing recommendations on claims process improvements, objectives, and KPIs.
 </li>
 <li>
 Attend meetings, huddles, one-on-ones, and training sessions.
 </li>
 <li>
 Perform other related duties as assigned.
 
 
 </li>
 </ul>
 
 Your profile
 
 
 
 <ul>
 <li>
 Strong investigative mindset with the ability to analyze evidence and draw logical conclusions.
 </li>
 <li>
 Demonstrated ability to exercise independent judgment and decision-making beyond standard guidelines.
 </li>
 <li>
 Excellent communication skills, with the ability to clearly and concisely explain complex situations.
 </li>
 <li>
 Strong organizational skills with the ability to manage a pending inventory of claims independently.
 </li>
 <li>
 Proficient in reviewing and evaluating estimates with a strong understanding of collision damage theory.
 </li>
 <li>
 Professional, empathetic communication style in both verbal and written interactions.
 </li>
 <li>
 Ability to multitask across technical platforms.
 </li>
 <li>
 Proficiency with macOS and Google Docs.
 
 
 </li>
 </ul>
 For this role, the target base salary range in Phoenix is $50,000&ndash;$62,000 annually. This role is also eligible for equity and benefits. In general, our ranges reflect the market-based target for new hire salaries based on the level and location of the role. Within the range, individual pay is determined by objective factors assessed during the application and interview process, such as job-related skills, experience, and relevant education or training. We encourage you to talk with your recruiter to learn more about the total compensation and benefits available for this role.
 
 
 
 Turo highly values having employees working in-office to foster a collaborative work environment and company culture. This role will be in-office on a hybrid schedule &mdash; Turists will be expected to work in the office 3 days per week on Mondays, Wednesdays, and Thursdays. Your recruiter can share more information about the various in-office perks Turo offers.
 
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 Competitive salary, equity, benefits, and perks for all full-time employees
 </li>
 <li>
 Employer-paid medical, dental, and vision insurance (Country specific)
 </li>
 <li>
 Retirement employer match
 </li>
 <li>
 Learning &amp; Development stipend to invest in your professional development
 </li>
 <li>
 Turo host matching program
 </li>
 <li>
 Turo travel credit
 </li>
 <li>
 Cell phone and internet stipend
 </li>
 <li>
 Paid time off to relax and recharge
 </li>
 <li>
 Paid holidays, volunteer time off, and parental leave
 </li>
 <li>
 For those who are in the office full-time or hybrid we have in-office lunch, office snacks, and fun activities
 
 
 </li>
 </ul>
 We are committed to building a diverse team. If you are from a background that's underrepresented in tech, we'd love to meet you.
 
 
 Aside from an award winning work environment and the opportunity to be part of the world&rsquo;s largest car sharing marketplace, we are also growing the team quickly - join us! Even if you don't meet every qualification, we are looking for people with enthusiasm for what we do and we will consider you for this and other possibilities.
 
 
 
 About Turo
 
 
 
 Turo is the world&rsquo;s largest car sharing marketplace where you can book the perfect car for wherever you&rsquo;re going from a vibrant community of trusted hosts across the US, UK, Canada, Australia, and France. Whether you're flying in from afar or looking for a car down the street, searching for a rugged truck or something smooth and swanky, Turo puts you in the driver's seat of an extraordinary selection of cars shared by local hosts.
 
 
 Discover Turo at https://turo.com, the App Store, and Google Play, and check out our blog, Field Notes.
 
 
 Read more about the Turo culture according to Turo CEO, Andre Haddad.
 
 
 Turo is an Equal Opportunity Employer and a participant in the U.S. Federal E-Verify program. Women, minorities, individuals with disabilities and protected veterans are encouraged to apply. We welcome people of different backgrounds, experiences, abilities and perspectives.
 
 
 Turo will consider qualified applicants with criminal histories in a manner consistent with the San Francisco Fair Chance Ordinance, as applicable.
 
 
 We welcome candidates with physical, mental, and/or neurological disabilities. If you require assistance applying for an open position, or need accommodation during the recruiting process due to a disability, please submit a request to People Operations by emailing PeopleOps@turo.com.
</div>

Claims Specialist