claims specialist

evaluation scenario writer - ai agent testing specialist Jobs in Arizona United States

evaluation scenario writer ai agent testing specialist

Mindrift

<div>
 
 
 Please submit your CV in English and indicate your level of English proficiency.
 
 
 
 
 Mindrift connects specialists with project-based AI opportunities for leading tech companies, focused on testing, evaluating, and improving AI systems.
 
 Participation isproject-based, not permanent employment.
 
 
 
 
 What This Opportunity Involves
 
 
 
 You'll create challenging coding test cases that push AI coding systems to their limits:
 
 
 <ul>
 <li>
 Review and refine realistic coding tasks based on provided production codebases with realistic scope, requirements and information sources
 </li>
 <li>
 Write comprehensive functional tests that validate actual end-to-end behavior and edge-cases, not just superficial checks
 </li>
 <li>
 Craft "fair but hard" challenges where the AI has all the context it needs, but has to work for it (information scattered across files and external sources, complex reasoning required)
 </li>
 <li>
 Analyze AI failures to understand what the model struggles with vs. what it masters
 </li>
 <li>
 Iterate based on feedback from expert QA reviewers who score your work on 7 quality criteria
 
 
 </li>
 </ul>
 
 What We Look For
 
 
 
 This opportunity is a good fit for experienced developers, software engineers, and/or test automation specialists open to part-time, non-permanent projects. Ideally, contributors will have:
 
 
 <ul>
 <li>
 Degree in Computer Science, Software Engineering or related fields
 </li>
 <li>
 5+ years in software development, primarily Python (pytest, async/await, subprocess, file operations)
 </li>
 <li>
 Background in Full-Stack development, with an equal focus on building React-based interfaces and robust Back-end systems
 </li>
 <li>
 Experience writing tests (functional, integration - not just running them)
 </li>
 <li>
 Docker containers (running evaluations locally in containers)
 </li>
 <li>
 CI/CD understanding (GitHub Actions as a user: triggers, labels, reading results)
 </li>
 <li>
 English proficiency - B2
 
 
 </li>
 </ul>
 
 How It Works
 
 
 
 Apply &rarr; Pass qualification(s) &rarr; Join a project &rarr; Complete tasks &rarr; Get paid
 
 
 
 Effort estimate
 
 
 
 Tasks for this project are estimated to take 20 hours to complete, depending on complexity. This is an estimate and not a schedule requirement; you choose when and how to work. Tasks must be submitted by the deadline and meet the listed acceptance criteria to be accepted.
 
 
 
 Payment
 
 
 
 <ul>
 <li>
 Paid contributions, with rates up to $80/hour*
 </li>
 <li>
 Fixed project rate or individual rates, depending on the project
 </li>
 <li>
 Some projects include incentive payments
 </li>
 <li>
 Note: Rates vary based on expertise, skills assessment, location, project needs, and other factors. Higher rates may be offered to highly specialized experts. Lower rates may apply during onboarding or non-core project phases. Payment details are shared per project
 </li>
 </ul>
</div>

Evaluation Scenario Writer - AI Agent Testing Specialist

<div>
 
 
 This opportunity is only for candidates currently residing in the specified country. Your location may affect eligibility and rates. Please submit your resume in English and indicate your level of English.
 
 
 
 
 Mindrift is looking for highly skilled Vibecode specialists to join the Tendem project (https://tendem.ai/) and drive specialized data scraping workflows within our hybrid AI + human system.
 
 
 In this role, as an
 
 AI Pilot
 
 - that's how we refer to this role at Mindrift - you'll collaborate with Tendem Agents that handle repetitive tasks, while you provide critical thinking, domain expertise, and quality control to deliver accurate and actionable results.
 
 
 This part-time remote opportunity is ideal for technical professionals with hands-on experience in web scraping, data extraction and processing.
 
 
 
 What We Do
 
 
 
 The Mindrift platform connects specialists with AI projects from major tech innovators. Our mission is to unlock the potential of Generative AI by tapping into real-world expertise from across the globe.
 
 
 
 About The Role
 
 
 
 This is a freelance role for a Tendem project. As a Vibe Code specialist, you'll handle data scraping tasks requiring technical precision for web extraction and processing, utilizing various tools such as our provided Apify and OpenRouter alongside your own resourceful approaches.
 
 
 
 Key Responsibilities
 
 
 
 <ul>
 <li>
 Own end-to-end data extraction workflows across complex websites, ensuring complete coverage, accuracy, and reliable delivery of structured datasets.
 </li>
 <li>
 Leverage internal tools (Apify, OpenRouter) alongside custom workflows to accelerate data collection, validation, and task execution while meeting defined requirements.
 </li>
 <li>
 Ensure reliable extraction from dynamic and interactive web sources, adapting approaches as needed to handle JavaScript-rendered content and changing site behavior.
 </li>
 <li>
 Enforce data quality standards through validation checks, cross-source consistency controls, adherence to formatting specifications, and systematic verification prior to delivery.
 </li>
 <li>
 Scale scraping operations for large datasets using efficient batching or parallelization, monitor failures, and maintain stability against minor site structure changes
 
 
 </li>
 </ul>
 
 How To Get Started
 
 
 
 Simply apply to this post, qualify, and get the chance to contribute to projects that match your technical skills, on your own schedule. From coding and automation to fine-tuning AI outputs, you'll play a key role in advancing AI capabilities and real-world applications.
 
 
 
 Requirements
 
 
 
 <ul>
 <li>
 At least 1 year of relevant experience in data analysis, AI automation, data engineering, or software development is desirable.
 </li>
 <li>
 Bachelor's or Master's Degree in Engineering, Applied Mathematics, Computer Science, or related technical fields is a plus
 </li>
 <li>
 Python web scraping: Build reliable scraping scripts using BeautifulSoup, Selenium (or equivalents) for multi-level sites, dynamic JS content (infinite scroll, AJAX), and API endpoints via provided proxy
 </li>
 <li>
 Data extraction expertise: Navigate complex hierarchical structures (regions &rarr; companies &rarr; details), handling archived pages and varied HTML formats
 </li>
 <li>
 Data processing: Clean, normalize, and validate scraped data; deliver high-quality datasets in well-structured formats (CSV, JSON, Google Sheets) with clear, consistent presentation
 </li>
 <li>
 Hands-on experience with LLMs and AI frameworks to enhance automation and problem-solving
 </li>
 <li>
 Strong attention to detail and commitment to data accuracy
 </li>
 <li>
 Self-directed work ethic with ability to troubleshoot independently
 </li>
 <li>
 English proficiency: Upper-intermediate (B2) or above (required)
 
 
 </li>
 </ul>
 
 Benefits
 
 
 
 
 Why this freelance opportunity might be a great fit for you?
 
 
 
 <ul>
 <li>
 Get paid for your expertise, with rates that can go up to $32/hour depending on your skills, experience, and project needs
 </li>
 <li>
 Work fully remote on your own schedule with just a laptop and stable internet connection.
 </li>
 <li>
 Gain hands-on experience in a unique hybrid environment where human expertise and AI agents collaborate seamlessly &mdash; a distinctive skill set in a rapidly growing field.
 </li>
 <li>
 Participate in performance-based bonus programs that reward high-quality work and consistent delivery
 </li>
 </ul>
</div>

Vibe Coding Web Scraping Expert (Freelance)

DataAnnotation

<div>
 DataAnnotation is committed to creating high-quality AI. We are looking for a Finance Manager to join our team to help train the next generation of AI while enjoying the flexibility of remote work and the freedom to set your own schedule. This role is designed to fit a variety of lifestyles &mdash; whether you&rsquo;re looking to contribute part-time alongside a current position, pursue it full-time, or engage periodically as a flexible professional opportunity.
 
 
 We're currently expanding into an exciting new area &ndash; teaching AI Assistant models to be a more useful tool for finance professionals. We're seeking experienced finance professionals with advanced degrees (MBA+) and professional experience to use their expertise to help shape how AI understands financial principles and decision-making.
 
 
 
 Role
 
 
 
 We&rsquo;re growing a team of finance experts, and as the team grows, so will your opportunities. In this role, you might:
 
 
 <ul>
 <li>
 Review and improve AI Assistant answers to questions about macro trends, corporate finance, and capital markets
 </li>
 <li>
 Leverage your education and work experience to check the reasoning and accuracy of an AI Assistant's work
 </li>
 <li>
 Push the models with complex, real-world scenarios and edge cases to see where their reasoning holds up &ndash; and where it doesn&rsquo;t.
 </li>
 <li>
 Share clear, structured feedback to help make each new version of the AI smarter and more reliable.
 
 
 </li>
 </ul>
 To succeed in this position, you should have expert-level financial reasoning and formal training in a finance-related discipline. A Master&rsquo;s or PhD (completed or in progress) is strongly preferred. Relevant backgrounds include Financial Accounting, Investment Banking, Corporate Development, Wealth Management, and Insurance Planning.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at USD $50-$60 per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 Detail-oriented
 </li>
 <li>
 Proficient in financial analysis, financial modeling, data analysis, and other reasoning exercises related to finance management
 </li>
 <li>
 A current, in progress, or completed Masters and/or PhD is is preferred but not required
 
 
 </li>
 </ul>
 Note: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in United States will be considered for this role. This is an independent contract position.
 
 
 #finance
</div>

Finance Manager

<div>
 We are looking for medical experts to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 In This Role You Will Need To Be An Expert In Healthcare. We Are Interested In a Wide Range Of Expertise, So Relevant Backgrounds Include:
 
 
 Physicians &amp; Advanced Practice Clinicians: Physicians of all specialties (e.g., Internists, Cardiologists, Oncologists), Physician Assistants, Nurse Practitioners, Certified Nurse-Midwives, Certified Registered Nurse Anesthetists, Clinical Nurse Specialists, Registered Nurses.
 
 
 Therapists: Physical Therapists, Occupational Therapists, Speech-Language Pathologists, Respiratory Therapists, Athletic Trainers, Massage/Recreational Therapists.
 
 
 Diagnostic &amp; Laboratory Professionals: Radiologic Technologists, Sonographers, MRI &amp; Nuclear Medicine Technologists, Medical Laboratory Scientists, Phlebotomists, Histology &amp; Genetics Technicians.
 
 
 Public Health &amp; Specialized Roles: Dietitians/Nutritionists, Genetic Counselors, Epidemiologists, Public Health Nurses.
 
 
 
 Benefits:
 
 
 
 This is a full-time or part-time REMOTE position
 
 
 You&rsquo;ll be able to choose which projects you want to work on
 
 
 You can work on your own schedule
 
 
 Projects are paid hourly starting at $40+ per hour, with bonuses on high-quality and high-volume work
 
 
 
 Responsibilities:
 
 
 
 Give AI chatbots diverse and complex healthcare related problems and evaluate their outputs
 
 
 Evaluate the quality produced by AI models for correctness and performance
 
 
 Ensure the medical accuracy of model performance
 
 
 
 Qualifications:
 
 
 
 Fluency in English (native or bilingual level)
 
 
 A current or in progress medical degree
 
 
 
 Notes:
 
 
 
 Payment is made via PayPal
 
 
 We will never ask for any money from you
 
 
 PayPal will handle any currency conversions from USD
 
 
 Only applicants in Canada will be considered for this role
 
 
 This is an independent contract position.
 
 
 #MEDICINE
 
 
 
 
 Benefits:
 
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at $30+ USD per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 
 Responsibilities:
 
 
 
 
 <ul>
 <li>
 Give AI chatbots writing and editing tasks and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 
 
 </li>
 </ul>
 
 
 Qualifications:
 
 
 
 
 <ul>
 <li>
 Fluency in English
 </li>
 <li>
 Detail-oriented
 </li>
 <li>
 Excellent command of grammar, style, and brand voice
 </li>
 <li>
 A current, in progress, or completed Bachelor's degree is preferred but not required
 
 
 </li>
 </ul>
 
 
 Notes:
 
 
 
 
 Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. This is an independent contract position.
 
 
 #medicine
</div>

Clinical Nurse

<div>
 We are looking for a Registered Nurse to join our team to train AI models. You will measure the progress of these AI chatbots, evaluate their logic, and solve problems to improve the quality of each model.
 
 
 
 Physicians &amp; Advanced Practice Clinicians
 
 
 
 In this role you will need to be an expert in healthcare. We are interested in a wide range of expertise, so relevant backgrounds include:
 
 
 Physicians of all specialties (e.g., Internists, Cardiologists, Oncologists), Physician Assistants, Nurse Practitioners, Certified Nurse-Midwives, Certified Registered Nurse Anesthetists, Clinical Nurse Specialists, Registered Nurses.
 
 
 
 Therapists
 
 
 
 Physical Therapists, Occupational Therapists, Speech-Language Pathologists, Respiratory Therapists, Athletic Trainers, Massage/Recreational Therapists.
 
 
 
 Diagnostic &amp; Laboratory Professionals
 
 
 
 Radiologic Technologists, Sonographers, MRI &amp; Nuclear Medicine Technologists, Medical Laboratory Scientists, Phlebotomists, Histology &amp; Genetics Technicians.
 
 
 
 Public Health &amp; Specialized Roles
 
 
 
 Dietitians/Nutritionists, Genetic Counselors, Epidemiologists, Public Health Nurses.
 
 
 
 Benefits
 
 
 
 <ul>
 <li>
 This is a full-time or part-time REMOTE position
 </li>
 <li>
 You&rsquo;ll be able to choose which projects you want to work on
 </li>
 <li>
 You can work on your own schedule
 </li>
 <li>
 Projects are paid hourly starting at $50-$60 USD per hour, with bonuses on high-quality and high-volume work
 
 
 </li>
 </ul>
 
 Responsibilities
 
 
 
 <ul>
 <li>
 Give AI chatbots diverse and complex healthcare related problems and evaluate their outputs
 </li>
 <li>
 Evaluate the quality produced by AI models for correctness and performance
 </li>
 <li>
 Ensure the medical accuracy of model performance
 
 
 </li>
 </ul>
 
 Qualifications
 
 
 
 <ul>
 <li>
 Fluency in English (native or bilingual level)
 </li>
 <li>
 A current or in progress medical degree
 
 
 </li>
 </ul>
 Notes: Payment is made via PayPal. We will never ask for any money from you. PayPal will handle any currency conversions from USD. Only applicants in the United States will be considered for this role. This is an independent contract position.
 
 
 #medicine
</div>

Registered Nurse

Guidant Financial

<div>
 
 This is a fantastic opportunity to work from the comfort of your own home! We are excited to offer a fully remote position with a reputable and thriving company. Currently, we are recruiting in select states across the US: AZ, FL, GA, ID, KS, LA, MI, MN, NC, NE, TX, UT, VA, MO, OK.
 
 
 
 
 The wage for this position is $23.55 per hour, equivalent to an annual salary of $49,000
 
 
 
 
 
 Please apply to only one location only per role. As a fully remote company, we post to multiple locations. Applying for more than one location could result in receiving multiple communications around a position.
 
 
 
 
 Are you thrilled by the idea of assisting entrepreneurs nationwide in achieving their dreams of launching their own businesses?
 
 
 For more than 23 years, Guidant Financial has been committed to empowering entrepreneurs and enhancing the growth of small businesses.. So far, we have distributed an impressive $5 billion in funding, supporting the launch of over 30,000 businesses and generating nearly 100,000 jobs that enrich the economy. We aim to inspire and empower you just as we do for the entrepreneurs we assist, allowing you to make a meaningful impact on their success and your own career path.
 
 
 At Guidant, we believe in empowering our employees to make a genuine difference for our clients and their own professional paths. Our commitment to ownership is reflected in our daily work through accountability, innovative problem-solving, and diligent follow-through. We are on an exciting journey towards becoming an ESOP company, further solidifying our dedication to shared accountability and long-term prosperity.
 
 
 
 Our Values
 
 
 
 
 Adaptability:
 
 Embracing change is your superpower. You are always ready to seize opportunities to adapt and inspire others along the way.
 
 
 
 Connection:
 
 Building meaningful connections is at the core of what you do. Elevating others is your way of growing and empowering yourself.
 
 
 
 Excellence:
 
 You thrive outside of your comfort zone, constantly seeking ways to improve. Your unquenchable curiosity fuels your drive to make things better.
 
 
 
 Role Summary
 
 
 
 As a Retirement Plan Specialist at Guidant Financial you will take the first step down the path of learning the full process of what it takes to administer, test and report on 401(k) plans. Through accelerated learning, in-depth training and practical experience, you will quickly grow from document review to administering basic plans. You'll review documents submitted by our clients, proactively keep clients updated on progress and help clients understand the year-end reporting documents necessary to complete government filings, including the annual 5500 return.
 
 
 
 What You'll Be Doing:
 
 
 
 <ul>
 <li>
 Work with clients to obtain necessary information to prepare the annual reporting
 </li>
 <li>
 Review documents submitted by clients for completeness and accuracy
 </li>
 <li>
 Reconcile plans with only rollovers and plan stock using Excel and administration software
 </li>
 <li>
 Utilize administration software to complete 401(k) testing and reporting
 </li>
 <li>
 Make outgoing reminder calls and send emails to clients during the plan year reporting process
 </li>
 <li>
 Prepare required 401(k) plan government filings, including the IRS Form 5500, using filing software
 </li>
 <li>
 Educate clients regarding the administration of their 401(k) plan and 401(k) plan documents
 
 
 </li>
 </ul>
 
 Extra Tasks
 
 
 
 <ul>
 <li>
 Share knowledge and explain complex topics simply
 </li>
 <li>
 Training on concepts for a Retirement Plan Administrator I
 </li>
 <li>
 Performs other duties as required
 
 
 </li>
 </ul>
 
 Requirements
 
 
 
 <ul>
 <li>
 Must be 18 years of age
 </li>
 <li>
 Excellent verbal and written communication skills
 </li>
 <li>
 Meticulous attention to detail and proactive approach to problem solving
 </li>
 <li>
 Skills in customer service, organization, and collaboration
 </li>
 <li>
 Adequate math and analytical skills
 </li>
 <li>
 Proficiency in using Microsoft Excel, Word, and Outlook
 </li>
 <li>
 Associate's or bachelor's degree in a related field (business, finance, etc.) or equivalent work experience preferred
 </li>
 <li>
 Basic knowledge of 401(K) plan structure, ROBS, and IRS/DOL Regulations is preferred
 </li>
 <li>
 Ability to work in an autonomous environment where you can be a self-directed worker
 </li>
 <li>
 Ability to create an inclusive workplace where everyone feels a sense of belonging by empowering all our employees to speak up, ask questions, and been seen
 
 
 </li>
 </ul>
 
 Additional Requirements
 
 
 
 <ul>
 <li>
 Must pass a background check
 </li>
 <li>
 Must meet work from home requirements to qualify
 </li>
 <li>
 Complete continuing education based upon current certifications
 
 
 </li>
 </ul>
 
 For this position, we require you to meet minimum work from home requirements to ensure you can effectively carry out your responsibilities in a remote work from home (WFH) environment. These requirements consist of:
 
 
 
 <ul>
 <li>
 A designated workspace (such as a home office) where you can concentrate and be productive
 </li>
 <li>
 Reliable high-speed internet to ensure seamless communication and collaboration
 
 
 </li>
 </ul>
 
 Benefits
 
 
 
 <ul>
 <li>
 Ownership Matters: We are working toward becoming an ESOP company, reinforcing a culture of shared accountability and long-term success.
 </li>
 <li>
 Our People are Vital to Our Success: Join a team of intelligent, imaginative, and highly motivated individuals
 </li>
 <li>
 Growing Together: Constantly expand your knowledge with ongoing training, career development opportunities, and supportive leaders. Over the past three years, more than 20% of our organization has been promoted into new and exciting roles!
 </li>
 <li>
 Wellness at its Best: Enjoy top-notch benefits and perks, including comprehensive health coverage (with a free premium option), competitive salary, paid parental leave, and a generous 401k match. Take advantage of 3 weeks of Paid Time Off plus 11 Paid Holidays to truly disconnect
 </li>
 <li>
 Flexible Work Arrangements: Work comfortably from your home office. We believe in giving you the freedom to structure your work while providing all the necessary equipment
 </li>
 <li>
 An Amazing Culture: Experience Guidant's award-winning culture, where you can connect with senior leaders over coffee or be part of our company sporting events
 </li>
 <li>
 Empowering Equity and Inclusion: Join an inclusive work environment that values employee voices, encourages active participation, and provides unwavering support
 
 
 </li>
 </ul>
 
 This is a fantastic opportunity to work from the comfort of your own home! We are excited to offer a fully remote position with a reputable and thriving company. Currently, we are recruiting in select states across the US: AZ, FL, GA, ID, KS, LA, MI, MN, NC, NE, TX, UT, VA, MO, OK.
 
 
 
 
 The wage for this position is $23.55 per hour, equivalent to an annual salary of $49,000
 
 
 
 
 
 Please apply to only one location only per role. As a fully remote company, we post to multiple locations. Applying for more than one location could result in receiving multiple communications around a position.
 
 
</div>

Retirement Plan Operations Specialist

Piping Estimator

Explore 8+ Evaluation scenario writer ai agent testing specialist in Arizona United States this March. Find roles across top companies and apply now to boost your career.