generative ai specialist

Vyro

<div><h3 class="jobSectionHeader"><b>Job Title: AI Video Prompt Specialist</b></h3><p><b>
Department:</b> Creative</p><p><b>
Location:</b> On-Site (H12, NSTP, Islamabad)</p><h3 class="jobSectionHeader"><b>
About VYRO</b></h3><p>
VVyro is an <b>AI-first creative technology company</b> empowering creators worldwide. With <b>5M+ active users every month</b> and <b>20+ AI-powered apps</b>, we help millions create, edit, and design across images, videos, and digital media&mdash;faster and more intuitively.</p><p><b>
ImagineArt</b>, Vyro&rsquo;s flagship generative AI platform, enables artists and designers to create high-quality visual art from simple text prompts, making creativity more accessible and limitless.</p><p>
Powered by <b>AI and Machine Learning</b>, Vyro&rsquo;s products remove friction from the creative process while unlocking new possibilities for creators at every level.</p><p>
We are an <b>8-figure, fully bootstrapped company</b> built by a focused team of builders&mdash;<b>Vyronauts</b>&mdash;who move fast, take ownership, and build with long-term impact.</p><p>
We&rsquo;re shaping the future of AI-powered creativity&mdash;and we&rsquo;re just getting started.</p><p><b>
Job Summary</b></p><p>
As an AI Prompt Engineer at Imagine Art, you will be at the intersection of technology and creativity, designing generative AI workflows that bring cinematic ideas to life. You will work on creating high-quality visual, audio, and textual content, helping our team push the boundaries of digital storytelling. This role is perfect for someone with a strong technical background and a passion for filmmaking, looking to translate creative vision into cutting-edge generative outputs.</p><p></p><p><b>
Requirements</b></p><ul><li><p><b>
4+ years of experience in Prompt Engineering</b> for generative AI.</p></li><li><p>
Hands-on experience with generative models including <b>text-to-text, text-to-image, text-to-audio, video-to-video, video-to-audio, image-to-video</b>, and others.</p></li><li><p>
Deep understanding of the AI model landscape and emerging trends.</p></li><li><p>
Practical experience in <b>training Low-Rank Adaptation (LoRA) models</b>.</p></li><li><p>
Experience in building and configuring agents based on LLMs and other generative models.</p></li><li><p>
Proven expertise in <b>ComfyUI</b>, including creating custom nodes and advanced pipelines.</p></li><li><p>
Background in <b>filmmaking</b> (production, direction, or post-production experience preferred) to ensure outputs meet cinematic storytelling standards.</p></li><li><p>
Strong adaptability and eagerness to learn new technologies quickly.</p></li><li><p>
Excellent communication and collaboration skills across research, engineering, and creative teams.</p></li><li><p>
Willingness to work in a <b>fast-paced, high-intensity startup environment</b> (80+ hours per week).</p></li><li><p>
Intuitive &ldquo;vibe coding&rdquo; mindset to translate creative ideas into generative outputs</p></li></ul><h3 class="jobSectionHeader"><b>
Responsibilities</b></h3><ul><li><p>
Develop, optimize, and maintain complex generative content pipelines across <b>images, video, audio, and text</b>.</p></li><li><p>
Configure parameters for generative models to maximize quality, efficiency, and artistic impact.</p></li><li><p>
Build advanced pipelines in <b>ComfyUI</b>, including developing custom nodes for specific creative workflows.</p></li><li><p>
Troubleshoot and resolve issues across generative processes quickly and effectively.</p></li><li><p>
Collaborate closely with ML, product, and creative teams to integrate new features and enhancements.</p></li><li><p>
Apply expertise in filmmaking to guide generative outputs for cinematic quality and narrative impact.</p></li><li><p>
Continuously optimize existing workflows to improve efficiency and output quality.</p></li></ul><h3 class="jobSectionHeader"><b>
Benefits</b></h3><ul><li><p>
Quarterly Bonuses</p></li><li><p>
Competitive compensation and comprehensive benefits package</p></li><li><p>
Medical Insurance (IPD/OPD)</p></li><li><p>
Company Sponsored Lunch</p></li><li><p>
Free Platform Subscriptions</p></li><li><p>
Yearly Performance-Based Increments</p></li><li><p>
Paid Leaves</p></li><li><p>
EOBI</p></li></ul><h3 class="jobSectionHeader"><b>
Company Culture</b></h3><ul><li><p><b>
Impact:</b> Opportunity to work on innovative AI products like <b>ImagineArt</b>, shaping the future of creativity and user interaction.</p></li><li><p><b>
Team:</b> Be part of a passionate, fast-moving team that values innovation and data-driven decisions.</p></li><li><p><b>
Growth:</b> A culture where learning, growth, and experimenting with new ideas are deeply encouraged.</p></li></ul></div><p></p>

AI Video Prompt Specialist

Capgemini

<div><div><div><p>Hyderabad, Pune, Bangalore</p><div>
Data Engineer(Python with Gen AI)</div><p>
</p></div></div><div></div><div><div><div><div><div><p>
At Capgemini Invent, we believe difference drives change. As inventive transformation consultants, we blend our strategic, creative and scientific capabilities, collaborating closely with clients to deliver cutting-edge solutions. Join us to drive transformation tailored to our client's challenges of today and tomorrow. Informed and validated by science and data. Superpowered by creativity and design. All underpinned by technology created with purpose.</p><div><div><div><div><b>
Your Role</b></div></div><div><ul><li>
We are seeking an exceptional Data Scientist with specialized expertise in developing multi-agent AI systems.
</li><li>In this role, you will design, implement, and optimize complex AI ecosystems where multiple intelligent agents collaborate to solve sophisticated problems.
</li><li>You will leverage your deep understanding of generative AI, retrieval-augmented generation (RAG), and prompt engineering to create cutting-edge solutions that push the boundaries of artificial intelligence</li></ul></div></div><div><div><div><b>
Your Profile</b></div></div><div><ul><li>
Design and develop generative AI-based multi-agent systems that can collaborate, communicate, and coordinate to achieve complex objectives</li><li>
Architect and implement RAG-based chatbot solutions that effectively leverage knowledge bases and external data sources</li><li>
Create sophisticated prompt engineering strategies to optimize AI agent behavior and inter-agent communication</li><li>
Build, train, and fine-tune generative AI models for various applications within multi-agent systems</li><li>
Develop robust evaluation frameworks to measure and improve multi-agent system performance</li><li>
Implement efficient knowledge sharing mechanisms between AI agents</li></ul></div></div><div><div><div><b>
What you'll love about working here</b></div></div><div><ul><li>
We recognize the significance of flexible work arrangements to provide support. Be it remote work, or flexible work hours, you will get an environment to maintain healthy work life balance.</li><li>
At the heart of our mission is your career growth. Our array of career growth programs and diverse professions are crafted to support you in exploring a world of opportunities.</li><li>
Equip yourself with valuable certifications in the latest technologies such as Generative AI.</li></ul></div></div></div><p>
Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.</p></div></div><div><div><div>
<div><div>Ref. code</div><div>
307089-en_GB</div></div><div><div>
Posted on</div><div>
11 Aug 2025</div></div><div><div>
Experience level</div><div>
Experienced Professionals</div></div><div><div>
Contract type</div><div>
Permanent</div></div><div><div>
Location</div><div>
Hyderabad, Pune, Bangalore</div></div><div><div>
Business unit</div><div>
INVENT</div></div><div><div>
Brand</div><div>
Capgemini Invent</div></div><div><div>
Professional communities</div><div>
Data &amp; AI</div></div></div></div></div></div></div></div></div><p></p>

Data Engineer(Python with Gen AI)

ATV Tech Solutions

<p><b>(Senior)Data Scientist &ndash; AI / ML / Generative AI</b></p><p><b>Company</b></p><p>ATV Tech Solutions</p><p><b>Experience</b></p><p>5&ndash;8 Years (Relevant Industry Experience)</p><p><b>Location</b></p><p>Jaipur &ndash; On-site (Mandatory)</p><p><b>Employment Type</b></p><p>Full-time</p><p><b>About the Role</b></p><p>ATV Tech Solutions is looking for an experienced <b>(Senior)Data Scientist &ndash; AI / ML / Generative AI</b> to design, develop, and deploy production-grade AI solutions. This role focuses on <b>Machine Learning, Deep Learning, and Generative AI</b>, including LLM-based systems and scalable ML pipelines.</p><p>You will work closely with engineering and business teams to deliver high-impact, data-driven and AI-powered solutions.</p><p><b>Key Responsibilities</b></p><ul><li>Design, develop, and deploy <b>end-to-end AI/ML solutions</b> for real-world business problems</li><li>Build and optimize <b>Generative AI and LLM-based applications</b></li><li>Implement <b>RAG pipelines</b>, embeddings, prompt engineering, and fine-tuning</li><li>Perform <b>data analysis, feature engineering, model training, validation, and evaluation</b></li><li>Work with large-scale structured and unstructured datasets</li><li>Build, deploy, and maintain <b>scalable ML pipelines</b></li><li>Monitor model performance and drive continuous improvement</li><li>Collaborate with cross-functional teams for production integration</li><li>Mentor junior team members and contribute to AI best practices</li></ul><p><b>Required Skills &amp; QualificationsProgramming &amp; Libraries</b></p><ul><li>Python</li><li>Pandas, NumPy</li><li>Scikit-learn</li><li>PyTorch, TensorFlow</li></ul><p><b>Machine Learning &amp; AI</b></p><ul><li>5&ndash;8 years of hands-on experience in <b>Data Science / AI / Machine Learning</b></li><li>Strong foundation in <b>statistics, probability, and machine learning algorithms</b></li><li>Experience with classical ML, NLP, and deep learning techniques</li><li>Strong experience with <b>Generative AI and Large Language Models (LLMs)</b></li><li>Hands-on knowledge of <b>RAG architectures, embeddings, prompt engineering, and fine-tuning</b></li></ul><p><b>Data, Cloud &amp; MLOps</b></p><ul><li>Strong SQL and data handling skills</li><li>Experience with <b>AWS / Azure / GCP / Snowflake</b></li><li>Practical exposure to <b>MLOps</b> (model versioning, CI/CD, deployment, monitoring)</li><li>Experience deploying and maintaining <b>production ML systems</b></li></ul><p><b>Good to Have</b></p><ul><li>Experience building <b>enterprise-scale GenAI solutions</b></li><li>Knowledge of <b>model explainability, responsible AI, and AI governance</b></li><li>Prior mentoring or technical leadership experience</li></ul><p><b>What We Offer</b></p><ul><li>Opportunity to work on <b>cutting-edge AI and Generative AI projects</b></li><li>High-impact, production-focused problem solving</li><li>Collaborative and growth-oriented work culture</li><li>Competitive compensation (industry-aligned)</li><li>Strong career growth in AI &amp; GenAI</li></ul><p><b>How to Apply</b></p><p>Send your resume, GitHub, or project portfolio to <b>hello@atvtechsolutions.com</b> with the subject line: &ldquo;Application &ndash; Data Scientist (Jaipur)&rdquo;</p><p>Job Types: Full-time, Permanent</p><p>Pay: ₹600,000.00 - ₹1,200,000.00 per year</p><p>Ability to commute/relocate:</p><ul><li>Jaipur, Rajasthan: Reliably commute or planning to relocate before starting work (Required)</li></ul><p>Education:</p><ul><li>Bachelor's (Required)</li></ul><p>Experience:</p><ul><li>Data science: 5 years (Required)</li></ul><p>Work Location: In person</p>

(Senior)Data Scientist

Kyndryl

<div><p><b>Who We Are
</b></p><p>At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward &ndash; always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.<br>
</p><p><br>
<b>The Role
</b></p><p>We are seeking a skilled Data Scientist with a minimum of 6 years of experience in Data Science and Machine Learning, preferably with experience in NLP, Generative AI, LLMs, MLOps, Optimization techniques, and AI solution Architecture. In this role, you will play a key role in the development and implementation of GenAI solutions, leveraging your technical expertise. The ideal candidate should have familiarity with business processes in 1 domain(.e.g Financial, Telecom, Retail) to quickly understand the requirements and come up with solutions to address the issues in these domains. Candidate is expected to have a deep understanding of AI technologies and experience in designing and implementing cutting-edge AI systems. Additionally, expertise in data engineering, DevOps, and MLOps practices will be valuable in this role.
<br>
<br>
<b>Responsibilities<br>
</b><br>
</p><ul><li>Development and deploy Generative AI and LLM projects, ensuring alignment with business objectives.
</li><li>Participate in the development of gen AI solutions, AI Agent development
</li><li>Design and deploy Proof of Concepts (POCs) and Points of View (POVs) across various industry verticals, demonstrating the potential of Generative AI applications.
</li><li>Engage effectively with customers, showcasing and demonstrating the relevance and impact of Generative AI applications in their businesses.
</li>Collaborate with cross-functional teams to integrate AI/ML solutions into cloud environments (Azure, GCP, AWS, etc.).<br>
<li>
</li></ul><p><br>
<b>Who You Are<br>
 </b></p><p></p><p><b>Required Skills:
</b></p><ul><li>Strong understanding of Transformers, LLMs, Fine Tuning, Agents and RAG techniques.
</li><li>Comfortable with development of AI Agents
</li><li>PhD or equivalent with 6-8 years of experience in the IT Industry.
</li><li>Minimum 4 years of hands on experience Data Scientist Role or Similar role.
</li><li>Experience in AI/ML, with a focus on Generative AI and Large Language Models. Proven track record in working with major cloud platforms (Azure, GCP, AWS).
</li><li>Understanding of how to deploy LLMs on cloud/ on-premise and use APIs to build Industry solutions.
</li><li>Experience in using LLM models on cloud i.e. OpenAI @ Azure, Aws Bedrock, GCP Vertex AI.
</li><li>Should have done 2-3 projects in domain of expertise,
</li><li>Should have strong good knowledge of Python
</li><li>Be fluent in English (written and spoken).
 </li></ul><p></p><p><b>Preferred Skills:
</b></p>Should have strong good knowledge of C/C++<br>
<ul><li>
</li></ul><p><br>
<b>Being You
</b></p><p>Diversity is a whole lot more than what we look like or where we come from, it&rsquo;s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we&rsquo;re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you &ndash; and everyone next to you &ndash; the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That&rsquo;s the Kyndryl Way.<br>
</p><p><br>
<b>What You Can Expect
</b></p><p>With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter &ndash; wherever you are in your life journey. Our employee learning programs give you access to the best learning in the industry to receive certifications, including Microsoft, Google, Amazon, Skillsoft, and many more. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed.
</p><p></p><p><b>Get Referred!
</b></p><p>If you know someone that works at Kyndryl, when asked &lsquo;How Did You Hear About Us&rsquo; during the application process, select &lsquo;Employee Referral&rsquo; and enter your contact's Kyndryl email address.</p></div>

Data Scientist

Uber AI Solutions

Uber AI Solutions is Uber's new marketplace connecting freelancers with Generative AI researchers. We're inviting experienced domain professionals to collaborate on a new client project at the frontier of GenAI. This is a freelance, paid, project-based opportunity - remote, flexible, and designed for experienced professionals who want to apply their expertise in finance, engineering, physics, chemistry, or medicine to advance Generative AI systems. Project details Duration: Tasks for this project are expected to be available for 3 months (with potential extensions). You'll retain flexibility to accept tasks of your choice. Location: Remote (You must be based in the United States) Payout structure: Task-based pay model. Competitive rates per completed task, determined by the complexity and required experience. Type: Freelance / Independent contractor. Why this matters Your expertise may help train and evaluate the next generation of Generative AI systems. By shaping the data and benchmarks these models rely on, you'll help ensure that Generative AI is accurate, reliable, and aligned with the standards of your field. This is a unique opportunity to apply your domain knowledge at the cutting edge of technology. Contact email

Freelance STEM Prompt Generation Professional

<div><p><b>Who We Are
</b></p><p>At Kyndryl, we design, build, manage and modernize the mission-critical technology systems that the world depends on every day. So why work at Kyndryl? We are always moving forward &ndash; always pushing ourselves to go further in our efforts to build a more equitable, inclusive world for our employees, our customers and our communities.<br>
</p><p><br>
<b>The Role
</b></p><p>We are seeking a skilled Data Scientist with a minimum of 6 years of experience in Data Science and Machine Learning, preferably with experience in NLP, Generative AI, LLMs, MLOps, Optimization techniques, and AI solution Architecture. In this role, you will play a key role in the development and implementation of GenAI solutions, leveraging your technical expertise. The ideal candidate should have familiarity with business processes in 1 domain(.e.g Financial, Telecom, Retail) to quickly understand the requirements and come up with solutions to address the issues in these domains. Candidate is expected to have a deep understanding of AI technologies and experience in designing and implementing cutting-edge AI systems. Additionally, expertise in data engineering, DevOps, and MLOps practices will be valuable in this role.
<br>
<br>
<b>Responsibilities<br>
</b><br>
</p><ul><li>Development and deploy Generative AI and LLM projects, ensuring alignment with business objectives.
</li><li>Participate in the development of gen AI solutions, AI Agent development and Elastic ML deliverables
</li><li>Design and deploy Proof of Concepts (POCs) and Points of View (POVs) across various industry verticals, demonstrating the potential of Generative AI applications.
</li><li>Engage effectively with customers, showcasing and demonstrating the relevance and impact of Generative AI applications in their businesses.
</li>Collaborate with cross-functional teams to integrate AI/ML solutions into cloud environments (Azure, GCP, AWS, etc.).<br>
<li>
</li></ul><p><br>
<b>Who You Are
</b></p><p><b>Required Skills:
 </b></p><p></p><ul><li>PHD/ Postgraduate in computer science or computer engineering or equivalent with 6-8 years of experience in the IT Industry.
</li><li>Minimum 4 years of hands on experience Data Scientist Role or Similar role.
</li><li><b>Strong understanding of Transformers, LLMs, Fine Tuning, Agentic AI and RAG techniques.
</b></li><li><b>Comfortable with development of AI Agents
</b></li><li>Experience in AI/ML, with a focus on Generative AI and Large Language Models. Proven track record in working with major cloud platforms (Azure, GCP, AWS).
</li><li>Understanding of how to deploy LLMs on cloud/ on-premise and use APIs to build Industry solutions.
</li><li>Experience in using LLM models on cloud i.e. OpenAI @ Azure, Aws Bedrock, GCP Vertex AI.
</li><li>Should have done 2-3 projects in domain of expertise,
</li><li>Should have strong good knowledge of Python.
</li><li>Be fluent in English (written and spoken).
</li></ul><p></p><p><b>Preferred Skills:
</b></p>Should have strong good knowledge of C/C++.<br>
<ul><li>
</li></ul><p><br>
<b>Being You
</b></p><p>Diversity is a whole lot more than what we look like or where we come from, it&rsquo;s how we think and who we are. We welcome people of all cultures, backgrounds, and experiences. But we&rsquo;re not doing it single-handily: Our Kyndryl Inclusion Networks are only one of many ways we create a workplace where all Kyndryls can find and provide support and advice. This dedication to welcoming everyone into our company means that Kyndryl gives you &ndash; and everyone next to you &ndash; the ability to bring your whole self to work, individually and collectively, and support the activation of our equitable culture. That&rsquo;s the Kyndryl Way.<br>
</p><p><br>
<b>What You Can Expect
</b></p><p>With state-of-the-art resources and Fortune 100 clients, every day is an opportunity to innovate, build new capabilities, new relationships, new processes, and new value. Kyndryl cares about your well-being and prides itself on offering benefits that give you choice, reflect the diversity of our employees and support you and your family through the moments that matter &ndash; wherever you are in your life journey. Our employee learning programs give you access to the best learning in the industry to receive certifications, including Microsoft, Google, Amazon, Skillsoft, and many more. Through our company-wide volunteering and giving platform, you can donate, start fundraisers, volunteer, and search over 2 million non-profit organizations. At Kyndryl, we invest heavily in you, we want you to succeed so that together, we will all succeed.
</p><p></p><p><b>Get Referred!
</b></p><p>If you know someone that works at Kyndryl, when asked &lsquo;How Did You Hear About Us&rsquo; during the application process, select &lsquo;Employee Referral&rsquo; and enter your contact's Kyndryl email address.</p></div>

GenAI & Agentic AI Developer

Kore.ai

<div><p>Kore.ai is a pioneering force in enterprise AI transformation, empowering organizations through our comprehensive agentic AI platform. With innovative offerings across &quot;AI for Service,&quot; &quot;AI for Work,&quot; and &quot;AI for Process,&quot; we're enabling over 400+ Global 2000 companies to fundamentally reimagine their operations, customer experiences and employee productivity.</p><p>
Our end-to-end platform enables enterprises to build, deploy, manage, monitor, and continuously improve agentic applications at scale. We've automated over 1 billion interactions every year with voice and digital AI in customer service, and transformed employee experiences for tens of thousands of employees through productivity and AI-driven workflow automation.</p><p>
Recognized as a leader by Gartner, Forrester, IDC, ISG, and Everest, Kore.ai has secured Series D funding of $150M, including strategic investment from NVIDIA to drive Enterprise AI innovation. Founded in 2014 and headquartered in Florida, we maintain a global presence with offices in India, UK, Germany, Korea, and Japan.</p><p>
You can find full press coverage at https://kore.ai/press/</p><p></p><p><b><br>
POSITION: </b><b>Senior AI Evaluation Specialist</b></p><p></p><p><b><br>
POSITION SUMMARY: </b>We are seeking a <b>Senior AI Evaluation Specialist</b> to design and execute robust evaluation methodologies for Generative and Agentic AI systems. This role bridges <b>AI product quality, evaluation science, and responsible AI governance</b> &mdash; ensuring every AI feature, agent, and model release is measured, benchmarked, and validated using standardised frameworks.</p><p>
The ideal candidate combines a <b>QA mindset</b>, <b>ML evaluation rigour</b>, and <b>hands-on coding expertise</b> to benchmark LLMs, multi-agent workflows, and GenAI APIs, driving consistent, measurable, and safe AI product performance.</p><p></p><p><b><br>
LOCATION: Hyderabad (Work from Office)</b></p><p></p><p><b><br>
RESPONSIBILITIES:</b></p><p></p><p><b><br>
1. AI Evaluation &amp; Benchmarking</b></p><p>
Build and maintain <b>end-to-end evaluation pipelines</b> for Generative and Agentic AI features (e.g., chat, reasoning agents, RAG workflows, summarization, classification).</p><p>
Implement standardized evaluation frameworks such as <b>RAGAS, G-Eval, HELM, PromptBench, MT-Bench</b>, or <b>custom evaluation harnesses</b>.</p><p>
Define and measure core <b>AI quality metrics</b> &mdash; accuracy, groundedness, coherence, contextual recall, hallucination rate, and response time.</p><p>
Create reproducible <b>benchmarks, leaderboards, and regression tracking</b> for models and agents across multiple releases or providers (OpenAI, Anthropic, Mistral, etc.).</p><p></p><p><b><br>
2. Agentic AI Evaluation</b></p><p>
Evaluate <b>multi-agent systems</b> and <b>autonomous AI workflows</b>, measuring task success rates, reasoning trace quality, and tool-use efficiency.</p><p>
Assess <b>Agentic AI behaviors</b> such as planning accuracy, goal completion rate, context handoff success, and inter-agent communication reliability.</p><p>
Validate <b>decision-making transparency</b> and <b>error recovery</b> mechanisms in autonomous agent frameworks (LangGraph, AutoGen, CrewAI, etc.).</p><p>
Design agent-specific evaluation scenarios &mdash; simulated environments, user-in-the-loop testing, and &ldquo;mission-based&rdquo; performance scoring.</p><p></p><p><b><br>
3. Experimentation &amp; Automation</b></p><p>
Develop <b>Python-based evaluation scripts</b> to automate testing using OpenAI, Anthropic, and Hugging Face APIs.</p><p>
Conduct large-scale comparative studies across <b>prompts, models, and fine-tuned variants</b>, analyzing quantitative and qualitative differences.</p><p>
Integrate evaluations into CI/CD pipelines to enable <b>continuous AI quality monitoring</b>.</p><p>
Visualize results using dashboards (Plotly, Streamlit, Dash, or Grafana).</p><p></p><p><b><br>
4. Quality Governance &amp; Reporting</b></p><p>
Define and enforce <b>AI acceptance thresholds</b> before deployment.</p><p>
Collaborate with <b>Responsible AI teams</b> to evaluate bias, fairness, safety, and privacy implications.</p><p>
Produce detailed <b>evaluation reports and audit logs</b> for model releases and governance boards.</p><p>
Present findings to Product, Data Science, and Executive stakeholders &mdash; transforming metrics into actionable insights.</p><p></p><p><b><br>
5. Collaboration &amp; Continuous Improvement</b></p><p>
Work closely with <b>Prompt Engineers, ML Scientists, and QA Engineers</b> to close the loop between testing and improvement.</p><p>
Support Product teams in defining <b>evaluation-driven release criteria</b>.</p><p>
Mentor junior evaluators in AI testing methodologies, benchmarking, and analysis.</p><p>
Keep abreast of advances in <b>LLM evaluation research</b>, <b>Agentic AI frameworks</b>, and <b>tool-calling reliability testing</b>.</p><p><b><br>
QUALIFICATIONS / SKILLS REQUIRED:</b></p><p></p><div><div><div><p><b><br>
Category</b></p><p><b><br>
Expected Expertise</b></p></div><div><p><b><br>
Programming</b></p><p><br>
Python (Pandas, NumPy, LangChain, LangGraph, OpenAI/Anthropic SDKs)</p></div><div><p><b><br>
Evaluation Frameworks</b></p><p><br>
RAGAS, HELM, G-Eval, MT-Bench, PromptBench, custom scoring pipelines</p></div><div><p><b><br>
GenAI APIs</b></p><p><br>
OpenAI GPT-4/5, Claude, Gemini, Mistral, Azure OpenAI</p></div><div><p><b><br>
Agentic AI</b></p><p><br>
Understanding of multi-agent orchestration, tool use, reasoning traces, and planning frameworks (AutoGen, CrewAI, LangGraph)</p></div><div><p><b><br>
Metrics Knowledge</b></p><p><br>
BLEU, ROUGE, cosine similarity, factuality, coherence, bias, toxicity, reasoning success rate</p></div><div><p><b><br>
Data &amp; Analytics</b></p><p><br>
JSON parsing, prompt dataset curation, result visualization</p></div><div><p><b><br>
Tooling</b></p><p><br>
Git, Jupyter/Colab, Jira, Confluence, evaluation dashboards</p></div><div><p><b><br>
Soft Skills</b></p><p><br>
Analytical communication, documentation excellence, cross-team collaboration</p></div></div></div><p></p><p><b><br>
EDUCATION QUALIFICATION:</b></p><ul><li>
Bachelor&rsquo;s or Master&rsquo;s degree in Computer Science, AI, Data Science, or related discipline.</li><li>
5 to 10 years total experience with at least 3+ years in <b>AI evaluation, GenAI QA, or LLM quality analysis</b>.</li><li>
Strong understanding of <b>AI/ML model lifecycle</b>, <b>prompt engineering</b>, and <b>RAG or agentic architectures</b>.</li><li>
Experience contributing to AI safety, reliability, or responsible AI initiatives.</li></ul></div><p></p>

AI Evaluation Specialist - QA

Search 8643+ Generative ai specialist jobs near you this January. New roles added regularly across leading industries. Apply today.