AI Solutions Analyst

Role Overview:

We are seeking an AI Solutions Analyst who will bridge the gap between business intelligence, data engineering, and AI application teams. The ideal candidate will design and refine how Large Language Models (LLMs), such as GPT, interact with enterprise datasets to deliver accurate, contextual, and compliant business insights.

Responsibilities:



Serve as the link between business, data, and AI teams, ensuring effective interaction between the LLM and modeled datasets.


Define and curate context and knowledge bases to enable GPT to provide relevant, precise, and compliant outputs.


Collaborate with Data Analysts, Engineers, and SMEs to identify, structure, and tag data for AI usage.


Design, test, and refine prompt strategies and context frameworks aligned with business objectives.


Conduct LLM evaluation and performance testing (evals) to validate accuracy, completeness, and contextual relevance.


Develop and maintain an LLM Interaction Design Framework documenting context injection, prompt templates, and retrieval logic.


Partner with IT, Compliance, and Governance teams to ensure ethical, secure, and responsible AI deployment.


Produce reports and dashboards summarizing AI performance, usage trends, and continuous improvement metrics.




Key Deliverables:



LLM Interaction Design Framework: Documentation of GPT connections, context injection, and prompt architecture.


Knowledge Base Configuration: Curated domain and business rules repository for GPT accuracy.


Evaluation Scripts & Reports: Test datasets, scoring criteria, and LLM output analysis.


Prompt Library & Usage Guidelines: Standardized templates for consistent and effective AI interactions.


AI Performance Dashboard: Visual summary of LLM accuracy, response quality, and improvement metrics.


Governance & Compliance Documentation: Inputs to bias prevention, ethical use, and AI control frameworks.