Business Systems Analyst - AI and Virtual Agent (Onsite)

Role:

The Business Systems Analyst serves as the organization's subject matter expert for AI-driven interaction platforms, including the Kore.ai virtual voice and digital agent and other automation technologies. This role is responsible for understanding how automated conversations behave in real operational scenarios and ensuring the systems respond accurately, consistently, and in alignment with business intent. The analyst develops deep functional knowledge of conversational flows, intents, training data, and configuration, and translates operational needs into clear requirements for implementation. The role works closely with operations and technology teams to refine automation, improve customer experience, and prevent inconsistent or unintended behavior. This position supports business decision-making but does not establish business policy or product strategy.

Essential Functions & Responsibilities:

• Maintain expert knowledge of conversational AI platforms, including Kore.ai, and related automation technologies.

• Monitor and analyze automated interactions to identify misunderstandings, logic gaps, or incorrect responses.

• Tune conversational behavior to improve intent recognition accuracy and customer experience.

• Ensure consistent automated responses across channels (voice, chat, digital).

• Maintain documentation of conversational logic, decision paths, and behavior standards.

• Track performance metrics including containment rate, fallback rate, and escalation frequency.

• Improve automation accuracy through structured tuning and refinement of logic.

• Translate operational scenarios into clear conversational and functional requirements.

• Define expected system behavior for new intents, enhancements, or workflow changes.

• Provide detailed use cases and acceptance criteria for implementation.

• Validate that automation responses match business expectations.

• Support rollout and testing of new automation capabilities.

• Investigate incorrect or failed automated interactions.

• Determine root cause (training data, configuration, workflow logic, or defect).

• Coordinate resolution between operations and technical teams.

• Guide business teams on automation expectations and system limitations.

• Configure approved changes where applicable.

• Support conversational testing scenarios and validation cycles.

• Evaluate downstream operational impact prior to deployment.

• Maintain change history and behavior documentation.

• Other job duties as assigned.

Performance Measurements:

1. Accuracy & Automation Quality Goals

• Intent Recognition Accuracy – Achieve and maintain target accuracy rates (e.g., 90–95%).

• Reduction in Unhandled Intents and Fallback Rates.

• Improvement in Escalation Accuracy and Proper Routing.

• Response Consistency Across Channels.

2. Customer Experience Goals

• Increased Containment Rate while maintaining satisfaction levels.

• Reduced Member Effort and Confusion.

• Improved Conversational Flow and Natural Dialogue Quality.

• Positive Member Sentiment or CSAT Feedback on Automated Interactions.

3. Operational Effectiveness Goals

• Reduction in automation-related complaints or errors.

• Measurable reduction in operational rework due to incorrect automation.

• Timely implementation of enhancements and updates.

• Effective collaboration with operations, IT, and compliance teams.

4. Continuous Improvement & Optimization

• Regular review and tuning of conversational performance.

• Documented improvement cycles and performance tracking.

• Identification of automation opportunities that improve operational efficiency.

Knowledge and Skills:

Experience: One to three years of similar or related experience in banking, contact center operations, automation platforms, or customer service workflow environments.

Education: (1) A bachelor's degree, or (2) achievement of formal certifications recognized in the industry as equivalent to a bachelor's degree (e.g., information technology certifications in lieu of a degree)

Interpersonal Skills: Work involves regular collaboration with operations, IT, and business stakeholders. Requires diplomacy, discretion, and the ability to communicate clearly across technical and non-technical audiences.

Other Skills:

• Strong analytical thinking and problem-solving ability.

• Ability to understand how users interact with systems and why failures occur.

• Strong written communication and documentation skills.

• Experience with conversational AI platforms preferred (Kore.ai, Dialogflow, Amazon Lex, Nuance, etc.).

• Experience writing decision logic, conversational flows, or scripts preferred.

• QA, UAT, or system configuration experience preferred.

• Understanding of customer interaction journeys.

• Basic familiarity with data analysis or reporting tools preferred.

• Ability to manage multiple projects with limited supervision.

• Strong attention to detail and ability to anticipate edge cases and user misunderstandings.

Physical Requirements:

While performing the duties of the job, the employee is regularly required to:

• Position yourself to move about the work area.
• Ability to sit for at least 4 hours at a time.
• Reach, grab, sort.
• Communicate in person and via telephone with others.
• Identify and recognize small font communication.
• Ability to transport a minimum of 10 pound.