AI Agent Reporting helps you understand how an AI Agent processes customer messages and generates responses. It provides visibility into agent performance at both the agent and message levels, including execution steps such as LLM and tool calls.

The report captures key metrics such as:

• Latency
• Token consumption
• Guardrail violations
• Fallback behaviour
• Routing actions

Use these insights to analyse agent performance, troubleshoot issues, and understand how individual customer interactions are handled.

AI Agent Containment Metrics

Track containment rate, case volume, and agent handling metrics across all AI Agents from a single dashboard. These metrics are available as standard reporting metrics.

Create a Containment Dashboard

1. Select New Tab.
2. Under Sprinklr Service, go to Analyze > Care Reporting.
3. Open an existing dashboard or create a new one.
4. Select + Add Widget.

5. In the

   Create Custom Widget

   window:

   • Enter a widget name.
   • Select Service Analytics as the data source.

6. Add the following metrics:

   • Bot Containment Rate
   • Case Volume
   • Cases Handled by Human Agent
7. Select Add to Dashboard.

Turn-Level Reporting

A single customer message can trigger multiple execution steps before the AI Agent generates a response. Turn-Level Reporting provides visibility into each step while maintaining message and case context.

Available report fields include:

• Case Number
• Message
• Task Name
• Bot Application
• Step Type
• Step Name
• Guardrail Execution Status

The following fields will be available in a future release:

• CSAT
• Guardrail Name
• Latency

For example, a customer message may trigger an LLM step, followed by a RAG retrieval step and a tool execution step. Each step is reported separately to help you understand how the request was processed.

Task-Level Metrics (Upcoming Release)

Task-Level Latency

Monitor average task latency across customer conversations. If latency increases, you can investigate individual execution steps to identify the source of the delay, including tool calls and knowledge retrieval operations.

Task-Level Containment

Task-level containment reporting helps identify high-volume tasks with low containment rates, making it easier to prioritise optimisation efforts.

Create a Task Containment Report

1. Follow the dashboard creation steps described in AI Agent Containment Metrics.

2. Add the following columns:

   • Task Name
   • Containment Rate
   • Total Volume

Step Types

The Step Type field identifies the action performed by the AI Agent after receiving a customer message.

Tool-Level Metrics (Upcoming Release)

Tool-level reporting helps identify frequently used tools, measure execution latency, and analyse error rates.

Create a Tool-Level Report

1. Follow the dashboard creation steps described in AI Agent Containment Metrics.

2. Add the following metrics:

   • Step Type
   • Executions
   • Message
   • Step Duration (ms)
   • Error Rate

Use these metrics to identify tools that contribute most to latency or execution failures.

Token and Cost Metrics

Turn-Level Reporting provides visibility into language model usage and estimated costs.

Use these metrics to monitor token consumption and manage LLM-related costs.

Guardrail Reporting (Upcoming Release)

Guardrail Reporting provides visibility into guardrail activity at the message level.

You can:

• Identify which guardrails are triggered most frequently.
• Analyse messages affected by guardrail enforcement.
• Determine whether a customer message or AI Agent response was blocked by a guardrail.

When a guardrail is triggered, the report records the corresponding guardrail for analysis.

RAG Reporting

RAG Reporting provides insights into retrieval performance and knowledge usage within AI Agent interactions.

Access RAG Reports

1. Follow the dashboard creation steps described in AI Agent Containment Metrics.
2. Open the Smart FAQ Report from the AI Agent Turn Reporting dashboard.