Functional Reporting For RAG
Updated
Functional Reporting for Retrieval-Augmented Generation (RAG) provides visibility into how AI Agents retrieve and use knowledge to generate responses. It captures key details from each RAG interaction, helping administrators understand retrieval behaviour, validate response grounding, and analyse the knowledge sources contributing to generated answers.
These reporting metrics enable organisations to audit AI Agent performance, troubleshoot knowledge retrieval issues, and build dashboards for operational monitoring and analysis.
Available Reporting Metrics
The following metrics are available for RAG reporting:
Metric | Description |
User Message | The original customer query submitted to the AI Agent. |
Reworded Question | The internally rewritten query used to improve knowledge retrieval accuracy. |
Answer | The final response generated by the AI Agent. |
Message ID | The unique identifier associated with the generated response. |
Case Number | The unique identifier of the case associated with the interaction. |
Engine ID | The identifier of the retrieval engine used to process the request. |
Response Grounded | Indicates whether the generated response was grounded using retrieved knowledge. |
KB Articles | Lists the knowledge base articles referenced during response generation. Multiple values may be returned. |
KB Article IDs | Displays the unique identifiers of the referenced knowledge base articles. Multiple values may be returned. |
Output Guardrail Time (ms) | Measures the time taken by output guardrails to evaluate the generated response before it is returned to the customer. |
Add RAG Reporting Metrics to a Dashboard
To add RAG reporting metrics to a Care Reporting dashboard:
- Click the New Tab icon.
- Under Sprinklr Service, select Care Reporting from the Analyze section.
- Open the dashboard where you want to add the metrics.
- Click + Add Widget in the upper-right corner.
On the Create Custom Widget window:
- Enter a Widget Name.
- Select Service Analytics as the Data Source.
Under Select Metric/Dimension, add the following metrics:
- User Message
- Reworded Question
- Answer
- Response Grounded
- Message ID
- Case Number
- Engine ID
- KB Articles
- KB Article IDs
- Output Guardrail Time (ms)
- Review the widget preview displayed in the right pane.
- Click Add to Dashboard.
The selected metrics are added to the dashboard and begin displaying RAG execution data based on the applied reporting filters.