Message-Level Metrics
Drill down into each individual message to understand how your employee performs on a granular level.
What This Section Covers
While overview metrics help you track general trends, message-level analytics let you examine every user-employee exchange in detail. This is crucial for identifying subtle issues like slow responses, misunderstood questions, or inefficient token usage.
Key Metrics per Message
Tokens Used
The number of tokens consumed for both the prompt and the employee’s response.
- Useful for monitoring usage efficiency
- Can help reduce costs by optimizing prompt structure
Latency
The time between receiving a message and sending a response.
- Helps identify performance bottlenecks
- High latency may signal model overload or integration delays
Tool and Document Activation
Shows which tools (e.g., lead form, email handoff) or knowledge documents were triggered in a given response.
- Useful for validating RAG behavior or tool usage
- Helps improve retrieval quality
Flags
System-generated alerts based on heuristics or patterns, such as:
- Repeated questions
- User frustration signals (e.g., “You’re not helping”)
- Abandonment after a confusing reply
Flags help surface conversations that may require review.
Sentiment and Emotion (if enabled)
If emotion tracking is activated, each message is scored for:
- Sentiment: Positive, Neutral, or Negative
- Emotion: Anger, frustration, curiosity, satisfaction, etc.
This can guide tone adjustments or reveal missed expectations.
Response Completeness
An internal evaluation of whether the employee answered the question in full, partially, or not at all — based on heuristics, document usage, or fallback triggers.
Viewing and Filtering
You can explore message-level metrics by:
- Clicking on a conversation in the Raw Conversations view
- Hovering over or selecting a specific message
- Filtering by flags, employee, user ID, or timeframe
Example Use Cases
- Identify which user questions are most costly in tokens
- Spot slow or failed responses that may frustrate users
- Discover patterns of incomplete answers
- Debug conversations where no documents were activated