Skip to main content

Anomalies and Flags

Automatically detect when things go wrong — without reading every conversation manually.

What This Section Covers​

Optimly’s analytics engine monitors conversations in real time and detects anomalies that signal confusion, frustration, or system inefficiencies.

These are surfaced as flags, allowing teams to act quickly and improve both agent behavior and customer experience.

What Are Flags?​

Flags are internal labels applied to specific messages or sessions when certain conditions are met. They indicate that something may have gone wrong or needs attention.

Each flag is timestamped, associated with an agent and session, and visible in the analytics dashboard.

Types of Flags​

Repetition​

Triggered when a user repeats the same question or intent within the same session.

  • Signals misunderstanding or ineffective response
  • Often linked to missing or ambiguous knowledge

Abandonment After Response​

Flagged when a user leaves the conversation shortly after receiving a reply.

  • May indicate dissatisfaction, confusion, or lack of engagement

High Token Usage​

Flagged when a response uses an unusually high number of tokens compared to the average for that agent or topic.

  • Often tied to verbose or inefficient answers
  • Can signal prompt quality issues or unnecessary elaboration

Empty or Generic Response​

Triggered when the agent produces a vague, generic, or default fallback message.

  • Indicates that the model was not able to generate a useful response

Frustration Detected​

Detected via sentiment and language analysis when the user expresses negative emotions such as:

  • “You’re not helping”
  • “This is useless”
  • “Can I speak to a human?”

This flag can be used to train agents to recognize emotional cues.

Viewing Flags​

You can view flags in:

  • The Raw Conversations dashboard (flag icon per message)
  • The Anomaly summary panel
  • Filtered exports and reports
  • Custom alerts (coming soon)

Use Cases​

  • Quickly review only the problematic sessions
  • Detect content gaps or misunderstood topics
  • Monitor agent regressions after prompt or model updates
  • Trigger human takeover workflows or email alerts
  • Guide training and fine-tuning for new use cases

Next: Document Usage (RAG)