Document Usage (RAG)
Understand how your connected knowledge sources are used — and which ones are being ignored.
What This Section Covers
If your agents are connected to a retrieval-augmented generation (RAG) setup, Optimly tracks which knowledge documents are being used to answer user queries.
This visibility helps ensure that:
- Your most important content is actually being used
- Redundant or unused documents can be removed
- You can continuously improve your knowledge base for better accuracy
What We Track
Document Activation
For each agent response, we record whether it used any documents from your knowledge base — and if so, which ones.
This includes:
- PDFs
- Website content
- Google Docs
- Text entries
- Transcribed videos (YouTube, MP4)
You can see which documents are helping answer questions — and which are never referenced.
Activation Frequency
Optimly calculates how often each document is used over a given time range:
- Total activations per document
- Share of total document usage
- Usage trends over time
This helps you identify high-value vs low-impact content.
Redundancy and Overlap
Documents that are rarely or never activated can be reviewed, merged, or deleted.
When multiple documents cover similar topics, you can:
- Prioritize the clearest and most effective version
- Consolidate fragmented content
- Reduce token usage by trimming the knowledge base
Visualizations and Metrics
- Document usage table with total activations
- Heatmap of usage over time
- Pie chart of % contribution by document
- Unused document list
- Comparison across agents using the same content
Use Cases
- Optimize your RAG setup by focusing on high-performing documents
- Remove outdated or irrelevant files to save tokens and improve precision
- See which product areas get the most questions
- Track adoption of new help content after a product release