Introduction
Tracking the performance and usage of agents is an important part of the agent-building process.Every week, you’ll have thousands of questions and answers, themes, and insights to learn from and improve your agents. Mindset AI uses ThoughtSpot, a Business Intelligence (BI) tool embedded into the platform, to provide these valuable analytics and insights on AI agents.
What is ThoughtSpot?
ThoughtSpot is a BI system that provides enterprise features. Specifically, you are provided with:- Liveboard: A collection of related answers or search results that appear as tables or visualizations.
- Visualizations: A visual representation of data. In ThoughtSpot, it is the representation of an answer. In some cases, the terms visualization and answer are used interchangeably.
Reporting
Your Dashboard space includes three essential liveboards for tracking performance and user engagement:Daily Active Users
This liveboard provides comprehensive user analytics, including user breakdown, engagement patterns, and growth trends over time. Use this data to answer key questions about your user base:- How many users do I have?
- How have my user numbers grown over time?
- How many users are returning?
- How active are my users?
- What actions are they performing?
Agent Insights
This liveboard offers detailed analytics across the Agents feature, covering engagement metrics, prompting effectiveness, thread analysis, and additional performance indicators to help you optimize your Agent configuration.Message Themes
This liveboard groups messages together by themes, split into weekly and daily views. It refreshes on a regular schedule to help you identify the most common themes across your users.Measure and Learn
Regular data analysis is essential for understanding user behavior and identifying trends.While knowing what users ask and how often they engage is valuable, the real insight comes from deeper analysis and hypothesis formation around user actions and behaviors.
Raw User Messages
Pay attention to instances where the Agent declines to answer questions.These interactions provide valuable context about user needs and help you identify gaps in your content that may require attention.
User Sentiment
Monitor user satisfaction with Agent responses to understand where improvements are needed. User sentiment is a key indicator of when to iterate on your Agent configuration, helping you balance response quality with user expectations.Sources, Segments, and Citations
Identify the most and least popular content sources users select to understand which content is most effective.This also highlights underused content that may need updating or reworking.