​
Introduction

Tracking the performance and usage of agents is an important part of the agent-building process. Every week, you will have thousands of questions and answers, themes, and insights to learn from and improve your agents. Mindset AI uses ThoughtSpot, a Business Intelligence (BI) tool embedded into the platform, to provide these valuable analytics and insights on AI agents.

​
What is ThoughtSpot?

ThoughtSpot is a BI system that provides enterprise features. Specifically, you are provided with:
  • Liveboard: A collection of related answers or search results that appear as tables or visualizations.
  • Visualizations: A visual representation of data. In ThoughtSpot, it is the representation of an answer. In some cases, the terms visualization and answer are used interchangeably.

​
Reporting

Your Dashboard space includes three essential liveboards for tracking performance and user engagement.

​
Daily Active Users

This liveboard provides comprehensive user analytics, including user breakdown, engagement patterns, and growth trends over time. Use this data to answer key questions about your user base:
  • How many users do I have?
  • How have my user numbers grown over time?
  • How many users are returning?
  • How active are my users?
  • What actions are they performing?

​
Agent Insights

This liveboard offers detailed analytics across the Agents feature, covering engagement metrics, prompting effectiveness, thread analysis, and additional performance indicators to help you optimize your Agent configuration.

​
Message Themes

This liveboard groups messages together by themes, split into weekly and daily themes, and refreshes on a weekly and daily schedule to help you identify the most common themes across your users.

​
Measure and Learn

Regular data analysis is essential for understanding user behavior and identifying trends. While knowing what users ask and how frequently they engage is valuable, the real insight comes from deeper analysis and hypothesis formation around user actions and behaviors.

​
Raw User Messages

Pay attention to instances where the Agent declines to answer questions. These interactions provide valuable context about user needs and help you identify gaps in your content that may require attention.

​
User Sentiment

Monitor user satisfaction with Agent responses to understand where improvements are needed. User sentiment is a key indicator of when to iterate on your Agent configuration, helping you balance response quality with user expectations.

​
Sources, Segments, and Citations

Identify the most and least popular content sources users select to understand which content is most effective. This also helps highlight content that is underused and may need updating or reworking.

​
Message Themes

Analyze message themes to identify the most common topics across your user base. Comparing these themes to your existing content helps you spot trends and gaps, so you can prioritize content creation and Agent improvements.

​
Data Exporting

Data can be exported easily for deeper analysis. Users can also search and filter results by date range.