Skip to main content

Analytics explained - Chat Automation

  • Updated

This article gives an explanation of what each area in Analytics of your chatbot mean, which includes: 

 

Analytics

Time frame

Always start with selecting the time frame in the top right corner. 

Analytics_-_timeframe.gif

Conversations

The total number of conversations occurred in the selected time frame.

A conversation is one session and each conversation has a platform conversation ID.

❗ If you use Sunshine Conversations, their conversations are timeless, which means one sunshine conversation could translate to multiple conversations and conversation IDs in the ultimate.ai Dashboard.

 

Deflection Rate

Percentage of conversations that were not successfully escalated to an agent, by email, or via custom escalation.

It counts any conversations that were not escalated to an agent, such as:

  • Conversations that starts and ends with the bot only
  • Conversations that use the Default reply
  • Conversations that were escalated but not picked up by any agent (Failed escalation)

Bot Handled Rate

You only see this if you have an Automation Bot.

  • Having Automation Bot means you are using the automation engine to automate your chats or tickets. mceclip2.png

AVG Conversations / Hour

You only see this if you have a Combo Bot.

  • Having a Combo Bot means you are using both the automation engine and suggestion engine. Automation engine is having the bot as a virtual agent to answer your chats and/or tickets. Suggestion engine is the bot giving suggested replies to your agents as opposed to handling the chat for them. mceclip3.png

 

CHAT AUTOMATION AI

Here you can select to view either the ABS# (absolute total numbers) or Rate% (relative rate of messages/conversations out of all visitor messages/conversations in the selected time frame).

Note that the rates are rounded up to the first decimal. So if you need an exact report, look at ABS# instead.

Hover your cursor over the chart to get more details.

mceclip7.png

Messages

  • Visitor messages: Total number of visitor messages sent to the bot
  • Bot Answered (ABS#): Number of visitor messages the bot answered, where the AI confidence is greater than the set threshold
  • Bot Answered Rate (Rate%): Percentage of visitor messages that the bot answered with confidence, our of all visitor messages in the selected time frame
❗ The confidence threshold can be found under in Settings > Bot Settings > General.

Conversations 

  • Bot Conversations: Number of conversations the bot participated in in the selected time frame
 

ABS#

Absolute total numbers

Rate %

Calculated out of all visitor messages/conversations in the selected time frame
Bot Handled Number of fully automated conversations Percentage of fully automated conversations
Agent Escalation Number of conversations that were successfully escalated to an agent Percentage of conversations that were successfully escalated to an agent
Email Escalations Number of conversations that were successfully escalated by email Percentage of conversations that were successfully escalated by email
Failed Escalation

Number of conversations that had a failed escalation attempt

  • Only counts conversations that end with Failed Escalation
  • Conversations escalated successfully after a failed escalation, to an agent or via email/custom escalation, are not counted as failed escalation 
  • If there are multiple failed escalations in one conversation, they will be counted separately
 Percentage of conversations that had a failed escalation attempt, out of all conversations that had an escalation attempt
Custom Escalation Number of conversations that were successfully escalated to a custom destination Percentage of conversations that were successfully escalated to a custom destination
Failed Custom Escalation The number of conversations that had a failed custom escalation attempt, out of all conversations that had an escalation attempt. Percentage of conversations that had a failed custom escalation attempt, out of all conversations that had an escalation attempt

 

AI CONTENT

Intents

  • Intent Messages: Number of visitor messages for which this intent is the top predicted intent
  • First Intent: Number of times this intent was the first intent of this conversation
  • Intent Health: Average confidence of the intent for messages where this intent is the top predicted intent
    • Indicative only for the most frequently used intents
  • Escalations: Number of conversations escalated where this intent was the first meaningful intent
    • Very useful to consider which topics most frequently ultimately end up escalating
  • Expressions: Total number of human-annotated expressions of this intent
  • Replies: Number of replies under an intent
    • Including all Suggestions and Automation replies and their respective reply variations

❗ Hover over an intent and click the three dots to:

  1. View replies - Not available yet 
  2. Edit intent - Takes you directly to Dialogue Builder of that intent
  3. View conversations - Takes you to Conversation Logs and show you all conversations that have triggered that intent

mceclip9.png

 

Dialogues

  • Dialogue: Name of the reply/dialogue
    • When using reply variations, this will show the name of individual dialogue
  • Intent: Name of the intent
    • System / Template Replies will carry an empty field, as those do not belong to an intent by design
  • Started: Number of times this dialogue started
  • Completed: Number of times this dialogue completed
  • Completed Rate: Percentage of completed dialogues, out of all the instances of this dialogue
    • A low completion rate could mean the dialogue flow is too long or unclear, causing visitors to  slide into other intents by accident or leave the chat prematurely without reaching the intended answer

 

 

 

Related articles