Optimizing a bot means analyzing the bot to determine how to create an even better AI model to make the virtual agent understand better and improve the dialogues to create an even better customer experience.
There are a few areas to check and some strategies to implement to improve performance.
Analytics
The Analytics area is a great place to see where improvements could be done.
Key metrics to focus on are: Bot Understood %, Bot Handled %, Escalation %, Automation % - specifically how they relate to one another, as well as Intent-specific content analytics.
Improve Bot Understanding
Confusion matrix
Have a look at the Confusion Matrix to see if there is significant confusion between Intents. Fix confusion by moving expressions or training intents more to make them stronger.
If there is a lot of confusion between two Intents, we can also think about merging them into one and use conditional blocks to guide the users through different paths inside the dialogue.
One example of this is could be "My account has been locked" vs " I'm locked out of my account". Here, the first expression is about the company locking the customer's account due to many incorrect password attempts or similar whilst the second expression is about the customer having forgotten their password or username.
Another example is courier issues vs issues with my deliveries, two different topics, and answers, which can get easily confused by the bot.
Intents
Make sure you have enough expressions in your Intents overall.
The best practice in a chatbot is 50-300 and in a ticket bot 80-300. The number of expressions should be reflective of the frequency of the Intent. Make sure your intent structure is smart by getting rid of useless Intents if those exist.
Filter in Conversation Logs with not understood, read through them, and get an understanding if it’s a training issue (we have the intents but they don’t get recognized) or are you missing Intents. If that is the case, you can think about creating new ones. Every ‘not understood’ message does not need an intent, but if repetitive ones are found, which could be answered, one should be created.
In a ticket bot, we need to take into consideration the noisiness of incoming messages. With this, we mean forwarded messages, signatures, disclaimers, etc that emails often have. We can do that by using entities to sanitize emails. When we sanitize an email, the noise won't be taken into consideration in the AI model. We also need to make sure the expressions of the intents match the incoming data, so expressions also need to be cleaned up when we start sanitizing incoming messages.
Improve Deflection Rate
Read through conversation logs to see how well dialogues work and pay attention to:
- Do users break dialogues?
If yes, can you add intent listening or free text to guide them through the flow?
They might break a dialogue e.g. not using buttons or asking where to find their order number mid-escalation flow. - Are they not understanding the instructions? Try making the message shorter and easier to follow. People don’t always read long messages.
- Are you missing key information for them to go through the flow, think about what the agent would do in the situation it gets escalated, and add as much of that into the dialogue, if possible?
Activities to improve deflection:
- Adjust the Default Reply to help manage expectations and guide the visitor to flows that cause more confusion.
- Conduct Content Coverage Analysis to identify new potential Intents that can be automated
- Can an API integration automate more conversations? Identify new, suitable use cases for an API to increase automation/deflection rate.
Automation Rate
Resolutions States are a great tool to use to understand where your bot can be performing better, especially when looking at trends over time. Using the states of escalated and not resolved you will be able to identify which conversations are most troublesome, by filtering for them in the Conversation Logs.
Smarter Dialogues
Backend Integrations
Think about adding BE integrations to fetch data the bot could easily provide to users. This is obviously not the case for every customer, but if relevant.
Conditional Blocks
You can also use conditional blocks to jump from one flow to the middle part of another flow, directly providing the correct answer if certain keywords are recognized, set the bot to reply differently the second and third time to make it smarter, reduce repetitiveness and provide less guidance before escalation if they have been through the flow before.
confidence_score
Use native parameter: confidence_score as a fall-back in replies where the bot might feel less confident: in case confidence_score is below 90%, the bot can confirm customer intention in a more straightforward way (“Gotcha. Just to make sure I understood you correctly: you’ve lost your passwords and are wanting to generate a new one. Is that correct?”).
For Ticket Automation bots, in case the bot feels less confident about a topic, you might want to exclude the reply and trigger just the actions instead.
Escalation Templates
To streamline Intent replies it can be better, faster, and easier to manage the escalation process in one centralized place, rather than in each specific flow. This should utilize Operating Hours and Team Availability to manage expectations and escalate in the appropriate manner.
Note - Team availability is not available for every CRM
A/B Testing
A/B testing is a great way to optimize dialogue flows with data-driven decisions. There are a few ways to achieve this - to learn about the different options check out the how-to guide.