What is Content Coverage Analysis?
Content Coverage Analysis, CCA, is the way to comb through your data properly and understand what the "reality" is in your data.
Why do we need Content Coverage Analysis?
The purpose of performing CCA is to uncover the reasons why customers are contacting customer support. Using those findings, you can create new intents to ensure that the bot can understand and handle the majority of incoming customer requests.
The content coverage analysis process can be used to build and maintain the bot, by validating frequently asked questions and identifying the relevant intents. As processes, seasons, and customer trends change, it is important to perform content coverage analysis regularly, to ensure your bot can understand and handle the most repetitive intents.
How to perform Content Coverage Analysis?
- Select the bot you want to perform content coverage analysis on.
- Click Conversation Logs on the left side pane
- Select a timeframe in the top right corner
- Read through 100 conversations one by one, focusing on the first meaningful message of each conversation, i.e. the message where customers clearly state their contact reason.
- If an intent for the message does not exist yet, tag it with a label such as “Non-existing” or a topic that best describes it with the same process.
- If an intent for the message exists in your bot, tag the conversation with a label such as "Existing"
- To create a label, hover over the conversation, click on the label icon, type "Existing", then click on the plus sign.
- Each label only needs to be created once.
- You can have multiple labels per conversation
- Confirm your current content coverage percentage, by filtering conversations with the label “Existing".
- Content coverage between 60% and 80% can be viewed as a sufficient baseline for a functional AI solution.
Creating new intents from your Content Coverage Analysis
It is important to document your findings by using labels. After you have completed the process, filter for conversations with the label "Non-existing" and assess whether any of these meaningful messages appear multiple times. If they do, it may be worth adding a new intent for them.
You can also tag potential and most repetitive new intents from filtered conversations by using a label such as “New potential”. You can then add the most repetitive “New potentials” as new intents.
The rule of thumb here is that if out of the 100 conversations reviewed and labeled, a topic takes up 10% of it, that's a good indicator to adding an intent for that.