In a perfect world, AI would start off working well… perfectly! Sadly though, that’s a rare case.
What usually happens is our bots responses aren’t at the nuanced level a person can provide, yet. So we need to refine, or optimize, its responses so it gets there.
But if it’s your first time with an AI for Zendesk, it can be hard to know exactly what your AI needs and whether you’re making the right decisions.
That’s exactly where AI insights and reporting comes in.
Whether you’re looking into how often your Zendesk support agents are getting accurate policy information, or how many times customers are being routed to a live agent instead, we’ve got you covered.
What is AI reporting, and what insights can I get from it?
When it comes to AI, reporting usually involves finding out how well the AI is performing according to what you’re asking it to do. Good AI platforms give you graphs and figures that you can use to investigate any underperformance, and come up with solutions.
An example of a not-so-great figure might be the number of resolved tickets, when what is considered ‘resolved’ is actually just any ticket that gets closed – like a customer giving up and closing the chat. (Looking at you, Zendesk.)
Usually, you examine several data points such as:
- Number of replies
- Deflection rate
- Gaps in knowledge/training
- ROI
But as we know from observing those previously mentioned not-so-great analytics, it’s not enough to just report the numbers.
From here you can produce some AI insights. These are essentially descriptions or assessments that the AI makes for you. For example, a description of exactly why the AI could not generate a great response for a question. Or, specific examples of what replies were considered deflected and which ones were not.
You should have a clear understanding of your problem, which you can use to come up with possible solutions.
AI reporting: Finding answers from your past Zendesk replies
Every good AI will have a section where you can see how it’s replied to questions in the past. And keeping an eye on past replies can teach you a lot of things.
For example, you can figure out how accurately your AI is responding to questions, and how many of these questions are being deflected instead of escalated through to a Zendesk agent.
Our past replies in the history tab in the dashboard can help answer questions like:
- How is my AI handling questions that are confusing?
- What Sources are being used to answer specific question types?
- What questions show a gap in the AI’s Sources/training?
From here, you can adjust the AI in the Customize section of your app. Prompting it to handle certain questions in a different way. Ultimately, it reduces the amount of times a customer needs to interact with a bot. Saving you fees you pay on interactions, and reducing your Time to Response (TTR).
AI reports for your Zendesk bot
In the Reports tab of the dashboard, you can find data about your AI. A few handy things to note are:
- Gaps in training: identify the percentage of replies where the AI did not have all the necessary information – so you can feed it more information.
- Deflection rate: observe the percentage of replies that did not need any human escalation.
- ROI/Amount saved: calculate how much money and time has been saved through the use of eesel AI. (Use the three dots in the corner to adjust the calculations.)
We’ll go into more detail about each of these things below.
1. Gaps in training – when AI can’t answer a question
Your AI is trained with Sources that you provide it. A source can be anything from a Google Doc, to a PDF to a Help Center.
Just like a person, an AI bot needs to know how to answer a question when a customer asks it.
The gaps in training report shows a percentage of replies where it can’t answer a question, due to it missing a Source.
With eesel AI, you simply click on a graph and it takes you to your bot’s history. You can then hover over the ‘gap found’ icon to find a specific question that the AI couldn’t answer.
From here you can see a description of the missing information, and upload a Source that plugs the gap. Problem solved!
We have a great range of integrations for Zendesk that will make sure you can add whatever you need.
If you’d like to keep a copy of these AI insights, simply export your chats and eesel AI will give you a handy spreadsheet.
2. Deflection rate – when AI can answer a question without human intervention.
Deflections are a good thing. It means your AI has automatically solved the problem of whoever was speaking with it, successfully ‘deflecting’ the issue from coming to your team.
However, sometimes the bot can’t deflect a question. This could be because the question is too complex for the AI to respond to, or requires human intervention to be actioned or answered accurately.
Deflections are different from ‘Gaps in training’. If a bot isn’t deflecting, your documentation needs adjusting. Whereas if there’s a gap in knowledge, your bot is probably missing the document all together.
To improve your deflection rate, check your documentation and make sure that it’s clear and actually solves the problem your customers are having.
In eesel AI, you can find your deflection rate by heading to the reports tab in your dashboard. Here you can see the percentage of replies that have been successfully deflected by the AI.
If you click on the graph, it will take you to the History tab where you can get more details. You can hover over the yes or no deflection icon for a question to see a description of why a question was or was not deflected by the AI.
To keep a copy of these AI insights, export your chats and examine the spreadsheet. You’ll learn whether prompt changes or Source changes need to be updated.
ROI – find out if your AI is saving you money
Return On Investment (ROI) is an important measurement indicating whether your AI is doing what it’s supposed to do: saving you more resources than it’s costing you. You don’t want to pay for AI only to have it sap your time and money instead of improving efficiency and cost-saving.
To calculate the ROI for your AI Agents and Assistants in Zendesk, check out the Reports tab and select the three dots in the corner of the ‘Amount saved’ box. From there, you can edit the average minutes your Zendesk support agents spend on a question. You can also edit the cost per hour that your Zendesk support agents are costing the company.
The ‘Amount saved’ box will then calculate the total time and cost saved for each correctly deflected reply.
And now you’re primed in AI analytics with Zendesk and eesel AI.
There’s more that you can glean from the data AI provides. But this primer has hopefully given you a good understanding of the basics. Pay attention to the charts and figures, and you’ll be optimising your AI in no time at all. Taking you closer to enjoying the benefits of an optimized AI chatbot.
If you need more detailed information, please reach out to it@eesel.app.
If you’re someone who learns by doing, you can also sign up to a free 7-day trial for eesel AI here. You might have a little trouble getting enough data to optimize your chatbot in 7 days, but it will give you a good functional understanding of how the app works, and what you can customize once the data starts coming in.
