Confidence Score of a Response/Answer
When a user query is matched against a library, LFS returns relevant answers along with a confidence score. This score indicates how confident the answer is in matching the user’s query.
A confidence score is a number between 0 and 100. A score of 100 indicates an exact match, whereas a score of 0 indicates that no matching answer was found. The higher the score, the more confident the answer.
The following table indicates the typical confidence associated with a given score.
Score Value | Score Meaning |
|---|---|
90 - 100 | A near exact match of user query and a Library question |
> 70 | High confidence - typically a good answer that completely answers the user's query |
50 - 70 | Medium confidence - typically a fairly good answer that should answer the main intent of the user query |
30 - 50 | Low confidence - typically a related answer, that partially answers the user's intent |
< 30 | Very low confidence - typically does not answer the user's query, but has some matching words or phrases |
0 | No match, so the answer is not returned. |
Choose a score threshold
The table above shows the scores that are expected of most customers. However, since each customer is unique and has different types of words, intents, and goals, we recommend that you test and select the threshold that works best for you. The recommended threshold for most libraries is 70, and then gradually lower it as you build the library.
When determining your threshold, keep the balance of Accuracy and Coverage in mind, and adjust it based on your needs.
If Accuracy (or precision) is more important for your scenario, then increase your threshold. This way, every time you return an answer, it will be a much more CONFIDENT case, and much more likely to be the answer users are looking for. In this case, you might end up leaving more questions unanswered.
If Coverage (or recall) is more important and you want to answer as many questions as possible, even if there is only a partial relation to the user's question- then LOWER the threshold. This means there could be more cases where the answer does not answer the user's actual query but gives some other, somewhat related answer.
Improve confidence scores
To improve the confidence score of a particular response to a user query, you can add the user query to the library as an alternate question on that response. Refer: LFS AI - Best Practices For Building Response Library