US 12,248,942 B2
System and method for automatically evaluating and scoring the quality of agent-customer interactions
Laura Cattaneo, Rochester, MN (US); Boris Chaplin, Medina, MN (US); Kyle Smaagard, Forest Lake, MN (US); Chris Vanciu, Isle, MN (US); Dylan Morgan, Minneapolis, MN (US); and Catherine Bullock, Minneapolis, MN (US)
Assigned to Calabrio, Inc., Minneapolis, MN (US)
Filed by Calabrio, Inc., Minneapolis, MN (US)
Filed on Dec. 21, 2022, as Appl. No. 18/069,983.
Prior Publication US 2024/0211960 A1, Jun. 27, 2024
Int. Cl. G06Q 10/0639 (2023.01); G06F 40/279 (2020.01); G06Q 30/015 (2023.01)
CPC G06Q 30/015 (2023.01) [G06F 40/279 (2020.01); G06Q 10/06398 (2013.01)] 20 Claims
OG exemplary drawing
 
1. A system comprising:
at least one processor; and
memory storing instructions that, when executed by the at least one processor, causes the system to perform a set of operations, the set of operations comprising:
receive a piece of content, wherein the piece of content is a record of an interaction between an agent and a customer;
pre-process the piece of content into a labeled text-based transcript, wherein pre-processing the piece of content comprises providing an audio recording of the content to a machine learning model to generate the labeled text-based transcript, and wherein the machine learning model is operable to generate customer utterance labels and agent utterance labels for the labeled text-based transcript;
receive one or more dimensions to utilize in determining an interaction quality score, wherein a dimension is comprised of one or more metrics, the one or more metrics are associated with a score generated by one or more additional machine learning models, the one or more additional machine learning models are selected based upon one or more metric types, and wherein the one or more dimensions comprise an informativeness dimension, wherein scoring the informativeness dimension further comprises:
analyzing textual data using more than sentiment (MTS) frequency-based natural language processing (NLP) to quantify text structures in the labeled text-based transcript to determine information density for boilerplate information and specific information, wherein the information density is used to determine an informativeness score;
determine the interaction quality score; and
report the interaction quality score, wherein reporting the interaction quality score comprises:
generating descriptive information related to how the interaction quality score was determined;
generating a recommendation to improve agent performance; and
sending the descriptive information, the recommendation, and the interaction quality score to a remote device.