Calabrio, Inc.

Catherine Bullock

Dylan Morgan

Kyle Smaagard

Chris Vanciu

Boris Chaplin

Laura Cattaneo

Matt Matsui

Paul Gordon

Aspects of the present disclosure relate to evaluating agent performance during interactions with a customer by generating a repeatability metric based on statements in an associated interaction transcript. The repeatability metric is a performance indicator that quantifies the amount of repetition an individual experiences or utilizes during an interaction between the agent and the customer. The repeatability metric may be calculated by analyzing a transcript and the associated metadata to identify repetitive statements within the transcript. Once a repeatability metric is calculated it may be aggregated across a plurality of levels within a contact center and within the wider enterprise context to improve customer-agent interactions. Thus, the insights provided by the repeatability metric may highlight areas where additional agent training is required, improve agent response time, and increase customer satisfaction with their contact center experience.

Evaluating transcripts through repetitive statement analysis

Disclosed herein are computer-implemented devices, systems, and methods of sanitizing a transcript. In an example, such a method includes selecting a transcript to be sanitized, identifying potential redactions to be made in the transcript, and redacting the transcript at the potential redactions to sanitize the transcript. The potential redactions are identifiable via a multi-pass process that includes: generating initial redactions to be made based on surrounding context within the transcript; generating matching redactions to be made based on the initial redactions, and generating character redactions to be made based on at least one of the initial redactions and the matching redactions.

Devices, systems, and methods for transcript sanitization

The present disclosure relates to automatically evaluating an agent-customer interaction utilizing aspects of machine learning to score the quality of the interaction. In some embodiments, one or more machine learning models are utilized to generate an interaction quality score which is a comprehensive evaluation of agent performance during the interaction. The interaction quality score is a combination of two sub-scores, a conversation score and service score which are each based on one or more dimension scores. The conversation score is a measure of how well the agent engages with the customer during the interaction. The service score is an evaluation of the quality of the agent's service during the interaction in terms of customer's perception of the agent's performance. Each of the conversation score and service score are determined by an analysis of one or more dimensions such as fluency, relevance, appropriateness, informativeness, assurance, responsiveness, empathy, compliance, and sentiment.

System and method for automatically evaluating and scoring the quality of agent-customer interactions

Systems and methods are provided for determining calls and contacts associated with calls that are contextually similar based on content of the calls. Data associated with a call include one or more utterances made by speakers and a set of values indicating relevance between content of the call and topic categories of the call. The disclosed technology generates a topic vector associated with a call and/or respective speakers of the call. The topic vector includes a multi-dimensional vector where each dimension corresponds to a topic category. The disclosed technology determines calls that are contextually similar by comparing angular distances between topic vectors. A search query receiver receives a search query that queries contacts and calls that are contextually similar to a given call and/or a speaker. The disclosed technology identifies calls with topic vectors that are within a predetermined angular distance.

Searching calls based on contextual similarity among calls

Systems and methods disclosed relate to contextually tagging statements associated with calls. In particular, the contextual tagging is directed to training a call tagging model for predicting one or more categories associated with a statement for tagging. The disclosed technology generates training data for training the call tagging model based on a list of known phrases used in contacts in a contextual category and matching phrases and words in the list of known phrases against words and phrases used in statements in sample call transcripts. The call tagging model is fine-tuned by using sample statements that appear in contacts. Once trained, the call tagging model is used to determine a probability distribution of categories associated with statements in a contact and further determine contact-level category distributions using multi-dimensional vectors. The tagged contacts are used to determine contacts that are contextually similar to a given contact.

Call tagging using machine learning model

Disclosed herein are computer-implemented devices, systems, and methods for sanitizing an audio recording of a conversation. In an example, such a computer-implemented method can include selecting a transcript of the conversation. The transcript can include a plurality of redaction markers within the audio recording. The plurality of redaction markers can be identifiable via one or more redaction marker start and/or end points. The computer-implemented method can include identifying a first redaction marker and a second redaction marker of the one or more redaction markers. The first redaction marker can be different from the second redaction marker. In this regard, a redaction segment is formable between the first and second redaction markers. The computer-implemented method can include selecting the audio recording. The computer-implemented method can include redacting the audio recording between the first redaction marker and the second redaction marker.

Devices, systems, and methods for audio sanitization

Aspects of the present disclosure relate to evaluating a contact center agent using an automated evaluation process that employs aspects of machine learning to review pieces of content, identify context within a piece of content where a compliance statement is required, and determine if a compliance statement was given by the agent. In some embodiments, a compliance model is trained and utilized to recognize context within the customer-agent interaction indicating that a compliance statement should be given by the agent. The presence or absence of a compliance statement in the piece of content may then be evaluated by the model and reported to the contact center supervisor. The automated nature of the invention efficiently and effectively reduces the unnecessary randomness introduced by a manual review process while providing improved assurance that compliance requirements are consistently provided during customer interactions.

System and method for identifying compliance statements from contextual indicators in content

Aspects of the present disclosure relate to systems and methods for detecting emerging events. In various examples, a method for detecting emerging events includes obtaining communication data associated with communication between multiple sources, segmenting communication data into multiple segments, determining whether a data segment belongs to a familiar topic or none, and generating a notification when a familiar topic is mentioned for more or less than a mention prediction. Additionally, or alternatively, a notification may be generated when an unfamiliar topic emerges from a set of unfamiliar data segments if an associated segment count exceeds a critical mass threshold. To determine whether a data segment belongs to a familiar topic, the data segment may be transformed into a feature vector and mapped onto a feature space, where a distance-based similarity score may be determined.

Growing community of inventors

Minneapolis, MN, United States of America

Catherine Bullock