Detecting statistical anomalies in voice interactions

Abstract

The systems and methods disclosed herein detect indicators of undesirable intent during interactions between users and digital avatars by evaluating emotional signals in user speech. The system generates emotion feature vectors from user input streams that capture characteristics such as pitch variation, speech rate, vocal intensity, spectral features, and so forth. The system uses a two-level evaluation that compares the emotion feature vector against a population baseline derived from historical user interactions to generate a population anomaly score and further compares the emotion feature vector against an allow list database and a deny list database to generate a threat match score based on known protected categories (e.g., threat categories, allowed categories). The system aggregates these scores to produce a risk score, which is evaluated against threshold values to determine a response action (e.g., logging the interaction, injecting clarifying questions, routing to human agents, terminating the interaction).