Fmr Corp.

Tieyi Guo

Byung Chun

Hua Hao

Chao Yu

Pengrui Sun

Hao Fu

Chen Bi

Yong Zou

Ou Li

Sijing Lv

Bing Cui

Yachao He

Jia You

Wenbin Ren

Brian Christoper Mansfield

Methods and apparatuses for conversation dialogue orchestration in virtual assistant communication sessions include a server that establishes a chat session between a virtual assistant (VA) application and a client device. The VA application captures an utterance generated by a user and processes the utterance to instantiate a dialogue behavior tree comprising workflow agents each associated with executable code for completing a corresponding workflow action. The VA application traverses the behavior tree to generate a response to the utterance, including evaluating one or more conditions associated with a workflow agent to determine whether to execute the code in the workflow agent, and when the conditions associated with the workflow agent are met, executing the code to complete the workflow action and storing a sub-response in a dialogue memory. The VA application coalesces the sub-responses to generate a final response and transmits the final response to the client device.

Conversation dialogue orchestration in virtual assistant communication sessions

Methods and apparatuses for multimodal enhancement of interactions in conversation service applications include a server that establishes a chat-based communication session between a conversation service application and a client computing device. Query input is captured from a user during the session. The server converts an audio segment corresponding to a video into a video transcript using speech recognition, and determines a transcript portion for the video that is responsive to the query input using a dense passage retriever model. The server selects video frames from the video that correspond to the transcript portion and determines, for the selected frames, a predicted relevancy between the selected frame and the query input. The server generates a response to the query input comprising the selected frames based upon the relevancy. The server transmits the response to the client device for display.

Multimodal enhancement of interactions in conversation service applications

Methods and apparatuses are described for enhancing chatbot recognition of user intent through graph analysis. A computing device classifies user messages associated with a chat session as either comprehensible user messages that are mapped to a user intent or incomprehensible utterances that are not mapped to a user intent. The computing device arranges the comprehensible messages into first clusters and arranges the incomprehensible messages into second clusters. The computing device generates a first sub-graph for each first cluster and generates a second sub-graph for each second cluster. The computing device calculates a graph kernel matrix using the sub-graphs and trains a classification model using the graph kernel matrix. The computing device executes the trained classification model using the graph kernel matrix as input to generate a predicted intent identifier for the incomprehensible messages.

Enhancing chatbot recognition of user intent through graph analysis

Methods and apparatuses are described for automatic refinement of intent classification for virtual assistant applications. A computing device creates utterance groups comprising messages. The device determines, using a first machine learning (ML) classifier, a first predicted intent associated with each utterance group and determines, using a second ML classifier, a second predicted intent associated with each utterance group. The device combines the first and second predicted intents to generate a final predicted intent, determines, for each incomprehensible utterance group, whether the final predicted intent overlaps with the final predicted intent for other utterance groups, and selects messages having no overlapping intent to create new intents. The device determines, for each comprehensible utterance group, whether the final predicted intent overlaps with a preassigned intent for other utterance groups and combines the messages that have an overlapping intent to create and label a new utterance group with the overlapping intent.

Automatic refinement of intent classification for virtual assistant applications

Growing community of inventors

Frisco, TX, United States of America

Tieyi Guo