Oracle International Corporation

Crystal C Pan

Vishal Vishnoi

Mark Edward Johnson

Thanh Long Duong

Srinivasa Phani Kumar Gadde

Elias Luqman Jalaluddin

Gautam Singaraju

Balakota Srinivas Vinnakota

Manish Parekh

Cong Duy Vu Hoang

Vladislav Blinov

Xin Xu

Duy Vu

Muruganantham Chinnananchi

Saba Amsalu Teserra

Jae Min John

Stephen Andrew McRitchie

Fen Wang

The present disclosure relates to techniques for identifying out-of-domain utterances. One particular technique includes receiving an utterance and a target domain of a chatbot, generating a sentence embedding for the utterance, obtaining an embedding representation for each cluster of in-domain utterances associated with the target domain, predicting, using a metric learning model, a first probability that the utterance belongs to the target domain based on a similarity or difference between the sentence embedding and each embedding representation for each cluster, predicting, using an outlier detection model, a second probability that the utterance belongs to the target domain based on a determined distance or density deviation between the sentence embedding and embedding representations for neighboring clusters, evaluating the first probability and the second probability to determine a final probability, and classifying the utterance as in-domain or out-of-domain for the chatbot based on the final probability.

Techniques for out-of-domain (OOD) detection

Techniques are described for invoking and switching between chatbots of a chatbot system. In some embodiments, the chatbot system is capable of routing an utterance received while a user is already interacting with a first chatbot in the chatbot system. For instance, the chatbot system may identify a second chatbot based on determining that (i) such an utterance is an invalid input to the first chatbot or (ii) that the first chatbot is attempting to route the utterance to a destination associated with the first chatbot. Identifying the second chatbot can involve computing, using a predictive model, separate confidence scores for the first chatbot and the second chatbot, and then determining that a confidence score for the second chatbot satisfies one or more confidence score thresholds. The utterance is then routed to the second chatbot based on the identifying of the second chatbot.

Routing for chatbots

Techniques for automatically switching between chatbot skills in the same domain. In one particular aspect, a method is provided that includes receiving an utterance from a user within a chatbot session, where a current skill context is a first skill and a current group context is a first group, inputting the utterance into a candidate skills model for the first group, obtaining, using the candidate skills model, a ranking of skills within the first group, determining, based on the ranking of skills, a second skill is a highest ranked skill, changing the current skill context of the chatbot session to the second skill, inputting the utterance into a candidate flows model for the second skill, obtaining, using the candidate flows model, a ranking of intents within the second skill that match the utterance, and determining, based on the ranking of intents, an intent that is a highest ranked intent.

Automatic out of scope transition for chatbot

Techniques are described to determine whether an input utterance is unrelated to a set of skill bots associated with a master bot. In some embodiments, a system described herein includes a training system and a master bot. The training system trains a classifier of the master bot. The training includes accessing training utterances associated with the skill bots and generating training feature vectors from the training utterances. The training further includes generating multiple set representations of the training feature vectors, where each set representation corresponds to a subset of the training feature vectors, and configuring the classifier with the set representations. The master bot accesses an input utterance and generates an input feature vector. The master bot uses the classifier to compare the input feature vector to the multiple set representations so as to determine whether the input feature falls outside and, thus, cannot be handled by the skill bots.

Detecting unrelated utterances in a chatbot system

Techniques for chatbots, and more particularly, to techniques for using a directed acyclic graph (DAG) based framework to build and train models. In one particular aspect, a computer implemented method is provided that includes generating, by a DAG based framework, a first model and a second model, executing the first model for a chatbot in run-time and second model for the chatbot in design-time, collecting attributes for intent classification associated with a set of utterances with the chatbot running the first model and the second model, evaluating, using one or more metrics, performance of the first model and the second model based on an analysis of the attributes for the intent classification, determining whether the performance of the second model is improved as compared to the performance of the first model, and executing the first model or the second model for the chatbot in run-time based on the performance determination.

Directed acyclic graph based framework for training models

Growing community of inventors

Palo Alto, CA, United States of America

Crystal C Pan