Servicenow, Inc.

Paymentus Corporation

Billeo, Inc.

Edwin Vito Sapugay

Srinivas Satyasai Sunkara

Anil Kumar Madamala

Maxim Naboka

Murali B Subbarao

Lewis Savio Landry Santos

Omer Anil Turkkan

Jonggun Park

Phani Bhushan Kumar Nivarthi

Sathwik Tejaswi Madhusudhan

Srinivas SatyaSai Sunkara

Sagar Davasam Suryanarayan

Gopal Sarda

Aniruddha Madhusudan Thakur

Sriram Palapudi

Rammohan Narendula

Edward Cheung

Vipulkumar Popat Mahadik

Roshnee Sharma

Hari Subramani

Anne Katharine Heaton-Dunlap

Masayo Iida

A natural language understanding (NLU) framework includes an ensemble scoring system designed to receive indicators determined by various systems of the NLU framework when inferencing a user utterance. The ensemble scoring system uses the received indicators, along with a set of ensemble scoring weights, to determine a respective ensemble score for each artifact of the utterance identified during inference. For example, segmentations provided by a lookup source system may be used to boost scores of intent and/or entities identified during a meaning search operation of a NLU system. The NLU framework may also include an ensemble scoring weight optimization subsystem that automatically determines optimized ensemble scoring weight values from labeled training data using an optimization plugin. Accordingly, the NLU framework enables these indicators to be suitably weighted and combined to provide a desired level of performance (e.g., computational resource consumption, precision, recall) of the NLU framework during operation.

Ensemble scoring system for a natural language understanding (NLU) framework

Present embodiment include a prosody subsystem of a natural language understanding (NLU) framework that is designed to analyze collections of written messages for various prosodic cues to break down the collection into a suitable level of granularity (e.g., into episodes, sessions, segments, utterances, and/or intent segments) for consumption by other components of the NLU framework, enabling operation of the NLU framework. These prosodic cues may include, for example, source prosodic cues that are based on the author and the conversation channel associated with each message, temporal prosodic cues that are based on a respective time associated with each message, and/or written prosodic cues that are based on the content of each message. For example, to improve the domain specificity of the agent automation system, intent segments extracted by the prosody subsystem may be consumed by a training process for a ML-based structure subsystem of the NLU framework.

Written-modality prosody subsystem in a natural language understanding (NLU) framework

A natural language understanding (NLU) framework includes a domain-aware vector encoding (DAVE) framework. The DAVE framework enables a designer to create a DAVE system having a domain-agnostic semantic (DAS) model and a corresponding trained vector translator (VT) model. The DAVE system uses the DAS model to generate domain-agnostic semantic vectors for portions of a user utterance, and then uses the VT model to translate the domain-agnostic semantic vectors into a domain-aware semantic vectors to be used by a NLU system of the NLU framework during a meaning search operation. The VT model is also designed to provide predicted intent classifications for the portions the user utterance. Both the NLU system and the DAVE system of the NLU framework are highly configurable and refer to various NLU constraints during operation, including performance constraints and resource constraints provided by a designer or user of the NLU framework.

Domain-aware vector encoding (DAVE) system for a natural language understanding (NLU) framework

A natural language understanding (NLU) framework includes a lookup source framework, which enables a lookup source system to be defined having one or more lookup sources. The lookup source system can operate in a number of different manners to facilitate repository-aware inference of user utterances, for example, by facilitating vocabulary injection during compilation of an utterance meaning model and/or an understanding model. Additionally, the lookup source system can be leveraged to cleanse client-specific training data of sensitive values to generate generic training data that can be used to train the NLU framework of other clients. The lookup sources can be compiled in a synchronous or asynchronous manner, which enables lookup sources to be compiled in an on-demand basis from test source data. Additionally, understanding models that reference lookup sources can be periodically recompiled while leveraging the latest versions of the lookup sources for vocabulary injection.

System and method for repository-aware natural language understanding (NLU) using a lookup source framework

A natural language understanding (NLU) framework includes a lookup source framework, which enables a lookup source system to be defined having one or more lookup sources. Each lookup source of the lookup source system includes a respective source data representation that is compiled from respective source data. For example, a source data representation may include source data arranged in a finite state transducer (IFST) structure as a set of finite-state automata (FSA) states, wherein each state is associated with a token that represents underlying source data. Different producers can be applied during compilation of a source data representation to derive additional states within the source data representation from the source data. Certain states of the source data representation that contain sensitive data can be selectively protected through encryption and/or obfuscation, while other portions of the source data representation that are not sensitive may remain in clear-text form.

Lookup source framework for a natural language understanding (NLU) framework

A natural language understanding (NLU) framework includes an a concept system that performs concept matching of user utterances. The concept system generates a concept cluster model from sample utterances of an intent-entity model, and then trains a machine learning (ML) concept model based on the concept cluster model. Once trained, the concept model receives semantic vectors representing potential concepts extracted from utterances, and provides concept indicators to an ensemble scoring system. These concept indicators include indications of which concepts of the concept model that matched to the potential concepts, which intents of the intent-entity model are related to these concepts, and concept-relationship scores indicating a strength and/or uniqueness of the relationship between each concept-intent combination. Based on these concept-related indicators, the ensemble scoring system may determine and apply an ensemble scoring adjustment when determining an ensemble artifact score for each of the artifacts extracted from an utterance.

Concept system for a natural language understanding (NLU) framework

A natural language understanding (NLU) framework includes a modeling and optimization system that enables enhanced understanding and explainability to the operation of the NLU framework. The NLU framework includes a configuration vector storing settings of various components that may be applied during NLU inference of an utterance, such as which components should be activated or deactivated, as well as which numerical values (e.g., threshold values, coefficients, weight values) that are used by these components during operation. By using this configuration vector to systematically disable and adjust numerical parameters of the components of the NLU framework, and then determining the performance of the NLU framework in these configurations, the modeling and optimization system determines relationships between, as well as the relative importance of, the components of the NLU framework. The modeling and optimization system automatically determines or optimizes configurations for the NLU framework to accommodate various NLU performance and/or resource constraints.

Operational modeling and optimization system for a natural language understanding (NLU) framework

A natural language understanding (NLU) framework includes a lookup source system having one or more lookup sources. Each lookup source includes a respective source data representation that is compiled from respective source data. Once compiled, a user utterance can be submitted to the lookup source system, which generates segmentations of the user utterance. Each segmentation generally includes a collection of non-overlapping segments, and each segment generally describes how tokens of the user utterance can be grouped together and matched to the states of the source data representations. During lookup source inference, matches can be made to produced states or using fuzzy matchers that have corresponding of scoring adjustments. These scoring adjustments may be used by a segmentation scoring subsystem, potentially in combination with one or more additional segmentation scoring plugins, to score and rank the segmentations determined by the lookup source system for the user utterance.

System and method for lookup source segmentation scoring in a natural language understanding (NLU) framework

An agent automation system includes a memory configured to store a reasoning agent/behavior engine (RA/BE) including a first persona and a current context and a processor configured to execute instructions of the RA/BE to cause the first persona to perform actions comprising: receiving intents/entities of a first user utterance; recognizing a context overlay cue in the intents/entities of the first user utterance, wherein the context overlay cue defines a time period; updating the current context of the RA/BE by overlaying context information from at least one stored episode associated with the time period; and performing at least one action based on the intents/entities of the first user utterance and the current context of the RA/BE.

System for focused conversation context management in a reasoning agent/behavior engine of an agent automation system

An agent automation system includes a memory configured to store a natural language understanding (NLU) framework and a model, wherein the model includes at least one original meaning representation. The system includes a processor configured to execute instructions of the NLU framework to cause the agent automation system to perform actions including: performing rule-based generalization of the model to generate at least one generalized meaning representation of the model from the at least one original meaning representation of the model; performing rule-based refinement of the model to prune or modify the at least one generalized meaning representation of the model, or the at least one original meaning representation of the model, or a combination thereof; and after performing the rule-based generalization and the rule-based refinement of the model, using the model to extract intents/entities from a received user utterance.

Templated rule-based data augmentation for intent extraction

The present approaches are generally related to an agent automation framework that is capable of extracting meaning from user utterances, such as requests received by a virtual agent (e.g., a chat agent), and suitably responding to these user utterances. In certain aspects, the agent automation framework includes a NLU framework and an intent-entity model having defined intents and entities that are associated with sample utterances. The NLU framework may include a meaning extraction subsystem designed to generate meaning representations for the sample utterances of the intent-entity model to construct an understanding model, as well as generate meaning representations for a received user utterance to construct an utterance meaning model. The disclosed NLU framework may include a meaning search subsystem that is designed to search the meaning representations of the understanding model to locate matches for meaning representations of the utterance meaning model.

Deriving multiple meaning representations for an utterance in a natural language understanding (NLU) framework

An agent automation system implements a virtual agent that is capable of learning new words, or new meanings for known words, based on exchanges between the virtual agent and a user in order to customize the vocabulary of the virtual agent to the needs of the user or users. The agent automation framework has access to a corpus of previous exchanges between the virtual agent and the user, such as one or more chat logs. New words and/or new meanings for known words are identified within the corpus and new word vectors are generated for these new words and/or new meanings for known words and added to refine a word vector distribution model. The refined word vector distribution model is then utilized by the agent automation system to interact with the user.

Systems and method for vocabulary management in a natural learning framework

The present disclosure is directed to an agent automation framework that is capable of extracting meaning from user utterances and suitably responding using a search-based natural language understanding (NLU) framework. The NLU framework includes a meaning extraction subsystem capable of detecting multiple alternative meaning representations for a given natural language utterance. Furthermore, the NLU framework includes a meaning search subsystem that enables elastic confidence thresholds (e.g., elastic beam-width meaning searches), forced diversity, and cognitive construction grammar (CCG)-based predictive scoring functions to provide an efficient and effective meaning search. As such, the disclosed meaning extraction subsystem and meaning search subsystem improve the performance, the domain specificity, the inference quality, and/or the efficiency of the NLU framework.

System and method for performing a meaning search using a natural language understanding (NLU) framework

An agent automation system includes a memory configured to store a natural language understanding (NLU) framework and a processor configured to execute instructions of the NLU framework to cause the agent automation system to perform actions. These actions comprise: generating an annotated utterance tree of an utterance using a combination of rules-based and machine-learning (ML)-based components, wherein a structure of the annotated utterance tree represents a syntactic structure of the utterance, and wherein nodes of the annotated utterance tree include word vectors that represent semantic meanings of words of the utterance; and using the annotated utterance tree as a basis for intent/entity extraction of the utterance.

Hybrid learning system for natural language understanding

Described are systems and methods for product recommendation refinement in a topic-based virtual storefront embedded in a topical community web page. A system and method may facilitate determining user and community member activity in the virtual storefront based on which weighted keywords are derived. A topic set containing various weighted keywords may be iteratively configured for extracting and ordering one or more products that are extracted from a plurality of heterogeneous sources. A server may be configured for refining the topic set by modifying a baseline session keyword weight or a baseline contextual keyword weight based on a total topic set weight, an elasticity parameter, core topic keyword weights associated with core topic keywords, session keyword weights, and/or contextual keyword weights; and identifying, based at least on the configured topic set, products to be presented to a user as product recommendations in the virtual storefront.

Growing community of inventors

Foster City, CA, United States of America

Edwin Vito Sapugay