Microsoft Technology Licensing, LLC

Yun-Nung Chen

Dilek Zeynep Hakkani-Tur

Li Deng

Jianfeng Gao

Gokhan Tur

Asli Celikyilmaz

Ye-Yi Wang

Xiaodong He

A processing unit can train a model as a joint multi-domain recurrent neural network (JRNN), such as a bi-directional recurrent neural network (bRNN) and/or a recurrent neural network with long-short term memory (RNN-LSTM) for spoken language understanding (SLU). The processing unit can use the trained model to, e.g., jointly model slot filling, intent determination, and domain classification. The joint multi-domain model described herein can estimate a complete semantic frame per query, and the joint multi-domain model enables multi-task deep learning leveraging the data from multiple domains. The joint multi-domain recurrent neural network (JRNN) can leverage semantic intents (such as, finding or identifying, e.g., a domain specific goal) and slots (such as, dates, times, locations, subjects, etc.) across multiple domains.

Multi-domain joint semantic frame parsing

A processing unit can extract salient semantics to model knowledge carryover, from one turn to the next, in multi-turn conversations. Architecture described herein can use the end-to-end memory networks to encode inputs, e.g., utterances, with intents and slots, which can be stored as embeddings in memory, and in decoding the architecture can exploit latent contextual information from memory, e.g., demographic context, visual context, semantic context, etc. e.g., via an attention model, to leverage previously stored semantics for semantic parsing, e.g., for joint intent prediction and slot tagging. In examples, architecture is configured to build an end-to-end memory network model for contextual, e.g., multi-turn, language understanding, to apply the end-to-end memory network model to multiple turns of conversational input; and to fill slots for output of contextual, e.g., multi-turn, language understanding of the conversational input. The neural network can be learned using backpropagation from output to input using gradient descent optimization.

End-to-end memory networks for contextual language understanding

Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.

Knowledge-guided structural attention processing

A computer-implemented technique is described herein for detecting actionable items in speech. In one manner of operation, the technique can include receiving utterance information that expresses at least one utterance made by one participant of a conversation to at least one other participant of the conversation. The technique can also include converting the utterance information into recognized speech information and using a machine-trained model to recognize at least one actionable item associated with the recognized speech information. The technique can also include performing at least one computer-implemented action associated the actionable item(s). The machine-trained model may correspond to a deep-structured convolutional neural network. The technique can produce the machine-trained model using a source environment corpus that is not optimally suited for a target environment in which the model is intended to be applied. The technique can provide adaptation techniques for adapting a source-environment model to better suit the target environment.

Growing community of inventors

Bellevue, WA, United States of America

Yun-Nung Chen