Microsoft Technology Licensing, LLC

Other

Intel Corporation

Princeton University

Huawei Technologies Co., Limited

Micrsoft Corporation

Microsoft Corportion

Micriosoft Corporation

Li Deng

Alejandro Acero

Dong Yu

Jianfeng Gao

James G Droppo

Xiaodong He

Hagai Thomas Attias

Yifan Gong

Xuedong D Huang

Frank Torsten Bernd Seide

Jianshu Chen

Dilek Zeynep Hakkani-Tur

Gokhan Tur

Jinyu Li

Bin Liu

Yong She

Zhicheng Ding

Xiaolong Li

Lihong Li

Aiping Tan

Ye-Yi Wang

Xiujun Li

Leo Jingyu Lee

Nicholas S Siegele

Gang Li

Szymon Suckewer

Jian Wu

Brendan John Frey

Prabhdeep Singh

Yelong Shen

Asli Celikyilmaz

Yun-Nung Chen

Ji He

Yun-Nung Vivian Chen

Issam Bazzi

Zhengyou Zhang

Zicheng Liu

John Carlton Platt

Yu Liu

Larry Paul Heck

Yun-Cheng Ju

Geoffrey G Zweig

Xinying Song

Asela J R Gunawardana

Jian-Iai Zhou

Alex Acero

Po-Sen Huang

Faisal Ahmed

Michael Jack Sinclair

Ruhi Sarikaya

Qiang Fu

Michael Gamon

Xiaodong Liu

Ruofei Zhang

Chenlei Guo

Mei-Yuh Hwang

Patrick Pantel

Jian-Lai Zhou

Michael D Plumpe

Kaisheng N Yao

Paul Anthony Crook

LuMing Wang

Zijian Zheng

Xiang Li

Wen-tau Yih

Fethiye Asli Celikyilmaz

Frank Seide

Margaret Ann Mitchell

Anoop Deoras

Guowei Liu

Amarnag Subramanya

Saurabh Gupta

Mingyang Zhao

Chong Wang

Jui-Ting Huang

Gregory T Buehrer

Lin Xiao

Bhuwan Dhingra

Jianchang Mao

Abdel-rahman Samir Abdel-rahman Mohamed

Hamid Palangi

Mari Ostendorf

Forrest Iandola

Qiang Lou

Shizhen Wang

Paul Smolensky

Syzmon Suckewer

Rupesh Kumar Srivastava

Yunhong Zhou

Xuesong Yang

Xiadong He

Zachary Chase Lipton

Kevin Duh

Hao Fang

Gregoire Mesnil

Brian Hutchinson

Xiaodeng He

George Edward Dahl

Yaodong Zhang

Moontae Lee

A processing unit can train a model as a joint multi-domain recurrent neural network (JRNN), such as a bi-directional recurrent neural network (bRNN) and/or a recurrent neural network with long-short term memory (RNN-LSTM) for spoken language understanding (SLU). The processing unit can use the trained model to, e.g., jointly model slot filling, intent determination, and domain classification. The joint multi-domain model described herein can estimate a complete semantic frame per query, and the joint multi-domain model enables multi-task deep learning leveraging the data from multiple domains. The joint multi-domain recurrent neural network (JRNN) can leverage semantic intents (such as, finding or identifying, e.g., a domain specific goal) and slots (such as, dates, times, locations, subjects, etc.) across multiple domains.

Multi-domain joint semantic frame parsing

Systems and methods are disclosed for inquiry-based deep learning. In one implementation, a first content segment is selected from a body of content. The content segment includes a first content element. The first content segment is compared to a second content segment to identify a content element present in the first content segment that is not present in the second content segment. Based on an identification of the content element present in the first content segment that is not present in the second content segment, the content element is stored in a session memory. A first question is generated based on the first content segment. The session memory is processed to compute an answer to the first question. An action is initiated based on the answer. Using deep learning, content segments can be encoded into memory. Incremental questioning can serve to focus various deep learning operations on certain content segments.

Inquiry-based deep learning

A system in package includes a memory-die stack in memory module that is stacked vertically with respect to a processor die. Each memory die in the memory-die stack includes a vertical bond wire that emerges from a matrix for connection. Some configurations include the vertical bond wire emerging orthogonally beginning from a bond-wire pad. The matrix encloses the memory-die stack, the spacer, and at least a portion of the processor die.

Vertical bond-wire stacked chip-scale package with application-specific integrated circuit die on stack, and methods of making same

Ultra-short pulsed laser radiation is applied to a patient's eye to create a row of bubbles oriented perpendicular to the axis of vision. The row of bubbles leads to a region of the eye to be ablated. In a second step, a femtosecond laser beam guided through the row of bubbles converts it to a channel perpendicular to the axis of vision. In a third step, a femtosecond laser beam is guided through the channel to ablate a portion of the eye. Using a femtosecond laser with intensity in the range of 10-10W/cmfor the second and third steps facilitates multi-photon ablation that is practically devoid of eye tissue heating. Creating bubbles in the first step increases the speed of channel creation and channel diameter uniformity, thereby increasing the precision of the subsequent multi-photon ablation.

Introducing bubbles to improve cornea reshaping without the creation of a flap

A processing unit can extract salient semantics to model knowledge carryover, from one turn to the next, in multi-turn conversations. Architecture described herein can use the end-to-end memory networks to encode inputs, e.g., utterances, with intents and slots, which can be stored as embeddings in memory, and in decoding the architecture can exploit latent contextual information from memory, e.g., demographic context, visual context, semantic context, etc. e.g., via an attention model, to leverage previously stored semantics for semantic parsing, e.g., for joint intent prediction and slot tagging. In examples, architecture is configured to build an end-to-end memory network model for contextual, e.g., multi-turn, language understanding, to apply the end-to-end memory network model to multiple turns of conversational input; and to fill slots for output of contextual, e.g., multi-turn, language understanding of the conversational input. The neural network can be learned using backpropagation from output to input using gradient descent optimization.

End-to-end memory networks for contextual language understanding

A processing unit can operate a first recurrent computational model (RCM) to provide first state information and a predicted result value. The processing unit can operating a first network computational model (NCM) to provide respective expectation values of a plurality of actions based at least in part on the first state information. The processing unit can provide an indication of at least one of the plurality of actions, and receive a reference result value, e.g., via a communications interface. The processing unit can train the first RCM based at least in part on the predicted result value and the reference result value to provide a second RCM, and can train the first NCM based at least in part on the first state information and the at least one of the plurality of actions to provide a second NCM.

Multi-model controller

In some examples, a computing device refines feature information of query text. The device repeatedly determines attention information based at least in part on feature information of the image and the feature information of the query text, and modifies the feature information of the query text based at least in part on the attention information. The device selects at least one of a predetermined plurality of outputs based at least in part on the refined feature information of the query text. In some examples, the device operates a convolutional computational model to determine feature information of the image. The device network computational models (NCMs) to determine feature information of the query and to determine attention information based at least in part on the feature information of the image and the feature information of the query. Examples include a microphone to detect audio corresponding to the query text.

Multi-stage image querying

A system in package includes a stair-stacked memory module that is stacked vertically with respect to a processor die. A spacer is used adjacent to the processor die to create a bridge for the stair-stacked memory module. Each memory die in the stair-stacked memory module includes a vertical bond wire that emerges from a matrix for connection. The matrix encloses the stair-stacked memory module and at least a portion of the processor die.

Stair-stacked dice device in a system in package, and methods of making same

A pre-packaged stair-stacked memory module is mounted on a board with at least one additional component. A stair-stacked memory module includes a plurality of memory dice that are stacked vertically with respect to a processor die. A spacer is used adjacent to the processor die to create a bridge for the stair-stacked memory module. Each memory die in the stair-stacked memory module includes a vertical bond wire that emerges from a matrix for connection. The matrix encloses the stair-stacked memory module and at least a portion of the processor die. The matrix might also enclose the at least one additional component.

Prepackaged stair-stacked memory module in a chip scale system in package, and methods of making same

A processing unit can determine a first feature value corresponding to a session by operating a first network computational model (NCM) based part on information of the session. The processing unit can determine respective second feature values corresponding to individual actions of a plurality of actions by operating a second NCM. The second NCM can use a common set of parameters in determining the second feature values. The processing unit can determine respective expectation values of some of the actions of the plurality of actions based on the first feature value and the respective second feature values. The processing unit can select a first action of the plurality of actions based on at least one of the expectation values. In some examples, the processing unit can operate an NCM to determine expectation values based on information of a session and information of respective actions.

Multiple-action computational model training and operation

Technologies pertaining to slot filling are described herein. A deep neural network, a recurrent neural network, and/or a spatio-temporally deep neural network are configured to assign labels to words in a word sequence set forth in natural language. At least one label is a semantic label that is assigned to at least one word in the word sequence.

Assignment of semantic labels to a sequence of words using neural network architectures

Systems and methods for determining knowledge-guided information for a recurrent neural networks (RNN) to guide the RNN in semantic tagging of an input phrase are presented. A knowledge encoding module of a Knowledge-Guided Structural Attention Process (K-SAP) receives an input phrase and, in conjunction with additional sub-components or cooperative components generates a knowledge-guided vector that is provided with the input phrase to the RNN for linguistic semantic tagging. Generating the knowledge-guided vector comprises at least parsing the input phrase and generating a corresponding hierarchical linguistic structure comprising one or more discrete sub-structures. The sub-structures may be encoded into vectors along with attention weighting identifying those sub-structures that have greater importance in determining the semantic meaning of the input phrase.

Knowledge-guided structural attention processing

In classification tasks applicable to data that exhibit sequential output statistics, a classifier may be trained in an unsupervised manner based on a sequence of input samples and an unaligned sequence of output labels, using a cost function that measures the negative cross-entropy of an N-gram joint probability distribution derived from the sequence of output labels with respect to an expected N-gram frequency in a second sequence of output labels predicted by the classifier. In some embodiments, a primal-dual reformulation of the cost function is employed to facilitate optimization.

Unsupervised learning utilizing sequential output statistics

Growing community of inventors

Redmond, WA, United States of America

Li Deng