Microsoft Technology Licensing, LLC

Xia Song

Saurabh Kumar Tiwary

Paul Nathan Bennett

Chenyan Xiong

Corbin Louis Rosset

Nicholas Eric Craswell

Daniel Fernando Campos

Corby Louis Rosset

Chen Zhao

Rahul Lal

Armen Aghajanyan

Nicholas Craswell

Hongfei Zhang

Saksham Singhal

Zhun Liu

A computing system is disclosed that includes a processor and memory. The memory stores instructions that, when executed by the processor, cause the processor to perform several acts. The acts include receiving, by a generative model, input set forth by a user of a client computing device that is in network communication with the computing system. The acts also include generating, by the generative model, a query based upon the input set forth by the user; providing the query to a search engine. The acts further include receiving, by the generative model and from the search engine, content identified by the search engine based upon the query. The acts additionally include generating, by the generative model, an output based upon a prompt, where the prompt includes the content identified by the search engine based upon the query. The acts also include transmitting the output to the client computing device for presentment to the user.

Informational grounding with respect to a generative model

A computer-implemented technique is described herein for assisting a user in advancing a task objective. The technique uses a suggestion-generating system (SGS) to provide one or more suggestions to a user in response to at least a last-submitted query provided by the user. The SGS may correspond to a classification-type or generative-type neural network. The SGS uses a machine-trained model that is trained using a multi-task training framework based on plural groups of training examples, which, in turn, are produced using different respective example-generating methods. One such example-generating method constructs a training example from queries in a search session. It operates by identifying the task-related intent the queries, and then identifying at least one sequence of queries in the search session that exhibits a coherent task-related intent. A training example is constructed based on queries in such a sequence.

Using a multi-task-trained neural network to guide interaction with a query-processing system via useful suggestions

This document relates to natural language processing using a framework such as a neural network. One example method involves obtaining a first document and a second document and propagating attention from the first document to the second document. The example method also involves producing contextualized semantic representations of individual words in the second document based at least on the propagating. The contextualized semantic representations can provide a basis for performing one or more natural language processing operations.

Inter-document attention mechanism

A computer-implemented technique receives an input expression that a user submits with an intent to accomplish some objective. The technique then uses a machine-trained intent encoder component to map the input expression into an input expression intent vector (IEIV). The IEIV corresponds to a distributed representation of the intent associated with the input expression, within a vector intent vector space. The technique then leverages the intent vector to facilitate some downstream application task, such as the retrieval of information. Some application tasks also use a neighbor search component to find expressions that express an intent similar to that of the input expression. A training system trains the intent encoder component based on the nexus between queries and user clicks, as recorded in a search engine's search log.

Intent encoder trained using search logs

Computer-implemented techniques are described herein for generating and utilizing a universal encoder component (UEC). The UEC maps a linguistic expression in a natural language to a language-agnostic representation of the linguistic expression. The representation is said to be agnostic with respect to language because it captures semantic content that is largely independent of the syntactic rules associated with the natural language used to compose the linguistic expression. The representations is also agnostic with respect to task because a downstream training system can leverage it to produce different kinds to machine-trained components that serve different respective tasks. The UEC facilitates the generation of downstream machine-trained models by permitting a developer to train a model based on input examples expressed in a language j, and thereafter apply it to the interpretation of documents in language j, with no additional training required.

Computer-implemented generation and utilization of a universal encoder component

Growing community of inventors

Redmond, WA, United States of America

Xia Song