Meta Platforms, Inc.

Microsoft Technology Licensing, LLC

Facebook Technologies, LLC

Armen Aghajanyan

Anuj Kumar

Saurabh Kumar Tiwary

Xia Song

Ahmed Aly

Luke Zettlemoyer

Arun Babu

Michael William Lewis

Theodore Frank Levin

Crystal Annette Naomi Su Hua Nakatsu

Daniel Difranco

Jonathan David Christensen

Kirk LaBuda

Marjan Ghazvini Nejad

Brian Moran

Sida Wang

Sonal Gupta

Gargi Ghosh

In one embodiment, a method includes accessing a first document, accessing a plurality of second documents, calculating a relevance score for each of the plurality of second documents indicating a degree of relevance of the second document to the first document using an encoder of a machine-learning model, selecting a subset of the second documents based on their corresponding relevance scores, generating a target document by using the machine-learning model to process the subset of second documents and their corresponding relevance scores, and updating parameters of the machine-learning model based on a comparison between the first document and the generated target document.

Pretraining a language machine-learning model

In one embodiment, a method includes receiving a user input comprising a natural-language utterance by an assistant xbot from a client system associated with a user, determining a semantic representation of the user input based on a structural ontology defining a labeling syntax for parsing the natural-language utterance to semantic units comprising actions, objects, and attributes, wherein the semantic representation embeds at least one object within at least one action and declares at least one attribute of the embedded object to be acted upon, sending a request based on the semantic representation to an agent for executing a task corresponding to the user input, receiving results of the executed task mapped to a structure determined by the structural ontology from the agent, and sending from the assistant xbot to the client system instructions for presenting a response based on the results of the executed task.

Semantic representations using structural ontology for assistant systems

In one embodiment, a method includes receiving a user input comprising one or more words at a client system, wherein each word comprises one or more characters, inputting the words to a convolutional neural network (CNN) model stored on the client system, accessing a plurality of character-embeddings for a plurality of characters, respectively, from a data store of the client system, generating one or more word-embeddings for the one or more words, respectively, based on the accessed character-embeddings by processing the accessed character-embeddings with one or more convolutional layers and one or more gated linear units of the CNN model, determining one or more tasks corresponding to the user input for execution based on an analysis of the one or more word-embeddings by the CNN model, and providing an output responsive to the user input based on the execution of the one or more tasks at the client system.

On-device convolutional neural network models for assistant systems

Computer-implemented techniques are described herein for generating and utilizing a universal encoder component (UEC). The UEC maps a linguistic expression in a natural language to a language-agnostic representation of the linguistic expression. The representation is said to be agnostic with respect to language because it captures semantic content that is largely independent of the syntactic rules associated with the natural language used to compose the linguistic expression. The representations is also agnostic with respect to task because a downstream training system can leverage it to produce different kinds to machine-trained components that serve different respective tasks. The UEC facilitates the generation of downstream machine-trained models by permitting a developer to train a model based on input examples expressed in a language j, and thereafter apply it to the interpretation of documents in language j, with no additional training required.

Computer-implemented generation and utilization of a universal encoder component

Growing community of inventors

Bellevue, WA, United States of America

Armen Aghajanyan