Amazon Technologies, Inc.

Anjishnu Kumar

Ruhi Sarikaya

Arpit Gupta

Xing Fan

Markus Dreyer

Vasiliy Radostev

Abhinav Sethy

Chetan Nagaraj Naik

Janez Starc

Rekha Seshadrinathan

Imre Attila Kiss

Vinaya Nadig

Wei Ting Chen

David Spike Palfrey

Samuel M Tucker

Samarth Bhargava

Faisal Ladhak

William Clinton Dabney

Emilio Fabio Monti

Pavankumar Reddy Muddireddy

Daniele Bonadiman

Mariia Naslidnyk

Daniel Cristian Duma

Omar Zia Khan

Sainyam Kapoor

Supriya Medapati

Benjamin Djidi

Prakash Kumar Jha

Techniques for determining relevant and personalized supplemental content for a user are described. A system may prompt a language model to generate queries based on an entity (or entities) representing a user's affinity. In some examples, the language model may be prompted to generate an initial query corresponding to the entity and then generate one or more enriched queries based on the initial query. The one or more enriched queries may be used to search one or more content providers for supplemental content results. The supplemental content results may be aggregated, may be validated to ensure correspondence to the enriched query, and may be stored for future output. A system component may request supplemental content for output, at which time the system may select one or more supplemental content from storage, where the selection may be made to optimize relevancy for current user and situational context.

Determining supplemental content for output

Techniques for determining a command or intent likely to be subsequently invoked by a user of a system are described. A user inputs a command (either via a spoken utterance or textual input) to a system. The system determines content responsive to the command. The system also determines a second command or corresponding intent likely to be invoked by the user subsequent to the previous command. Such determination may involve analyzing pairs of intents, with each pair being associated with a probability that one intent of the pair will be invoked by a user subsequent to a second intent of the pair. The system then outputs first content responsive to the first command and second content soliciting the user as to whether the system to execute the second command.

Proactive command framework

Techniques for generating a visual response to a user input are described. A system may receive a natural language input and use a machine learning model to determine a first component is to determine a response to the natural language input while a second component is to determine supplemental content related to the natural language input. The system may receive, from the first component, first image data corresponding to the response. The system may also receive, from the second component, second image data corresponding to the supplemental content. The system may send, to a display, a command to present the first image data and the second image data.

Visual responses to user inputs

Techniques for generating a visual response to a user input are described. A system may receive input data corresponding to a user input, determining a first skill component is to determine a response to the user input, and determine a second skill component is to determine supplemental content related to the user input. The system may also determine a template for presenting a visual response to the user input, where the template is configured for presenting the response and the supplemental content. The system may receive, from the first skill component, first image data corresponding to the first response. The system may also receive, from the second skill component, second image data corresponding to the first supplemental content. The system may send, to a device including a display, a command to present the first image data and the second image data using the template.

A method comprising receiving first data representative of a query. A representation of the query is generated using an encoder neural network and the first data. Words for a rephrased version of the query are selected from a set of words comprising a first subset of words comprising words of the query and a second subset of words comprising words absent from the query. Second data representative of the rephrased version of the query is generated.

Query rephrasing using encoder neural network and decoder neural network

A method comprising sending query data representative of a query to a first and second query-answering subsystem of a query-answering system. It is determined that the query-answering system lacks a satisfactory answer to the query. Paraphrase data representative of a rephrased version of the query is received. The paraphrase data is sent to the first and second query-answering subsystems. Answer data representative of an answer to the rephrased version of the query is received from the first query-answering subsystem.

Query paraphrasing

A method is provided. The method includes determining a number of queries for which an answer was undetermined from a knowledge database and are related to a subject. The method includes determining a period of time associated with receipt of the queries by the knowledge database. The method includes generating, based on the number of queries and the period of time, rate data indicative of a failure rate. The method includes determining that the failure rate satisfies a failure rate condition. The method includes sending text data representative of a query of the queries to a query-answering component different from the knowledge database. Other methods and systems are also provided.

Query answering

A voice user interface (VUI) system use collaborative filtering to expand its own knowledge base. The system is designed to improve the accuracy and performance of the Natural Language Understanding (NLU) processing that underlies VUIs. The system leverages the knowledge of system users to crowdsource new information.

Voice user interface knowledge acquisition system

An interactive system makes use of a concise user representation, for example, in the process of making predictions of a user's next action. In some examples, the concise user representation is computed from a larger amount of user data, which is processed using a transformation trained using a Generative Adversarial Network (GAN) approach.

User representation using a generative adversarial network

An approach to extending the recognizable labels of a label recognizer makes use of an encoding of linguistic inputs and label attributes into comparable vectors. The encodings may be determined with artificial neural networks (ANNs) that are jointly trained, and a comparison between the encoding of a sentence input and the encoding of an intent attribute vector may use a fixed function, which does not have to be trained. The encoding of label attributes can generalize permitting adding of a new label via corresponding attributes, thereby avoiding the need to immediately retrain a label recognizer with example inputs.

Extendable label recognition of linguistic input

Features are disclosed for generating models, such as conditional random field ('CRF') models, that consume less storage space and/or transmission bandwidth than conventional models. In some embodiments, the generated CRF models are composed of fewer or alternate components in comparison with conventional CRF models. For example, a system generating such CRF models may forgo the use of large dictionaries or other cross-reference lists that map information extracted from input (e.g., 'features') to model parameters; reduce in weight (or exclude altogether) certain model parameters that may not have a significant effect on model accuracy; and/or reduce the numerical precision of model parameters.

Conditional random field model compression

A linguist classifier, for instance intent or slot classifier, is updated using data with only partial annotation indicating overall correctness rather that specific correct intent or slot values, which are treated as 'latent' (i.e., unknown) variables. Full annotation of the data is not required. A small amount of fully annotated data may be combined with a substantially larger amount of partially annotated data to update the linguistic classifier. In a specific implementation, the linguistic classifier is a neural network and the weights are trained using a reinforcement learning approach.

Growing community of inventors

Seattle, WA, United States of America

Anjishnu Kumar