Amazon Technologies, Inc.

Joe Pemberton

Ruhi Sarikaya

Shubham Katiyar

Rajesh Kumar Pandey

Isaac Joseph Madwed

Arun Kumar Thenappan

Young-Bum Kim

David Robert Thomas

Julia Kennedy Nemer

Vijitha Raji

Mohamed Farouk AbdelHady

Archit Jain

Jihwan Lee

Michael F Schwartz

Alexander Say Go

Steven Mack Saunders

Prasha Shrestha

Hao Yuan

Kai Chieh Liu

Xiaohu Liu

Chenlei Guo

Dhruva Lakshmana Rao Batni

Narendra Gyanchandani

Bigyan Rajbhandari

Zheng Du

Junqing Shang

Zhenyu Yao

Hung Tuan Pham

Lawrence Mariadas Chettiar

Artun Kutchuk

Naushad Zaveri

Sriharsha Venkata Chintalapati

Tara Raj

Jae Young Do

Mohammad Kachuee

Sahil Malik

Siddharth Mohan Misra

Arima Vu Ram Thayumanavar

Wangyao Ge

Rushi P Desai

Liyuan Zhang

Devices and techniques are generally described for incremental, asynchronous machine learning inference. In various examples, first feature data generated from a first process during a first time period may be input into a first subgraph of a neural network. Second feature data generated from the first process during a second time period may be input into a second subgraph of the neural network. First intermediate data may be generated by the first subgraph based on the first feature data. Second intermediate data may be generated by the second subgraph based on the second feature data. First output data may be generated by the neural network based on the first intermediate data and the second intermediate data.

Machine learning model architecture for incremental asynchronous inference

Techniques for selecting a skill component to process a natural language input are described. When a natural language input is received, natural language understanding (NLU) output data representing the natural language input is generated, and skill components (capable of processing the NLU output data) are determined. Thereafter, rules (for preventing the invocation of skill components) are implemented in a tiered manner, resulting in the determination of a subset of the skill components. The subset of skill components is ranked using a machine learning model(s), and the top-ranked skill component is called to process the NLU output data.

Natural language input routing

Techniques for processing speculative data using data history information are described. A system may receive, at a first component, first data and second data for use in a first operation. The component may use metadata associated with each of the first data and the second data to determine that the first data and the second data correspond to different versions of data of the same data type. Based on such a determination, the component may cease processing corresponding to the first operation with respect to the first data and the second data.

Processing speculative data using data history information

Devices and techniques are generally described for a speech processing routing architecture. First input data representing an input request may be received. First data including a semantic interpretation of the input request may be determined. Metadata of the first input data may be determined. The metadata may identify an entity associated with the input request. In some examples, a query may be sent to a first component. The query may include the metadata. In some examples, second data that identifies a first skill associated with the entity may be received from the first component. In various examples, the first skill may be selected for processing the first input data based at least in part on the first data and the second data.

Natural language processing routing

Techniques for performing centralized unsuperivised learning in a multi-domain system are described. A user may request labeled data for an ML task, where the request includes a prompt for obtaining relevant explicit user feedback. The system may use the prompt to collect explicit user feedback for relevant runtime user inputs. After a duration of time (in the user's request for labeled data) has elapsed, the system determines whether collected user feedback indicates processing of the user input was defective and, if so, determines a cause of the defective processing. The system then uses one or more label generators to generate labeled data using the collected user feedback, whether the processing was defective, and the determined defect cause.

Machine learning label generation

Techniques for implementing a streaming remote procedure call (RPC) mechanism using distributed processing components of a system are described. A first processing component sends a connect message to a second processing component. Thereafter, the first processing component sends different instances of data to the second processing component as the different instances of data are determined by the first processing component. The second processing component performs at least some processes as the second processing component receives the different instances of data. After the first processing component sends all relevant data to the second processing component, the first processing component sends a commit message to the second processing component. Based at least in part on receiving the commit message, the second processing component determines finishes its processing, and sends result data to the first processing component.

Processing orchestration for systems including distributed components

Devices and techniques are generally described for a speech processing routing architecture. First input data representing an input request may be received. First data may be sent to a first skill representing a first request for the first skill to evaluate an ability of the first skill to process the first input data. Second data may be sent to a second skill representing a second request for the second skill to evaluate an ability of the second skill to process the first input data. Third data may be received from the first skill indicating a first action performed by the first skill in response to receipt of the first input data. Fourth data may be received from the second skill indicating a second action performed by the second skill. The first skill may be selected for processing the first input data.

Devices and techniques are generally described for a speech processing routing architecture. In various examples, first data comprising a first feature definition is received. The first feature definition may include a first indication of first source data and first instructions for generating feature data using the first source data. In various examples, the feature data may be generated according to the first feature definition. In some examples, a speech processing system may receive a first request to process a first utterance. The feature data may be retrieved from a non-transitory computer-readable memory. The speech processing system may determine a first skill for processing the first utterance based at least in part on the feature data.

Techniques for performing runtime ranking of skill components are described. A skill developer may generate a rule indicating a skill component is to be invoked at runtime when a natural language input corresponds to a specific context. At runtime, a virtual assistant system may implement a machine learned model to generate an initial ranking of skill components. Thereafter, the virtual assistant system may use skill component-specific rules to adjust the initial ranking, and this second ranking is used to determine which skill component to invoke to respond to the natural language input. Overtime, if a rule results in beneficial user experiences, the virtual assistant system may incorporate the rule into the machine learned model.

Devices and techniques are generally described for using user feedback to determine routing decisions in a speech processing system. In various examples, first data representing a first utterance may be received. Second data representing a first semantic interpretation of the first utterance may be determined. A first intent data processing application may be selected for processing the second data. Feedback data may be determined related to the first intent data processing application processing the second data. Third data representing a semantic interpretation of a second utterance may be received, wherein the first semantic interpretation is the same as the second semantic interpretation. A second intent data processing application may be determined for processing the third data based at least in part on the feedback data.

Natural language processing

Routing of natural language inputs to speech processing applications

Growing community of inventors

Seattle, WA, United States of America

Joe Pemberton