Amazon Technologies, Inc.

Toyota Jidosha Kabushiki Kaisha

Toyota Research Institute, Inc.

Thomas Kollar

Spyridon Matsoukas

Avnish Sikka

Simon Peter Reavely

Cengiz Erbas

Arindam Mandal

Lambert Leo Mathias

Kevin Stone

Angeliki Metallinou

Michael Laskey

Mark Edward Tjersland

A robotic system is contemplated. The robotic system comprises a robot comprising a camera, a microphone, memory, and a controller that is configured to receive a natural language command for performing an action within a real world environment, parse the natural language command, categorize the action as being associated with guidance for performing the action, receive the guidance for performing the action, the guidance including a motion applied to at least one portion of the robot within the real world environment for performing the action, and store, in the memory, the natural language command in correlation with the motion that is applied to the at least one portion of the robot.

Motion guidance and natural language commands based robotic systems

A system capable of performing natural language understanding (NLU) on utterances including complex command structures such as sequential commands (e.g., multiple commands in a single utterance), conditional commands (e.g., commands that are only executed if a condition is satisfied), and/or repetitive commands (e.g., commands that are executed until a condition is satisfied). Audio data may be processed using automatic speech recognition (ASR) techniques to obtain text. The text may then be processed using machine learning models that are trained to parse text of incoming utterances. The models may identify complex utterance structures and may identify what command portions of an utterance go with what conditional statements. Machine learning models may also identify what data is needed to determine when the conditionals are true so the system may cause the commands to be executed (and stopped) at the appropriate times.

Processing complex utterances for natural language understanding

A method for 3D object perception is described. The method includes extracting features from each image of a synthetic stereo pair of images. The method also includes generating a low-resolution disparity image based on the features extracted from each image of the synthetic stereo pair images. The method further includes predicting, by a trained neural network, a feature map based on the low-resolution disparity image and one of the synthetic stereo pair of images. The method also includes generating, by a perception prediction head, a perception prediction of a detected 3D object based on the feature map predicted by the trained neural network.

System and method for 3D object perception trained from pure synthetic stereo data

A system capable of performing natural language understanding (NLU) without the concept of a domain that influences NLU results. The present system uses a hierarchical organizations of intents/commands and entity types, and trained models associated with those hierarchies, so that commands and entity types may be determined for incoming text queries without necessarily determining a domain for the incoming text. The system thus operates in a domain agnostic manner, in a departure from multi-domain architecture NLU processing where a system determines NLU results for multiple domains simultaneously and then ranks them to determine which to select as the result.

Fine-grained natural language understanding

Growing community of inventors

San Jose, CA, United States of America

Thomas Kollar