Google Inc.

X Development LLC

Alexander Toshev

Jay Yagnik

Vivek Kwatra

Dumitru Erhan

Sergey Ioffe

Oriol Vinyals

Samy Bengio

Nicolas Hudson

Krishna Shankar

Kuan Fang

King Hong Thomas Leung

Dragomir Dimitrov Anguelov

Sergey Levine

Anelia Angelova

Christian Szegedy

Garth Conboy

Seyed Mohammad Khansari Zadeh

Saurabh Singh

Debajit Ghosh

Soeren Pirk

John Rivlin

Poonam Suryanarayan

Xinxing Gu

Anthony Jacob Piergiovanni

Marek Fiser

Michael Sahngwon Ryoo

Yair Movshovitz-Attias

Jiwoong Jack Sim

Xiangxin Zhu

Karol Hausman

Ayzaan Wahid

Fereshteh Sadeghi

Deva K Ramanan

Yangqing Jia

Yunchao Gong

Matthew William Vaughan-Vail

Greg Don Hartrell

King Hong Leung

Techniques are disclosed that enable the generation of predicted sequences of terminals using a generator model portion of a prediction model. Various implementations include controlling actuators of a robot based on the predicted sequences of terminals. Additional or alternative implementations include jointly training the generator model portion of the prediction model using a discriminator model portion of the prediction model using, for example, stochastic adversarial based sampling.

Future prediction, using stochastic adversarial based sampling, for robotic control and/or other purpose(s)

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling an agent. One of the methods includes receiving a current observation characterizing a current state of the environment as of the time step; generating an embedding of the current observation; processing scene memory data comprising embeddings of prior observations received at prior time steps using an encoder neural network, wherein the encoder neural network is configured to apply an encoder self-attention mechanism to the scene memory data to generate an encoded representation of the scene memory data; processing the encoded representation of the scene memory data and the embedding of the current observation using a decoder neural network to generate an action selection output; and causing the agent to perform the selected action.

Controlling agents using scene memory data

Methods, apparatus, and computer-readable media for determining and utilizing corrections to robot actions. Some implementations are directed to updating a local features model of a robot in response to determining a human correction of an action performed by the robot. The local features model is used to determine, based on an embedding generated over a corresponding neural network model, one or more features that are most similar to the generated embedding. Updating the local features model in response to a human correction can include updating a feature embedding, of the local features model, that corresponds to the human correction. Adjustment(s) to the features model can immediately improve robot performance without necessitating retraining of the corresponding neural network model.

Update of local features model based on correction to robot action

Training and/or using a machine learning model for performing robotic tasks is disclosed herein. In many implementations, an environment-conditioned action sequence prediction model is used to determine a set of actions as well as a corresponding particular order for the actions for the robot to perform to complete the task. In many implementations, each action in the set of actions has a corresponding action network used to control the robot in performing the action.

Determining environment-conditioned action sequences for robotic tasks

Training and/or using both a high-level policy model and a low-level policy model for mobile robot navigation. High-level output generated using the high-level policy model at each iteration indicates a corresponding high-level action for robot movement in navigating to the navigation target. The low-level output generated at each iteration is based on the determined corresponding high-level action for that iteration, and is based on observation(s) for that iteration. The low-level policy model is trained to generate low-level output that defines low-level action(s) that define robot movement more granularly than the high-level action—and to generate low-level action(s) that avoid obstacles and/or that are efficient (e.g., distance and/or time efficiency).

Robot navigation using a high-level policy model and a trained low-level policy model

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating descriptions of input images. One of the methods includes obtaining an input image; processing the input image using a first neural network to generate an alternative representation for the input image; and processing the alternative representation for the input image using a second neural network to generate a sequence of a plurality of words in a target natural language that describes the input image.

Generating natural language descriptions of images

Training and/or using a recurrent neural network model for visual servoing of an end effector of a robot. In visual servoing, the model can be utilized to generate, at each of a plurality of time steps, an action prediction that represents a prediction of how the end effector should be moved to cause the end effector to move toward a target object. The model can be viewpoint invariant in that it can be utilized across a variety of robots having vision components at a variety of viewpoints and/or can be utilized for a single robot even when a viewpoint, of a vision component of the robot, is drastically altered. Moreover, the model can be trained based on a large quantity of simulated data that is based on simulator(s) performing simulated episode(s) in view of the model. One or more portions of the model can be further trained based on a relatively smaller quantity of real training data.

Viewpoint invariant visual servoing of robot end effector using recurrent neural network

The present disclosure provides systems and methods that enable distance metric learning using proxies. A machine-learned distance model can be trained in a proxy space in which a loss function compares an embedding provided for an anchor data point of a training dataset to a positive proxy and one or more negative proxies, where each of the positive proxy and the one or more negative proxies serve as a proxy for two or more data points included in the training dataset. Thus, each proxy can approximate a number of data points, enabling faster convergence. According to another aspect, the proxies of the proxy space can themselves be learned parameters, such that the proxies and the model are trained jointly. Thus, the present disclosure enables faster convergence (e.g., reduced training time). The present disclosure provides example experiments which demonstrate a new state of the art on several popular training datasets.

Distance metric learning using proxies

An example method is disclosed that includes identifying a training set of images, wherein each image in the training set has an identified bounding box that comprises an object class and an object location for an object in the image. The method also includes segmenting each image of the training set, wherein segments comprise sets of pixels that share visual characteristics, and wherein each segment is associated with an object class. The method further includes clustering the segments that are associated with the same object class, and generating a data structure based on the clustering, wherein entries in the data structure comprise visual characteristics for prototypical segments of objects having the object class and further comprise one or more potential bounding boxes for the objects, wherein the data structure is usable to predict bounding boxes of additional images that include an object having the object class.

System and method for using segmentation to identify object location in images

Growing community of inventors

San Francisco, CA, United States of America

Alexander Toshev