Google Inc.

International Business Machines Corporation

University of Michigan

Honglak Lee

Sergey Levine

Yunfei Bai

Seyed Mohammad Khansari Zadeh

Xinchen Yan

Jasmine Hsu

Abhinav Gupta

Irfan Essa

Weilong Yang

Shixiang Gu

James Christopher Davidson

Hung-Yu Tseng

Tianhao Zhang

Yinfei Yang

Jason Michael Baldridge

Arkanath Pathak

Lu Jiang

Jing Yu Koh

Quoc V Le

David Nahamoo

Rui Zhang

Han Zhang

Lazaros Polymenakos

Aleksandra Faust

Dragomir Radkov Radev

Soeren Pirk

Eugene Brevdo

Daiyi Peng

Yuanzheng Gong

Yingjie Miao

Jacob Buckman

Peter James Anderson

John Dalton Co-Reyes

Danijar Hafner

George Jay Tucker

Training and/or utilizing a hierarchical reinforcement learning (HRL) model for robotic control. The HRL model can include at least a higher-level policy model and a lower-level policy model. Some implementations relate to technique(s) that enable more efficient off-policy training to be utilized in training of the higher-level policy model and/or the lower-level policy model. Some of those implementations utilize off-policy correction, which re-labels higher-level actions of experience data, generated in the past utilizing a previously trained version of the HRL model, with modified higher-level actions. The modified higher-level actions are then utilized to off-policy train the higher-level policy model. This can enable effective off-policy training despite the lower-level policy model being a different version at training time (relative to the version when the experience data was collected).

Data-efficient hierarchical reinforcement learning

Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for generating and searching reinforcement learning algorithms. In some implementations, a computer-implemented system generates a sequence of candidate reinforcement learning algorithms. Each candidate reinforcement learning algorithm in the sequence is configured to receive an input environment state characterizing a state of an environment and to generate an output that specifies an action to be performed by an agent interacting with the environment. For each candidate reinforcement learning algorithm in the sequence, the system performs a performance evaluation for a set of a plurality of training environments. For each training environment, the system adjusts a set of environment-specific parameters of the candidate reinforcement learning algorithm by performing training of the candidate reinforcement learning algorithm to control a corresponding agent in the training environment. The system generates an environment-specific performance metric for the candidate reinforcement learning algorithm that measures a performance of the candidate reinforcement learning algorithm in controlling the corresponding agent in the training environment as a result of the training. After performing training in the set of training environments, the system generates a summary performance metric for the candidate reinforcement learning algorithm by combining the environment-specific performance metrics generated for the set of training environments. After evaluating each of the candidate reinforcement learning algorithms in the sequence, the system selects one or more output reinforcement learning algorithms from the sequence based on the summary performance metrics of the candidate reinforcement learning algorithms.

Reinforcement learning algorithm search

Implementations relate to training a point cloud prediction model that can be utilized to process a single-view two-and-a-half-dimensional (2.5D) observation of an object, to generate a domain-invariant three-dimensional (3D) representation of the object. Implementations additionally or alternatively relate to utilizing the domain-invariant 3D representation to train a robotic manipulation policy model using, as at least part of the input to the robotic manipulation policy model during training, the domain-invariant 3D representations of simulated objects to be manipulated. Implementations additionally or alternatively relate to utilizing the trained robotic manipulation policy model in control of a robot based on output generated by processing generated domain-invariant 3D representations utilizing the robotic manipulation policy model.

Robotic manipulation using domain-invariant 3D representations predicted from 2.5D vision data

A computer-implemented method includes receiving, by a computing device, a particular textual description of a scene. The method also includes applying a neural network for text-to-image generation to generate an output image rendition of the scene, the neural network having been trained to cause two image renditions associated with a same textual description to attract each other and two image renditions associated with different textual descriptions to repel each other based on mutual information between a plurality of corresponding pairs, wherein the plurality of corresponding pairs comprise an image-to-image pair and a text-to-image pair. The method further includes predicting the output image rendition of the scene.

Cross-modal contrastive learning for text-to-image generation based on machine learning models

A computing system for generating predicted images along a trajectory of unseen viewpoints. The system can obtain one or more spatial observations of an environment that may be captured from one or more previous camera poses. The system can generate a three-dimensional point cloud for the environment from the one or more spatial observations and the one or more previous camera poses. The system can project the three-dimensional point cloud into two-dimensional space to form one or more guidance spatial observations. The system can process the one or more guidance spatial observations with a machine-learned spatial observation prediction model to generate one or more predicted spatial observations. The system can process the one or more predicted spatial observations and image data with a machine-learned image prediction model to generate one or more predicted images from the target camera pose. The system can output the one or more predicted images.

Systems and methods for generating predicted visual observations of an environment using machine learned models

A method for generating an output image from an input image and an input text instruction that specifies a location and a modification of an edit applied to the input image using a neural network is described. The neural network includes an image encoder, an image decoder, and an instruction attention network. The method includes receiving the input image and the input text instruction; extracting, from the input image, an input image feature that represents features of the input image using the image encoder; generating a spatial feature and a modification feature from the input text instruction using the instruction attention network; generating an edited image feature from the input image feature, the spatial feature and the modification feature; and generating the output image from the edited image feature using the image decoder.

Image manipulation by text instruction

Deep machine learning methods and apparatus, some of which are related to determining a grasp outcome prediction for a candidate grasp pose of an end effector of a robot. Some implementations are directed to training and utilization of both a geometry network and a grasp outcome prediction network. The trained geometry network can be utilized to generate, based on two-dimensional or two-and-a-half-dimensional image(s), geometry output(s) that are: geometry-aware, and that represent (e.g., high-dimensionally) three-dimensional features captured by the image(s). In some implementations, the geometry output(s) include at least an encoding that is generated based on a trained encoding neural network trained to generate encodings that represent three-dimensional features (e.g., shape). The trained grasp outcome prediction network can be utilized to generate, based on applying the geometry output(s) and additional data as input(s) to the network, a grasp outcome prediction for a candidate grasp pose.

Robotic grasping prediction using neural networks and geometry aware object representation

An information processing system, a computer program product, and methods for modeling multi-party dialog interactions. A method includes learning, directly from data obtained from a multi-party conversational channel, to identify particular multi-party dialog threads as well as participants in one or more conversations. Each participant utterance being converted to a continuous vector representation updated in a model of the multi-party dialog relative to each participant utterance and according to each participant's role selected from the set of: sender, addressee, or observer. The method trains the model to choose a correct addressee and a correct response for each participant utterance, using a joint selection criterion. The method learns directly from the data obtained from the multi-party conversational channel, which dialog turns belong to each particular multi-party dialog thread.

Modeling multiparty conversation dynamics: speaker, response, addressee selection using a novel deep learning approach

Growing community of inventors

Ann Arbor, MI, United States of America

Honglak Lee