Google Inc.

Alexander Irpan

Sergey Levine

Yunfei Bai

Seyed Mohammad Khansari Zadeh

Julian Ibarz

Eric Jang

Matthew Bennice

Daniel Alex Lam

Konstantinos Bousmalis

Chelsea Finn

Brent Austin

Daniel Kappler

Nicolas Sievers

Vincent Olivier Vanhoucke

Kurt Konolige

Mrinal Kalakrishnan

Christopher Kenneth Harris

Kanury Kanishka Rao

Paul Wohlhart

Matthew Laurance Kelcey

Kanishka Rao

Chris Harris

Sergey Vladimir Levine

Implementations disclosed herein relate to mitigating the reality gap through training a simulation-to-real machine learning model ('Sim2Real' model) using a vision-based robot task machine learning model. The vision-based robot task machine learning model can be, for example, a reinforcement learning (“RL”) neural network model (RL-network), such as an RL-network that represents a Q-function.

Mitigating reality gap through training a simulation-to-real model using a vision-based robot task model

Implementations described herein relate to training and refining robotic control policies using imitation learning techniques. A robotic control policy can be initially trained based on human demonstrations of various robotic tasks. Further, the robotic control policy can be refined based on human interventions while a robot is performing a robotic task. In some implementations, the robotic control policy may determine whether the robot will fail in performance of the robotic task, and prompt a human to intervene in performance of the robotic task. In additional or alternative implementations, a representation of the sequence of actions can be visually rendered for presentation to the human can proactively intervene in performance of the robotic task.

System(s) and method(s) of using imitation learning in training and refining robotic control policies

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for off-policy evaluation of a control policy. One of the methods includes obtaining policy data specifying a control policy for controlling a source agent interacting with a source environment to perform a particular task; obtaining a validation data set generated from interactions of a target agent in a target environment; determining a performance estimate that represents an estimate of a performance of the control policy in controlling the target agent to perform the particular task in the target environment; and determining, based on the performance estimate, whether to deploy the control policy for controlling the target agent to perform the particular task in the target environment.

Off-policy control policy evaluation

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training an action selection neural network that is used to control a robotic agent interacting with a real-world environment.

Using simulation and domain adaptation for robotic control

Growing community of inventors

Palo Alto, CA, United States of America

Alexander Irpan