X Development LLC

Google Inc.

Facebook, Inc.

Anybots, Inc.

Anybots 2.0, Inc.

Benjamin Holson

Jeffrey Bingham

Ben Berkowitz

Scott C Wiley

Trevor Blackwell

Daniel Casner

Dan Barak

Eden Rephaeli

Mrinal Kalakrishnan

Adrian Li

Guy Satat

Daniel Alex Lam

Alexander Herzog

Jiajun Xu

Gabriel Aaron Cohen

Munaf Assaf

John L Sokol

Daniel Lam

Implementations disclosed herein relate to utilizing at least one existing manually engineered policy, for a robotic task, in training an RL policy model that can be used to at least selectively replace a portion of the engineered policy. The RL policy model can be trained for replacing a portion of a robotic task and can be trained based on data from episodes of attempting performance of the robotic task, including episodes in which the portion is performed based on the engineered policy and/or other portion(s) are performed based on the engineered policy. Once trained, the RL policy model can be used, at least selectively and in lieu of utilization of the engineered policy, to perform the portion of robotic task, while other portion(s) of the robotic task are performed utilizing the engineered policy and/or other similarly trained (but distinct) RL policy model(s).

Training a policy model for a robotic task, using reinforcement learning and utilizing data that is based on episodes, of the robotic task, guided by an engineered policy

A method includes receiving sensor data representative of surfaces in a physical environment containing an interaction point for a robotic device and determining, based on the sensor data, a height map of the surfaces in the physical environment. The method also includes determining, by inputting the height map and the interaction point into a pre-trained model, one or more candidate positions for a base of the robotic device to allow a manipulator of the robotic device to reach the interaction point. The method additionally includes determining a collision-free trajectory to be followed by the manipulator of the robotic device to reach the interaction point when the base of the robotic device is positioned at a selected candidate position of the one or more candidate positions and, based on determining the collision-free trajectory, causing the base of the robotic device to move to the selected candidate position within the physical environment.

Robot base position planning

Methods, systems, and apparatus, including computer-readable storage devices, for robot navigation using 2D and 3D path planning. In the disclosed method, a robot accesses map data indicating two-dimensional layout of objects in a space and evaluates candidate paths for the robot to traverse. In response to determining that the candidate paths do not include a collision-free path across the space for a two-dimensional profile of the robot, the robot evaluates a three-dimensional shape of the robot with respect to a three-dimensional shape of an object in the space. Based on the evaluation of the three-dimensional shapes, the robot determines a collision-free path to traverse through the space.

Robot navigation using 2D and 3D path planning

Methods, apparatus, and computer readable media applicable to balancing robots. Some implementations are directed to maintaining a given end effector pose (relative to a world frame) of an end effector of a balancing robot when there is a disturbance to a balancing base of the balancing robot. Some implementations are additionally or alternatively directed to transitioning a balancing robot from a fallen configuration to a balanced configuration. Some implementations are additionally or alternatively directed to mitigating the risk that a balancing robot will fall when interacting with actuable environmental objects (e.g., doors) and/or to lessen the disturbance to a balancing base when interacting with actuable environmental objects.

Robot control

A system includes a robotic device, a sensor disposed on the robotic device, and circuitry configured to perform operations. The operations include determining a map that represents stationary features of an environment and receiving, from the sensor, sensor data representing the environment. The operations also include determining, based on the sensor data, a representation of an actor within the environment, where the representation includes keypoints representing corresponding body locations of the actor. The operations also include determining that a portion of a particular stationary feature is positioned within a threshold distance of a particular keypoint and, based on thereon, updating the map to indicate that the portion is to be cleaned. The operations further include, based on the map as updated, causing the robotic device to clean the portion of the particular stationary feature.

Monitoring of surface touch points for precision cleaning

Utilizing at least one existing policy (e.g. a manually engineered policy) for a robotic task, in generating reinforcement learning (RL) data that can be used in training an RL policy for an instance of RL of the robotic task. The existing policy can be one that, standing alone, will not generate data that is compatible with the instance of RL for the robotic task. In contrast, the generated RL data is compatible with RL for the robotic task at least by virtue of it including state data that is in a state space of the RL for the robotic task, and including actions that are in the action space of the RL for the robotic task. The generated RL data can be used in at least some of the initial training for the RL policy using reinforcement learning.

Generating reinforcement learning data that is compatible with reinforcement learning for a robotic task

A method includes receiving sensor data representative of surfaces in a physical environment containing an interaction point for a robotic device, and determining, based on the sensor data, a height map of the surfaces in the physical environment. The method also includes determining, by inputting the height map and the interaction point into a pre-trained model, one or more candidate positions for a base of the robotic device to allow a manipulator of the robotic device to reach the interaction point. The method additionally includes determining a collision-free trajectory to be followed by the manipulator to reach the interaction point when the base of the robotic device is positioned at a selected candidate position of the one or more candidate positions and, based on determining the collision-free trajectory, causing the base of the robotic device to move to the selected candidate position within the physical environment.

Methods, apparatus, and computer readable media applicable to robots, such as balancing robots. Some implementations are directed to determining multiple measures of a property of a robot for a given time and determining a final measure of the property of the robot for the given time based on the multiple measures. One or more control commands may be generated based on the final measure of the property and provided to one or more actuators of the robot.

Sensor fusion

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selective human-robot interaction. In some implementations, sensor data describing an environment of a robot is received, and a person in the environment of the robot is detected based on the sensor data. Scores indicative of properties of the detected person are generated based on the sensor data and processed using a machine learning model. Processing the scores can produce one or more outputs indicative of a likelihood that the detected person will perform a predetermined action in response to communication from the robot. Based on the one or more outputs of the machine learning model, the robot initiates communication with the detected person.

Selective human-robot interaction

An example implementation includes a robotic system including a first wheel and a second wheel configured to rotate about a first axis. Each wheel of the first wheel and the second wheel includes a contact surface and a motor coupled to a rotatable component. Each motor is configured to rotate the rotatable component about a respective second axis. The rotatable component is frictionally engaged with the contact surface such that a rotation of the rotatable component about the respective second axis is translated to a rotation of the wheel about the first axis. The robotic system further includes a controller configured to operate the motor of the first wheel and the motor of the second wheel in order to cause the robotic system to maintain its balance and navigate within an environment based on data received from one or more sensors.

Growing community of inventors

Sunnyvale, CA, United States of America

Benjamin Holson