Google Inc.

Gdm Holding LLC

Ammar Husain

Joerg Mueller

Mikael Persson

Ting Lu

A method includes determining, for a robotic device that comprises a perception system, a robot planner state representing at least one future path for the robotic device in an environment. The method also includes determining a perception system trajectory by inputting at least the robot planner state into a machine learning model trained based on training data comprising at least a plurality of robot planner states corresponding to a plurality of operator-directed perception system trajectories. The method further includes controlling, by the robotic device, the perception system to move through the determined perception system trajectory.

Learning from demonstration for determining robot perception motion

Implementations set forth herein relate to generating training data, such that each instance of training data includes a corresponding instance of vision data and drivability label(s) for the instance of vision data. A drivability label can be determined using first vision data from a first vision component that is connected to the robot. The drivability label(s) can be generated by processing the first vision data using geometric and/or heuristic methods. Second vision data can be generated using a second vision component of the robot, such as a camera that is connected to the robot. The drivability labels can be correlated to the second vision data and thereafter used to train one or more machine learning models. The trained models can be shared with a robot(s) in furtherance of enabling the robot(s) to determine drivability of areas captured in vision data, which is being collected in real-time using one or more vision components.

Generating and/or using training instances that include previously captured robot vision data and drivability labels

A method includes receiving, from a sensor on a robotic device, a captured image representative of an environment of the robotic device when the robotic device is at a location in the environment. The method also includes determining, based at least on the location of the robotic device, a rendered image representative of the environment of the robotic device. The method further includes determining, by applying at least one pre-trained machine learning model to at least the captured image and the rendered image, a property of one or more portions of the captured image.

Joint training of a narrow field of view sensor with a global map for broader context

A method includes receiving one or more past trajectories navigated by a robotic device in an environment, wherein the one or more past trajectories are associated with initial environmental sensor data and one or more obstacle detection heuristics. The method also includes determining, based at least on subsequent environmental sensor data, one or more updated obstacle detection heuristics. The method further includes determining, based on the one or more updated obstacle detection heuristics and the initial environmental sensor data, one or more predicted drivable areas in the environment. The method additionally includes, based on the one or more predicted drivable areas including the one or more past trajectories, using the one or more updated obstacle detection heuristics to determine future navigation of the robotic device.

Growing community of inventors

San Francisco, CA, United States of America

Ammar Husain