X Development LLC

Google Inc.

Intrinsic Innovation LLC

Kevin William Watts

Kurt Konolige

Ethan Rublee

Frank Van Diggelen

John William Zevenbergen

Troy Straszheim

Stefan Nusser

Mohammed Khider

Peter Elving Anderson-Sprecher

Imad Fattouch

Hauke Malte Strasdat

Julian Mac Neille Mason

Imad Jean Fattouch

Zifei Tong

Arastoo Zabolestani Moradi

Kyle Lutz

David William Earle Orendorff

David Orendorff

Guoyu Fu

A method of processing signal paths includes receiving an estimated location for a GNSS receiver in an environment. The method also includes generating a plurality of candidate positions about the estimated location where each candidate position corresponds to a possible actual location of the GNSS receiver. The method further includes, for each available satellite at each candidate position, modeling a plurality of candidate signal paths by ray-launching a raster map of geographical data. Here, the plurality of candidate signal paths includes one or more reflected signal paths. At each candidate position, the method also includes comparing, the plurality of candidate signal paths modeled for each available satellite at the respective candidate position to measured GNSS signal data from the GNSS receiver and generating a likelihood that the respective candidate position includes the actual location of the GNSS receiver based on the comparison.

GNSS signal modeling

A method of localization using bearing from environmental features includes receiving an estimated location of a global navigation satellite system (GNSS) receiver associated with a user and a corresponding bearing for the GNSS receiver. The method also includes identifying one or more environmental features about the estimated location of the GNSS receiver. The method further includes determining whether an orientation of a respective environmental feature of the one or more environmental features correlates to the corresponding bearing for the GNSS receiver. When the orientation of the respective environmental feature correlates to the corresponding bearing for the GNSS receiver, the method includes generating an updated bearing for the GNSS receiver or locational system that matches the orientation of the respective environmental feature.

Localization using bearing from environmental features

Example systems and methods are disclosed for implementing vehicle operation limits to prevent vehicle load failure during vehicle teleoperation. The method may include receiving sensor data from sensors on a vehicle that carries a load. The vehicle may be controlled by a remote control system. The load weight and dimensions may be determined based on the sensor data. In order to prevent a vehicle load failure, a forward velocity limit and an angular velocity limit may be calculated. Vehicle load failures may include the vehicle tipping over, the load tipping over, the load sliding off of the vehicle, or collisions. The vehicle carrying the load may be restricted from exceeding the forward velocity limit and/or the angular velocity limit during vehicle operation. The remote control system may display a user interface indicating to a remote operator the forward velocity limit and the angular velocity limit.

Virtual moving safety limits for vehicles transporting objects

An example method includes determining a target area of a ground plane in an environment of a mobile robotic device, where the target area of the ground plane is in front of the mobile robotic device in a direction of travel of the mobile robotic device. The method further includes receiving depth data from a depth sensor on the mobile robotic device. The method also includes identifying a portion of the depth data representative of the target area. The method additionally includes determining that the portion of the depth data lacks information representing at least one section of the target area. The method further includes providing an output signal identifying at least one zone of non-traversable space for the mobile robotic device in the environment, where the at least one zone of non-traversable space corresponds to the at least one section of the target area.

Ground plane detection to verify depth sensor status for robot navigation

An example system includes one or more laser sensors on a robotic device, where the one or more laser sensors are configured to produce laser sensor data indicative of a first area within a first distance in front of the robotic device. The system further includes one or more stereo sensors on the robotic device, where the stereo sensors on the robotic device are configured to produce stereo sensor data indicative of a second area past a second distance in front of the robotic device. The system also includes a controller configured to receive the laser sensor data, receive the stereo sensor data, detect one or more objects in front of the robotic device based on at least one of the laser sensor data and the stereo sensor data, and provide instructions for the robotic device to navigate based on the one or more detected objects.

Using laser sensors to augment stereo sensor readings for robotic devices

Example implementations may relate to methods and systems for determining a safe trajectory for movement of an object by a robotic system. According to these various implementations, the robotic system may determine at least first and second candidate trajectories for moving the object. For at least a first point along the first candidate trajectory, the robotic system may determine a predicted cost of dropping the object at the first point along the first candidate trajectory. And for at least a second point along the second candidate trajectory, the robotic system may determine a predicted cost of dropping the object at the second point along the second candidate trajectory. Then, based on these various determined predicted costs, the robotic system may select between the first and second candidates trajectories and may then move the object along the selected trajectory.

Trajectory planning with droppable objects

An example method includes receiving instructions to pick up an object with one or more lift elements of an autonomous vehicle. Based on a current positioning of the vehicle, the method further includes identifying the object to be picked up and a particular side of the object under which to place the one or more lift elements of the vehicle. The method additionally includes determining an approach path toward the object for the vehicle to follow to place the lift elements of the vehicle under the particular side of the object. The method further includes causing the vehicle to move along the determined approach path toward the object. The method additionally includes determining that the lift elements of the vehicle are placed under the particular side of the object. The method also includes causing the vehicle to lift the object with the lift elements.

Autonomous approach and object pickup

Systems and methods are provided for automated route discovery. A computing device can receive location data for designated actor(s) of a plurality of actors operating within an environment. The plurality of actors can also include a robotic device. The computing device can determine a route network of paths taken by the designated actor(s) within the environment, where the route network includes information about frequencies of paths taken by the designated actor(s) based on the location data. The computing device can receive a starting location and a destination location for the robotic device. The computing device can select a selected path from the starting location to the destination location based on the route network taken by the designated actor(s). The computing device can provide an instruction to the robotic device to use the selected path to travel from the starting location to the destination location.

Automated lane and route network discovery for robotic actors

Methods and systems for remote perception assistance to facilitate robotic object manipulation are provided herein. From a model of objects in an environment of a robotic manipulator, a virtual boundary line separating two adjacent identified virtual objects may be identified. The robotic manipulator may be configured to perform a task involving a manipulation of at least one object in the environment represented by the identified virtual objects. Based on the identifying, a request for remote assistance with verifying the virtual boundary line may be sent to a remote assistor device. A response to the request, including a modification to the virtual boundary line, may then be received from the remote assistor device. The robotic manipulator may then be caused to perform the task based on the modification to the virtual boundary line.

Methods and systems for remote perception assistance to facilitate robotic object manipulation

Disclosed herein are methods and systems for determining a safe path for movement of an object by a robotic system. According to these implementations, the robotic system may determine a safety level for each of a plurality of relative orientations of an object. Each such relative orientation may define a spatial orientation of the object relative to direction of movement of the object. Based on the determined safety levels, the robotic system may then determine, for each of the plurality of relative orientations, a velocity limit for movement of the object with a particular relative orientation. Based at least in part on the determined velocity limits, the robotic system may then determine a path for moving the object from a first location to a second location. As such, the robotic system may move the object from the first location to the second location based on the determined path.

Safe path planning for collaborative robots

Methods and systems for distributing remote assistance to facilitate robotic object manipulation are provided herein. Regions of a model of objects in an environment of a robotic manipulator may be determined, where each region corresponds to a different subset of objects with which the robotic manipulator is configured to perform a respective task. Certain tasks may be identified, and a priority queue of requests for remote assistance associated with the identified tasks may be determined based on expected times at which the robotic manipulator will perform the identified tasks. At least one remote assistor device may then be requested, according to the priority queue, to provide remote assistance with the identified tasks. The robotic manipulator may then be caused to perform the identified tasks based on responses to the requesting, received from the at least one remote assistor device, that indicate how to perform the identified tasks.

Growing community of inventors

Palo Alto, CA, United States of America

Kevin William Watts