Google Inc.

Waymo LLC

Sheng Zhao

Volker Grabe

Esha D Nerurkar

Simon Lynen

Craig Lewin Robinson

Nicholas Armstrong-Crews

Changchang Wu

Antonio Teran Espinoza

Damien Dusha

Esha Nerurkar

The described aspects and implementations enable efficient calibration of a sensing system of an autonomous vehicle (AV). In one implementation, disclosed is a method and a system to perform the method, the system including the sensing system configured to collect sensing data and a data processing system, operatively coupled to the sensing system. The data processing system is configured to identify reference point(s) in an environment of the AV, determine multiple estimated locations of the reference point(s), and adjust parameters of the sensing system based on a loss function representative of differences of the estimated locations.

Calibration of sensors in autonomous vehicle applications

An improved, efficient method for mapping world points from an environment (e.g., points generated by a LIDAR sensor of an autonomous vehicle) to locations (e.g., pixels) within rolling-shutter images taken of the environment is provided. This improved method allows for accurate localization of the world point in a rolling-shutter image via an iterative process that converges in very few iterations. The method poses the localization process as an iterative process for determining the time, within the rolling-shutter exposure period of the image, at which the world point was imaged by the camera. The method reduces the number of times the world point is projected into the normalized space of the camera image, often converging in three or fewer iterations.

Efficient algorithm for projecting world points to a rolling shutter image

Systems and methods for extrinsic calibration of vehicle-mounted sensors are provided. One example method involves obtaining first sensor data collected by a first sensor and a second sensor while a vehicle is aligned in a first yaw direction. The method also involves obtaining second sensor data collected by the first sensor and the second sensor while the vehicle is aligned in a second yaw direction. The method also involves determining, based on the first sensor data and the second sensor data, (i) first pitch and roll misalignments of the first sensor relative to the vehicle and (ii) second pitch and roll misalignments of the second sensor relative to the first sensor. The method also involves determining third pitch and roll misalignments of the second sensor relative to the vehicle based on (i) the first pitch and roll misalignments and (ii) the second pitch and roll misalignments.

Extrinsic calibration of sensors mounted on a vehicle

An electronic device tracks its motion in an environment while building a three-dimensional visual representation of the environment that is used to correct drift in the tracked motion. A motion tracking module estimates poses of the electronic device based on feature descriptors corresponding to the visual appearance of spatial features of objects in the environment. A mapping module builds a three-dimensional visual representation of the environment based on a stored plurality of maps, and feature descriptors and estimated device poses received from the motion tracking module. The mapping module provides the three-dimensional visual representation of the environment to a localization module, which identifies correspondences between stored and observed feature descriptors. The localization module performs a loop closure by minimizing the discrepancies between matching feature descriptors to compute a localized pose. The localized pose corrects drift in the estimated pose generated by the motion tracking module.

System and method for concurrent odometry and mapping

An electronic device detects and recovers from fault conditions while tracking its motion and building a map of its environment. A front-end motion tracking module detects fault conditions while tracking motion over time and provides mapping data to a back-end mapping module. The front-end motion tracking module indicates to the back-end mapping module when a fault condition has been detected and when the fault condition is no longer detected. The back-end mapping module generates maps from the mapping data and merges the maps to build a three-dimensional representation of the environment. The back-end mapping module buffers or discards any mapping data received from the front-end motion tracking module during the existence of a fault condition. The back-end mapping module merges the maps generated based on mapping data received before and after the fault condition and adds the merged maps to the three-dimensional representation of the environment.

Growing community of inventors

Mountain View, CA, United States of America

Sheng Zhao