Waymo LLC

X Development LLC

Colin Andrew Braley

Volker Grabe

Christian Lauterbach

Volodymyr V Ivanchenko

Craig Lewin Robinson

Mingcheng Chen

Stephanie McArthur

Michael Steven Montemerlo

Carlos Hernandez Esteban

Andrew Crawford

Zachary Morriss

David Yonchar Margines

Xiaoxiang Hu

Gil Shotan

Alexander Meade

Dragomir Dimitrov Anguelov

Behnam Bastani

Mark Alexander Shand

Luke Wachter

Peter Michal Pawlowski

Aleksandar Gabrovski

Nicholas Charles Loomis

Christoph Sprunk

Ming Zou

Pierre-yves Droz

Mary Lou Jepsen

Yu Zhang

Caner Onal

Ralph Hamilton Shepard

Nicholas Armstrong-Crews

Michael Buchanan

Dorel Ionut Iordache

Carlin Vieri

Hui Seong Son

Arthur Safira

Yunshan Jiang

Ionut Iordache

Antonio Teran Espinoza

Stuart C B Abercrombie

Chase Salsbury

Po-Chao Huang

Hanna Revinskaya

Brian Choung Choi

Harrison Lee McKenzie Chapter

An example method includes receiving, from a light detector, information indicative of a light intensity of a field of view of an optical system. The optical system includes one or more optical components and a light detector configured to receive light from a field of view of an environment of the optical system by way of the one or more optical components. The optical system also includes an occlusion-detection camera configured to capture images of the one or more optical components. The example method also includes adjusting, based on the received information, at least one operating parameter of the occlusion-detection camera. The example method also includes causing the occlusion-detection camera to capture at least one image of the one or more optical components according to the at least one adjusted operating parameter.

Auto-exposure occlusion camera

One example method involves obtaining a plurality of scans of a field-of-view (FOV) of a light detection and ranging (LIDAR) device disposed inside a housing. Obtaining each scan of the plurality of scans comprises: transmitting, through a plurality of sections of the housing, a plurality of light pulses emitted from the LIDAR device in different directions toward the housing; and detecting a plurality of returning light pulses comprising reflected portions of the transmitted plurality of light pulses that are reflected back toward the LIDAR device. The method also involves detecting an obstruction that at least partially occludes the LIDAR device from scanning the FOV through the housing based on the plurality of scans.

Methods and systems for detecting obstructions on a sensor housing

Example embodiments relate to identification of proxy calibration targets for a fleet of sensors. An example method includes collecting, using a sensor coupled to a vehicle, data about one or more objects within an environment of the vehicle. The sensor has been calibrated using a ground-truth calibration target. The method also includes identifying, based on the collected data, at least one candidate object, from among the one or more objects, to be used as a proxy calibration target for other sensors coupled to vehicles within a fleet of vehicles. Further, the method includes providing, by the vehicle, data about the candidate object for use by one or more vehicles within the fleet of vehicles.

Identification of proxy calibration targets for a fleet of vehicles

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using naturally collected data in sensor calibration. One of the methods includes obtaining, using a first, calibrated sensor, a first plurality of raw sensor measurements of an environment; determining, from the first plurality of raw sensor measurements, a Truncated Signed Distance Field (TSDF)-based model of surfaces in the environment; obtaining, using a second sensor, a second plurality of raw sensor measurements of the environment; determining a multi-dimensional point cloud representation of the environment; and determining refined values of the set of calibration parameters of the second sensor based on a difference between, for each data point, (i) the multiple values that define the data point and (ii) multiple values that define a target data point derived from the TSDF-based model of surfaces in the environment.

In-situ sensor calibration for autonomous vehicles

In one example, a method is provided that includes receiving lidar data obtained by a lidar device. The lidar data includes a plurality of data points indicative of locations of reflections from an environment of the vehicle. The method includes receiving images of portions of the environment captured by a camera at different times. The method also includes determining locations in the images that correspond to a data point of the plurality of data points. Additionally, the method includes determining feature descriptors for the locations of the images and comparing the feature descriptors to determine that sensor data associated with at least one of the lidar device, the camera, or a pose sensor is accurate or inaccurate.

Methods and apparatus for validating sensor data

Example embodiments relate to methods for localizing light detection and ranging (lidar) calibration targets. An example method includes generating a point cloud of a region based on data from a light detection and ranging (lidar) device. The point cloud may include points representing at least a portion of a calibration target. The method also includes determining a presumed location of the calibration target. Further, the method includes identifying, within the point cloud, a location of a first edge of the calibration target. In addition, the method includes performing a comparison between the identified location of the first edge of the calibration target and a hypothetical location of the first edge of the calibration target within the point cloud if the calibration target were positioned at the presumed location. Still further, the method includes revising the presumed location of the calibration target based on at least the comparison.

Methods for localizing light detection and ranging (lidar) calibration targets

Methods, systems, and apparatus for generation and use of surfel maps to plan for occlusions. One of the methods includes receiving a previously-generated surfel map depicting an area in which a vehicle is located, the surfel map comprising a plurality of surfels, each surfel corresponding to a respective different location in the area in which a vehicle is located; receiving, from one or more sensors, sensor data representing the area in which the vehicle is located; determining, based on the sensor data, that the area in which a vehicle is located includes a dynamic object having a changed shape relative to its representation in the surfel map; and generating an updated path for the vehicle to travel that avoids an occlusion by the changed shape of the dynamic object of a line of sight of one or more sensors to an area of interest.

Planning for occlusions with surfel maps

Example embodiments relate to calibration and localization of a light detection and ranging (lidar) device using a previously calibrated and localized lidar device. An example embodiment includes a method. The method includes receiving, by a computing device associated with a second vehicle, a first point cloud captured by a first lidar device of a first vehicle. The first point cloud includes points representing the second vehicle. The method also includes receiving, by the computing device, pose information indicative of a pose of the first vehicle. In addition, the method includes capturing, using a second lidar device of the second vehicle, a second point cloud. Further, the method includes receiving, by the computing device, a third point cloud representing the first vehicle. Yet further, the method includes calibrating and localizing, by the computing device, the second lidar device.

Calibration and localization of a light detection and ranging (lidar) device using a previously calibrated and localized lidar device

Aspects and implementations of the present disclosure address shortcomings of the existing technology by enabling efficient object identification and tracking in autonomous vehicle (AV) applications by using velocity data-assisted mapping of first set of points obtained for a first sensing data frame by a sensing system of the AV to a second set of points obtained for a second sensing data frame by the sensing system of the AV, the first set of points and the second set of points corresponding to an object in an environment of the AV, and causing a driving path of the AV to be determined in view of the performed mapping.

Doppler-assisted object mapping for autonomous vehicle applications

The technology relates to localizing a vehicle. As one approach, a first LIDAR sensor scan data of an environment of the vehicle and localization data for the vehicle at a location where the first LIDAR sensor scan was captured are stored. Thereafter, the computing device is suspended and subsequently unsuspended. After the computing device is unsuspended, second LIDAR sensor scan data of the vehicle's environment is received. The first LIDAR sensor scan data is compared to the second LIDAR sensor scan data to determine whether the vehicle has moved. Based on the determination of whether the vehicle has moved from the location, the stored localization data is used to localize the vehicle. Other approaches are also described.

Localization initialization for autonomous vehicles

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for using a surfel map to generate long range localization. One of the methods includes obtaining, for a particular location of a vehicle having a camera and a detection sensor, surfel data including a plurality of surfels. Each surfel in the surfel data has a respective location and corresponds to a different respective detected surface in an environment. Image data captured by the camera is obtained. It is determined that a region of interest for detecting objects for a vehicle planning process is outside a detectable region for the detection sensor. In response, it is determined that the image data for the region of interest matches surfel color data for the surfels corresponding to the region of interest. In response, the vehicle planning process is performed with the region of interest designated as having no unexpected objects.

Long range localization with surfel maps

Aspects and implementations of the present disclosure address shortcomings of existing technology by enabling autonomous vehicle simulations based on retro-reflection optical data. The subject matter of this specification can be implemented in, among other things, a method that involves initiating a simulation of an environment of an autonomous driving vehicle, the simulation including a plurality of simulated objects, each having an identification of a material type of the respective object. The method can further involve accessing simulated reflection data based on the plurality of simulated objects and retro-reflectivity data for the material types of the simulated objects, and determining, using an autonomous vehicle control system for the autonomous vehicle, a driving path relative to the simulated objects, the driving path based on the simulated reflection data.

High fidelity simulations for autonomous vehicles based on retro-reflection metrology

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for the generation and use of a surfel map with semantic labels. One of the methods includes receiving a surfel map that includes a plurality of surfels, wherein each surfel has associated data that includes one or more semantic labels; obtaining sensor data for one or more locations in the environment, the sensor data having been captured by one or more sensors of a first vehicle; determining one or more surfels corresponding to the one or more locations of the obtained sensor data; identifying one or more semantic labels for the one or more surfels corresponding to the one or more locations of the obtained sensor data; and performing, for each surfel corresponding to the one or more locations of the obtained sensor data, a label-specific detection process for the surfel.

Annotated surfel maps

Growing community of inventors

Mountain View, CA, United States of America

Colin Andrew Braley