Zoox, Inc.

Patrick Blaes

Derek Adams

Michael Carsten Bosse

Daniel Miller

Jesse Sol Levinson

Elena Stephanie Stumm

Aleksandrs Ecins

Brice Rebsamen

Badeea Ferdaous Alferdaous Alazem

Jifei Qian

Hirotatsu Armstrong

Jeffrey Eric Tolliver

Techniques for integrating sensor data into a scene or map based on statistical data of captured environmental data are discussed herein. The data may be stored as a multi-resolution voxel space and the techniques may comprise first applying a pre-alignment or localization technique prior to fully integrating the sensor data.

System for aligning sensor data with maps comprising covariances

Techniques for estimating a ground plane based on lidar data and/or attributes of the ground plane are discussed herein. A vehicle captures radar data, e.g., 4D radar data including height information, as it traverses an environment. The radar data can include direct returns from an object and reflected or multipath returns, e.g., that reflect off a ground surface and the object. A position of the ground plane can be estimated based at least in part on a distance between direct returns and the reflected returns. Attributes of the ground plane may be determined from differences between the direct returns and the reflected returns.

Radar-based ground plane characteristic determination

Techniques for representing a scene or map based on statistical data of captured environmental data are discussed herein. In some cases, the data (such as covariance data, mean data, or the like) may be stored as a multi-resolution voxel space that includes a plurality of semantic layers. In some instances, individual semantic layers may include multiple voxel grids having differing resolutions. Multiple multi-resolution voxel spaces may be merged to generate combined scenes based on detected voxel covariances at one or more resolutions.

Using maps comprising covariances in multi-resolution voxels

Techniques are discussed for modifying map elements associated with map data. Map data can include three-dimensional data (e.g., LIDAR data) representing an environment, while map elements can be associated with the map data to identify locations and semantic information associated with an environment, such as regions that correspond to driving lanes or crosswalks. A trajectory associated with the map data can be updated, such as when aligning one or more trajectories in response to a loop closure, updated calibration, etc. The transformation between a trajectory and an updated trajectory can be applied to map elements to warp the map elements so that they correspond to the updated map data, thereby providing automatic and accurate techniques for updating map elements associated with map data.

Modifying map elements associated with map data

Techniques for representing a scene or map based on statistical data of captured environmental data are discussed herein. In some cases, the data (such as covariance data, mean data, or the like) may be stored as a multi-resolution voxel space that includes a plurality of semantic layers. In some instances, individual semantic layers may include multiple voxel grids having differing resolutions. Multiple multi-resolution voxel spaces may be merged or aligned to generate combined scenes based on detected voxel covariances at one or more resolutions.

System and method for generating multi-resolution voxel spaces

Techniques for updating data operations in a perception system are discussed herein. A vehicle may use a perception system to capture data about an environment proximate to the vehicle. The perception system may output the data about the environment to a system configured to determine positions of objects relative to the perception system over time. The positions of the objects may be used to estimate an object velocity and may be compared against machine learning model outputs in a self-supervised manner to train the machine learning model to output object velocities based on inputs from the perception system. The output of the machine learning model may include a two-dimensional velocity for objects in the environment. The two-dimensional velocity may be used for a vehicle system such that the vehicle can make environmentally aware operational decisions, which may improve reaction time(s) and/or safety outcomes of the vehicle.

Self-supervised global velocity determination for perception system

Techniques for detecting and modifying radar data anomalies using multistage clustering are described herein. In some examples, radar data including a set of radar points may be captured by a vehicle operating within an environment. The vehicle may determine a radar point subset by clustering the radar data based on the range and doppler values of the radar points. The vehicle may analyze the azimuth values associated with the radar point subset, including evaluating the azimuth values of the radar point subset relative to one or more azimuth error ranges determined based on the radar antenna configuration and/or multipath detections. When detecting anomalous and/or invalid radar points having azimuth values within an azimuth error range, the vehicle may modify the subset of radar points to mitigate the anomalous or invalid radar points.

Radar anomaly detection based on multistage clustering

This disclosure is directed to calibrating sensors for an autonomous vehicle. First sensor data and second sensor data can be captured by one or more sensors representing an environment. The first sensor data and the second sensor data can be associated with a grid in a plurality of combinations to generate a plurality of projected data. A number of data points of the projected data occupying a cell of the grid can be summed to determine a spatial histogram. An amount of error (such as an entropy value) can be determined for each of the projected data, and the projected data corresponding to the lowest entropy value can be selected as representing a calibrated configuration of the one or more sensors. Calibration data associated with the lowest entropy value can be determined and used to calibrate the one or more sensors, respectively.

Sensor calibration

Maps comprising covariances in multi-resolution voxels

A vehicle may include a localization and/or mapping component to understand what surrounds the autonomous vehicle and where it is in relation to the surroundings. The localization and/or mapping component may receive sensor data from sensor(s) of the vehicle and generate a map and/or position and/or orientation from the sensor data according to parameters that configure the way the localization and/or mapping component generates the map and/or position/orientation. A computing device may dynamically adjust these parameters, thereby changing the way the map and/or position/orientation are generated. This adjustment may be based on a condition detected from the sensor data and may increase a clarity (e.g., degree of distinctness/clarity) of the generated map and/or position/orientation.

Growing community of inventors

San Dimas, CA, United States of America

Patrick Blaes