Zoox, Inc.

Brice Rebsamen

Derek Adams

Michael Carsten Bosse

Nitesh Shroff

Elena Stephanie Stumm

Jesse Sol Levinson

Ashutosh Gajanan Rege

Francesco Papi

Patrick Blaes

Nathaniel Jon Kaiser

Guillermo Duenas Arana

Dragomir Dimitrov Anguelov

Subhasis Das

Zeng Wang

David Burdick Berman

Chengzhi Winston Liu

Gregory D Kravit

Anne-Claire Elisabeth Marie Le Hénaff

Techniques for a pose component that may determine a pose are described herein. A pose may refer to the inertial pose or position of a vehicle which may be updated in real-time or near real-time. For example, the techniques may include receiving a plurality of input signals at a pose component and monitoring the plurality of input signals. The pose component may determine, based on the monitoring of the plurality of input signals, a particular pose update algorithm of a plurality of pose update algorithms for determining the pose and determine, using the particular pose update algorithm, the pose based the plurality of input signals and IMU measurements associated with a primary IMU.

Pose component

Determining a localization performance metric using sensor data is described. A vehicle may capture sensor data from one or more sensors of the vehicle. The sensor data may be input into a localization component to determine localization data (e.g., a location/orientation of the vehicle) and auxiliary location data (e.g., residual data indicating a difference between the measured location/orientation and an actual or estimated location/orientation). The auxiliary localization data (which, in some examples, may include residuals associated with various sensor modalities) may be input into a machine learned model. The machine learned model may generate the localization performance metric. The vehicle may determine an action to take based on the localization performance metric. In some examples, the action may be tied to the performance meeting or exceeding a threshold.

Localization performance metric

Techniques for using a set of variables to estimate a vehicle velocity of a vehicle are discussed herein. A system may determine an estimated velocity of the vehicle using a minimization based on an initial estimated velocity, steering angle data and wheel speed data. The system may then control an operation of the vehicle based at least in part on the estimated velocity.

Estimating vehicle velocity

Techniques are described for using variables associated with vehicle wheels (e.g., linear velocity at a wheel and orientation of the wheel) to estimate velocity of a vehicle during a turn maneuver. In examples of the disclosure, in association with one or more wheels, a wheel orientation during the maneuver and a linear speed during the maneuver may be determined, and well as a yaw rate (e.g., from an inertial measurement unit, gyroscope, etc.) of the vehicle. Examples of the present disclosure include, based on the variables associated with the wheel(s) and the yaw rate associated with the turn maneuver, estimating a vehicle velocity, which may be used by various downstream components, such as to determine or update a pose of a vehicle as part of a localization operation.

Estimating vehicle velocity based on variables associated with wheels

Techniques for using a set of non-steering variables to estimate an angle of a wheel are described. For example, a yaw rate, a linear velocity of a wheel, and vehicle dimensions (e.g., offset between the wheel and a turn-center reference line), can be used to estimate the angle of the wheel. Among other things, estimating angles based on non-steering variables may provide redundancy (e.g., when determined in parallel with steering-based command angles or other commanded angles) and/or may be used to validate commanded angles based on steering components.

Estimating angle of a vehicle wheel based on non-steering variables

Techniques for determining distortion in a map caused by measurement errors are discussed herein. For example, such techniques may include implementing a model to estimate map distortion between the map frame and the inertial frame. Data such as sensor data, map data, and vehicle state data may be input into the model. A map distortion value output from the model may be used to compensate vehicle operations in a local region by approximating the distortion as linearly varying about the region. A vehicle, such as an autonomous vehicle, can be controlled to traverse an environment based on the trajectory.

Map distortion determination

Techniques for representing a scene or map based on statistical data of captured environmental data are discussed herein. In some cases, the data (such as covariance data, mean data, or the like) may be stored as a multi-resolution voxel space that includes a plurality of semantic layers. In some instances, individual semantic layers may include multiple voxel grids having differing resolutions. Multiple multi-resolution voxel spaces may be merged to generate combined scenes based on detected voxel covariances at one or more resolutions.

Using maps comprising covariances in multi-resolution voxels

Techniques are discussed for using multi-resolution maps, for example, for localizing a vehicle. Map data of an environment can be represented by discrete map tiles. In some cases, a set of map tiles can be precomputed as contributing to localizing the vehicle in the environment, and accordingly, the set of map tiles can be loaded into memory when the vehicle is at a particular location in the environment. Further, a level of detail represented by the map tiles can be based at least in part on a distance between a location associated with the vehicle and a location associated with a respective region in the environment. The level of detail can also be based on a speed of the vehicle in the environment. The vehicle can determine its location in the environment based on the map tiles and/or the vehicle can generate a trajectory based on the map tiles.

Multi-resolution maps for localization

Localization error handling using output is described. A computing system associated with a vehicle can receive sensor data from a sensor associated with vehicle. The computing system can determine, based at least partly on the sensor data, a first instruction for controlling the vehicle during a first period of time and a difference in pose information associated with a pose of the vehicle. Based at least partly on determining the difference, the computing system can retrieve a second instruction for controlling the vehicle during a second period of time prior to the first period of time and, based at least partly on comparing the first instruction and the second instruction, the computing system can determine whether the vehicle is to follow the first instruction or perform an alternate operation.

Localization error handling

Maps comprising covariances in multi-resolution voxels

Localization error monitoring using vehicle state information is described. A computing system associated with a vehicle may determine a current state of the vehicle, such as a location, position, orientation, velocity, acceleration, or the like. The vehicle computing system may determine one or more metrics associated with the state of the vehicle. The metrics may include a variance associated with the state, a residual associated with a measurement corresponding to the state, or a correction factor applied to correct an input error. The computing system may determine whether the metric exceeds a threshold value. Based on a threshold exceedance, the computing system may determine one or more errors associated with a state of the vehicle. The vehicle computing system may control the vehicle based on the one or more errors.

Localization error monitoring

Techniques for decimating portions of a map of an environment are discussed herein. The environment can be represented by a three-dimensional (3D) map including a plurality of polygons and semantic information associated with the polygons. In some cases, decimation operations may be based on semantic information associated with the environment. Differing decimation operations and/or levels may be applied to polygons of different semantic classifications or differing contribution levels. Boundaries between regions having different semantic information can be preserved. Meshes can be decimated using different decimation operators or decimation levels and an accuracy of localizing can be compared using the various decimated meshes. An optimal mesh can be selected and sent to vehicles for localizing the vehicles in the environment.

Mesh decimation based on semantic information

A system may receive a sensor dataset representing an environment and use the dataset to create or update a map. In creating or updating the map, the system may determine an object classification of one or more detected objects and only selectively incorporate data into the map based at least in part on the classification. The map may be associated with the classification (or semantic) information of the objects, as well as weights based on the classification. Similarly, datasets with selected classes of data removed may be used for system localization. Further, the system may determine an object track of the objects. When updating the map, voxels in a voxel space may indicate an occupied voxel based on a threshold number of observances. The object track and clean map can then be used for controlling an autonomous vehicle.

Growing community of inventors

San Jose, CA, United States of America

Brice Rebsamen