Ghost Autonomy Inc.

Zoox, Inc.

Gm Cruise Holdings LLC

Ghost Locomotion Inc.

Feng Tian

John Martin Hayes

Volkmar Uhlig

Akash J Sagar

Nima Soltani

Chuang Wang

Cooper Stokes Sloan

Sarah Tariq

Yi Xu

Christopher R Lumb

James William Vaisey Philbin

Kai Zhenyu Wang

Geoffrey Louis Chi-Johnston

Laura Athena Freeman

Noureldin Ehab Hendy

Seunghyun Min

Lei Huang

Daniel Tien

Christopher Brian Roland

Pengfei Duan

Diego Plascencia-Vega

Li Yon Tan

System, methods, and computer-readable media for configuring an autonomous vehicle based on safety scores determined by a safety score prediction algorithm, and an associated training technique, to output a safety score for a predicted and/or actual path. The safety score is a value indicating a likelihood of risky events per distance. The safety score prediction algorithm is trained with historical human driving datasets associated with paths (predicted or actual) taken by one or more AVs during human driving.

Baselining autonomous vehicle safety using driving data of human autonomous vehicle operators

Techniques for verifying a reliability of map data are discussed herein. In some examples, map data can be used by a vehicle, such as an autonomous vehicle, to traverse an environment. Sensor data (e.g., image data, lidar data, etc.) can be received from a sensor associated with a vehicle and may be used to generate an estimated map and confidence values associated with the estimated map. When the sensor data is image data, images data from multiple perspectives or different time instances may be combined to generate the estimated map. The estimated map may be compared to a stored map or to a proposed vehicle trajectory or corridor to determine a reliability of the stored map data.

Verifying reliability of data used for autonomous driving

The subject disclosure relates to techniques for filtering perception-related artifacts. The disclosed technology can include receiving, by a machine learning model, a first output generated by a perception system model onboard an autonomous vehicle, wherein the first output is based on sensor data received from sensors of the autonomous vehicle at a first time and includes an inaccurate perception of an environment around the autonomous vehicle, receiving, by the machine learning model, a second output generated by the perception system model onboard the autonomous vehicle, wherein the second output is based on sensor data received from sensors of the autonomous vehicle at a second time after the first time and includes an accurate perception of the environment around the autonomous vehicle, and altering, by the machine learning model, the inaccurate perception of the environment from the first output based on the accurate perception of the environment in the second output.

Filtering perception-related artifacts

The subject disclosure relates to techniques for enabling autonomous vehicles to reason about similarities of features between drivable areas. A process of the disclosed technology can include receiving first sensor data corresponding with a first roadway segment, receiving second sensor data corresponding with a second roadway segment, encoding the first sensor data into a first vector, wherein the first vector represents a first roadway characteristic associated with the first roadway segment, encoding the second sensor data into a second vector, wherein the second vector represents a second roadway characteristic associated with the second roadway segment, and determining a similarity between the first roadway segment and the second roadway segment based on the first vector and the second vector.

Road segment spatial embedding

Techniques relating to monitoring map consistency are described. In an example, a monitoring component associated with a vehicle can receive sensor data associated with an environment in which the vehicle is positioned. The monitoring component can generate, based at least in part on the sensor data, an estimated map of the environment, wherein the estimated map is encoded with policy information for driving within the environment. The monitoring component can then compare first information associated with a stored map of the environment with second information associated with the estimated map to determine whether the estimated map and the stored map are consistent. Component(s) associated with the vehicle can then control the object based at least in part on results of the comparing.

Map consistency checker

The present technology is directed to training and the use of a machine learning model to measure the safety of an autonomous vehicle (AV) driving in simulation. An AV management system can run a simulation of an AV autonomously piloting itself and collect simulation driving data. Further, the AV management system can parse the simulation driving data into kinematic and semantic environmental features and output a simulation safety score of the simulation based on the kinematic and semantic environmental features. The simulation safety score indicates a probability of a safety critical event such as a collision or a near-miss between the AV and the at least one simulated object in the simulation.

Safety measurement of autonomous vehicle driving in simulation

Selective training of neural networks using motion estimation, including: selecting, from a plurality of pixels in video data from a vehicle, based on motion relative to the vehicle, one or more pixels; and training a neural network based on the video data by zeroing out an error function applied to the selected one or more pixels.

Selective training of neural networks using motion estimation

Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle safety system can implement a model to output data indicating an intersection probability between the object and a portion of the vehicle in the future. The model may employ a rear collision filter, a distance filter, and a time to stop filter to determine whether a predicted collision may be a false positive, in which case the techniques may include refraining from reporting such predicted collision to other another vehicle computing device to control the vehicle.

Velocity regression safety system

Determining control operations for an autonomous vehicle may include receiving, by an operational model, an operational model input based on camera data from one or more cameras of an automated vehicle; determining, by the operational model, based on the input, a current environmental state and a predicted environmental state; providing, to a rules module, a differential between the current environmental state and a previously predicted environmental state; and determining, based on the rules module, one or more control operations for the automated vehicle.

Determining control operations for an autonomous vehicle

Camera data normalization for an autonomous vehicle are described herein, including: receiving, from one or more cameras of the autonomous vehicle, camera data; applying a color normalization to the camera data; applying a spherical reprojection to the camera data; and applying, based on a registration point, a stabilization to the camera data.

Camera data normalization for an autonomous vehicle

Detecting out-of-model scenarios for an autonomous vehicle including: determining, based on first sensor data from one or more sensors, an environmental state relative to the autonomous vehicle, wherein operational commands for the autonomous vehicle are based on a selected machine learning model, wherein the selected machine learning model comprises a first machine learning model; comparing the environmental state to a predicted environmental state relative to the autonomous vehicle; and determining, based on a differential between the environmental state and the predicted environmental state, whether to select a second machine learning model as the selected machine learning model.

Detecting out-of-model scenarios for an autonomous vehicle

Scene filtering using motion estimation, including identifying, in camera data from an autonomous vehicle, based on motion relative to the autonomous vehicle, one or more pixels; filtering, from the camera data, the one or more pixels; and training, based on the filtered camera data, a neural network.

Scene filtering using motion estimation

Controlling an automated vehicle using visual anchors, including receiving, from one or more cameras of an autonomous vehicle, first video data; identifying one or more visual anchors in the first video data; determining one or more differentials between the one or more visual anchors and one or more predicted visual anchors; and determining, based on the one or more differentials, one or more control operations for the autonomous vehicle to reduce the one or more differentials.

Growing community of inventors

Foster City, CA, United States of America

Feng Tian