Zoox, Inc.

Chuang Wang

Joshua Kriser Cohen

Cooper Stokes Sloan

James William Vaisey Philbin

Subhasis Das

Noureldin Ehab Hendy

Feng Tian

Yi Xu

Jacob Daniel Boydston

Subasingha Shaminda Subasingha

Sarah Tariq

Badeea Ferdaous Alferdaous Alazem

Shiwei Sheng

Yu Liu

Yassen Ivanchev Dobrev

Anton Mario Bongio Karrman

Yuesong Xie

Venkata Subrahmanyam Chandra Sekhar Chebiyyam

Marin Kobilarov

Joseph Funke

Michael Carsten Bosse

Kai Zhenyu Wang

Gowtham Garimella

Andrew Scott Crego

Josh Alexander Jimenez

Samantha Marie Ting

Sabeek Mani Pradhan

Jifei Qian

Nicholas George Charchut

Pengfei Duan

Ralph Michael Kling

William Anthony Silva

James Graham Dolan

Francesco Papi

Andreas Christian Reschka

Robert Edward Somers

Andrew Lewis King

Tod Cameron Semple

Leonardo Poubel Orenstein

Li Yon Tan

Giacomo Zavolta Taylor

William Harland Montgomery, Iv

Dilip Bethanabhotla

Yuan Zhang

Hang Ren

Matthew Lotocki

Takahiro Baba

A vehicle safety system of an autonomous vehicle may determine predicted velocity vectors for a potential collision, and use the velocity vectors to determine a trajectory for the autonomous vehicle to traverse the environment. The vehicle safety system may analyze sensor data to determine a likelihood of a potential collision with a dynamic object in the environment. Predicted velocity vectors may be determined for the autonomous vehicle and the dynamic object at a time and/or location associated with the potential collision. The predicted velocity vectors may be used to determine a point of impact, relative angle, and/or relative velocity between the autonomous vehicle and dynamic object at the potential collision. The likelihood of the potential collision and the predicted velocity vectors may be used to determine a trajectory for the autonomous vehicle, which may include following a current trajectory or transitioning to one or more contingent trajectories.

Collision avoidance and mitigation in autonomous vehicles using predicted trajectories

Techniques for updating data operations in a perception system are discussed herein. A vehicle may use a perception system to capture data about an environment proximate to the vehicle. The perception system may receive image data, lidar data, and/or radar data to determine information about an object in the environment. As different sensors may be associated with different time periods for capturing and/or processing operations, the techniques include updating object data with data from sensors associated with a shorter time period to generate intermediate object data. Such intermediate object data may reduce a delay in updating a position of an object in an environment.

Out of sequence perception system

Techniques for clustering sensor data are discussed herein. Sensors of a vehicle may detect data points in an environment. Clustering techniques can be used in a vehicle safety system to determine connection information between the data points. Further, the machine learned model can be trained to output motion data representing velocity data or a cluster association score. The connection information and/or motion data can be used by a vehicle computing device to detect objects in an environment and/or to control operation of a vehicle.

Perception system velocity determination

This disclosure describes techniques for using radar cross-section (RCS) data to classify objects detected by autonomous vehicles within driving environments. In some examples, the variance of the RCS data associated with an object may be evaluated to determine signal interference caused by multipath fading. The variance of the RCS data may be used to classify the object and to determine whether the autonomous vehicle can safely drive over the object. For instance, objects such as manhole covers, storm drains, and expansion joints may provide a significant radar signal, but low RCS variance indicating that they can be driven over by the vehicle. Based on the classification of the object, the autonomous vehicle may determine a trajectory around the object or directly over the object.

Radar object classification based on radar cross-section data

Techniques for verifying a reliability of map data are discussed herein. In some examples, map data can be used by a vehicle, such as an autonomous vehicle, to traverse an environment. Sensor data (e.g., image data, lidar data, etc.) can be received from a sensor associated with a vehicle and may be used to generate an estimated map and confidence values associated with the estimated map. When the sensor data is image data, images data from multiple perspectives or different time instances may be combined to generate the estimated map. The estimated map may be compared to a stored map or to a proposed vehicle trajectory or corridor to determine a reliability of the stored map data.

Verifying reliability of data used for autonomous driving

Techniques are discussed herein for analyzing radar data to determine that radar noise from one or more target detections potentially conceals additional objects near the target detection. Determining whether an object may be concealed can be based at least in part on a radar noise level based on a target detection, as well as distributions of radar cross sections and/or doppler data associated with particular object types. For a location near a target detection, a radar system may determine estimated noise levels, and compare the estimated noise levels to radar cross section probabilities associated with object types to determine the likelihood that an object of the object type could be concealed at the location. Based on the analysis, the system may determine a vehicle trajectory or otherwise may control a vehicle based on the likelihood that an object may be concealed at the location.

Radar data analysis and concealed object detection

Techniques relating to monitoring map consistency are described. In an example, a monitoring component associated with a vehicle can receive sensor data associated with an environment in which the vehicle is positioned. The monitoring component can generate, based at least in part on the sensor data, an estimated map of the environment, wherein the estimated map is encoded with policy information for driving within the environment. The monitoring component can then compare first information associated with a stored map of the environment with second information associated with the estimated map to determine whether the estimated map and the stored map are consistent. Component(s) associated with the vehicle can then control the object based at least in part on results of the comparing.

Map consistency checker

Techniques for clustering sensor data are discussed herein. Sensors of a vehicle may detect data points in an environment. Clustering techniques can be used in a vehicle safety system to determine connection information between the data points. The connection information can be used by a vehicle computing device that employs clustering and/or segmenting techniques to detect objects in an environment and/or to control operation of a vehicle.

Determining inputs for perception system

A collision avoidance system may validate, reject, or replace a trajectory generated to control a vehicle. The collision avoidance system may comprise a secondary perception component that may receive sensor data, receive and/or determine a corridor associated with operation of a vehicle, classify a portion of the sensor data associated with the corridor as either ground or an object, determine a position and/or velocity of at least the nearest object, determine a threshold distance associated with the vehicle, and control the vehicle based at least in part on the position and/or velocity of the nearest object and the threshold distance.

Collision avoidance perception system

Techniques are discussed for determining reflected returns in radar sensor data. In some instances, pairs of radar returns may be compared to one another. For example, a reflection point may be determined from a first position of a first radar return and a second position of a second radar return. Additional data, e.g., sensor data and/or map data, may be used to determine the presence of objects in the environment. The first return or the second return may be a reflected return if an object is disposed at the reflection point. In some instances, a vehicle, such as an autonomous vehicle, may be controlled at the exclusion of information from reflected returns.

Recognizing radar reflections using position information

Model-based control of dynamical systems typically requires accurate domain-specific knowledge and specifications system components. Generally, steering actuator dynamics can be difficult to model due to, for example, an integrated power steering control module, proprietary black box controls, etc. Further, it is difficult to capture the complex interplay of non-linear interactions, such as power steering, tire forces, etc. with sufficient accuracy. To overcome this limitation, a recurring neural network can be employed to model the steering dynamics of an autonomous vehicle. The resulting model can be used to generate feedforward steering commands for embedded control. Such a neural network model can be automatically generated with less domain-specific knowledge, can predict steering dynamics more accurately, and perform comparably to a high-fidelity first principle model when used for controlling the steering system of a self-driving vehicle.

Steering control for vehicles

Driving simulations may be generated based on driving log data and used to validate autonomous vehicle safety systems. A driving simulation system may receive driving log data from autonomous vehicles and determine events which caused a vehicle to diverge from a planned trajectory. Driving simulations may be generated based on the driving log data, and executed using a simulated vehicle that follows the initial planned trajectory of the autonomous vehicle. The driving simulations may be analyzed to detect potential collisions and near misses, and to determine success conditions for the behaviors of vehicle safety systems. Driving simulations and associated success conditions may be aggregated and used in validation suites for vehicle controllers and/or vehicle safety systems, thereby providing more comprehensive and robust autonomous vehicle validation based on real-world driving scenarios.

Autonomous vehicle safety system validation

Techniques for receiving and processing sensor data captured by a fleet of vehicle are discussed herein. In some examples, a fleet dashcam system can receive sensor data captured by electronic devices on a fleet of vehicles and can use that data to detect collision and near-collision events. The data of the collision or near-collision event can be used to determine a simulation scenario and a response of an autonomous vehicle control to the simulation scenario and/or it can be used to create a collision heat map to aid in operation of an autonomous vehicle.

Fleet dashcam system for autonomous vehicle operation

Techniques for analyzing driving scenarios are discussed herein. For example, techniques may include determining a level of exposure associated with scenarios, searching for similar scenarios, and generating new additional scenarios. A driving scenario may be represented as top-down multi-channel data. The top-down multi-channel data may be provided as input to a neural network trained to output a prediction of future events. A multi-dimensional vector representing the scenario can be received as an intermediate output from the neural network and may be stored to represent the scenario. Multi-dimensional vectors representing different scenarios may be stored in a multi-dimensional space, and similar scenarios may be identified by proximity searching of the multi-dimensional space.

Top-down scenario exposure modeling

Techniques for accurately predicting and avoiding collisions with objects detected in an environment of a vehicle are discussed herein. A vehicle safety system can implement a model to output data indicating an intersection probability between the object and a portion of the vehicle in the future. The model may employ a rear collision filter, a distance filter, and a time to stop filter to determine whether a predicted collision may be a false positive, in which case the techniques may include refraining from reporting such predicted collision to other another vehicle computing device to control the vehicle.

Velocity regression safety system

Growing community of inventors

Sunnyvale, CA, United States of America

Chuang Wang