Zoox, Inc.

John Bryan Carter

Francesco Papi

Qian Song

Zachary Sun

Shuangting Liu

Michael Carsten Bosse

Nicholas George Dilip Roy

Yunming Shao

Cong Ding

Murat Gevrekci

A modified Kalman filter may include one or more neural networks to augment or replace components of the Kalman filter in such a way that the human interpretability of the filter's inner functions is preserved. The neural networks may include a neural network to account for bias in measurement data, a neural network to account for unknown controls in predicting a state of an object, a neural network ensemble that is trained differently based on different sensor data, a neural network for determining the Kalman gain, and/or a set of Kalman filters including various neural networks that determine independent estimated states, which may be fused using Bayesian fusion to determine a final estimated state.

Interpretable Kalman filter comprising neural network component(s) for autonomous vehicles

Systems and techniques for determining more accurate object parameter values, such as orientation values (e.g., yaw, location, position, etc.), for objects detected in an environment are disclosed. A vehicle computing system may receive sensor data from multiple sensor systems that indicates a parameter value determined at the individual sensor systems. The vehicle computing system may determine values and probabilities for particular parameters modes based on the sensor data parameter values and may further filter these values using a mixture model to determine probability distributions for the modes and associated values. These filtered values and modes may then be used to determine predicted object trajectories that can be used to control a vehicle.

Determining object orientation based on parameter modes

Object detection and tracking systems may use machine-learned transformer models with self-attention for detecting, classifying, and/or tracking objects in an environment. Techniques described herein may include receiving sensor data generated by different sensor modalities of a vehicle, determining different bounding shapes based on the different sensor modalities, and using a machine-learned transformer model to determine associated and/or combined bounding shapes. The machine-learned transformer model may receive a variable number of input bounding shapes representing any number of objects and various sensor modalities. Multiple stages of the transformer may be used to determine associated bounding shapes and to assign attributes for the associated bounding shapes, based on the individual bounding shapes of the different sensor modalities and/or previous bounding shapes for objects detected and tracked in a previous scene in the environment.

Object detection and tracking using machine learning transformer models with attention

Techniques for performing object detection in a vehicle environment using sensor data captured by one or more sensors of the vehicle are described herein. In some cases, an object in a vehicle environment can be detected based on at least one of (i) a first similarity matrix that represents a first similarity value for two sensor observations associated with the vehicle environment, or (ii) a second similarity matrix that represents a second similarity value for a sensor observation and a track associated with the vehicle environment.

Growing community of inventors

Upton, MA, United States of America

John Bryan Carter