Zoox, Inc.

Shuangting Liu

Francesco Papi

Qian Song

Zachary Sun

John Bryan Carter

Michael Carsten Bosse

Po-Jen Lai

Nicholas George Dilip Roy

Cong Ding

Murat Gevrekci

Systems and techniques for determining more accurate object parameter values, such as orientation values (e.g., yaw, location, position, etc.), for objects detected in an environment are disclosed. A vehicle computing system may receive sensor data from multiple sensor systems that indicates a parameter value determined at the individual sensor systems. The vehicle computing system may determine values and probabilities for particular parameters modes based on the sensor data parameter values and may further filter these values using a mixture model to determine probability distributions for the modes and associated values. These filtered values and modes may then be used to determine predicted object trajectories that can be used to control a vehicle.

Determining object orientation based on parameter modes

Techniques for performing object detection in a vehicle environment using sensor data captured by one or more sensors of the vehicle are described herein. In some cases, an object in a vehicle environment can be detected based on at least one of (i) a first similarity matrix that represents a first similarity value for two sensor observations associated with the vehicle environment, or (ii) a second similarity matrix that represents a second similarity value for a sensor observation and a track associated with the vehicle environment.

Object detection using similarity determinations for sensor observations

Techniques for improving training data used to train a model to detect objects in an environment are disclosed. Portions of a dataset that are associated with significant operational conditions may be associated with indications of such conditions. The properties of these portions may be used to identify other portions of the dataset or other datasets for augmentation. Data in these portions of a dataset representing detections in an environment may be augmented generally, for example, by flipping geometric values and modifying associated data accordingly. Specific objects represented in such portions may also be augmented by determining particular objects that are represented in selected frames and channels and modifying the pixel values for pixels associated with such objects. The pixel values may be modified or zeroed based on various criteria. The resulting augmented data may be used to train a machine-learned model to more accurately detect objects in an environment.

Object detection using augmented data

Techniques for training a model for detecting objects in an environment are discussed herein. For example, techniques can include determining losses associated with spatial features of candidate bounding boxes output by a machine-learned (ML) model and utilizing the losses to train the ML model. Techniques may include determining candidate bounding box(es) associated with an object detected in an environment using the ML model and receiving a ground truth bounding box associated with the detected object. A yaw error loss may be determined by comparing yaw features of the candidate bounding box to the ground truth bounding box. The candidate bounding box may be axis aligned with respect to the ground truth bounding box and an intersection over union (IoU) loss may be determined based on an IoU between the axis aligned candidate bounding box and the ground truth bounding box. The ML model may be trained based on the losses.

Growing community of inventors

Foster City, CA, United States of America

Shuangting Liu