Waymo LLC

Honda Motor Co., Ltd.

Vasiliy Igorevich Karasev

Alper Ayvaci

Daniel Rudolf Maurer

Hang Yan

Bernd Heisele

Yu-Han Chen

Bayram Safa Cicek

Kratarth Goel

Yingying Zhu

Albert Zhao

Justin Yu Zheng

Feiyu Chen

A method includes obtaining, by a processing device, input data derived from a set of sensors associated with an autonomous vehicle (AV). The input data includes camera data and radar data. The method further includes extracting, by the processing device from the input data, a plurality of sets of bird's-eye view (BEV) features. Each set of BEV features corresponds to a respective timestep. The method further includes generating, by the processing device from the plurality of sets of BEV features, an object flow for at least one object. Generating the object flow includes performing at least one of: multi-frame temporal aggregation or multi-frame dense motion estimation. The method further includes causing, by the processing device, a driving path of the AV to be modified in view of the object flow.

Multi-frame temporal aggregation and dense motion estimation for autonomous vehicles

The described aspects and implementations enable efficient detection and classification of objects with machine learning models that deploy a bird's-eye view representation and are trained using depth ground truth data. In one implementation, disclosed are system and techniques that include obtaining images, generating, using a first neural network (NN), feature vectors (FVs) and depth distributions pixels of images, wherein the first NN is trained using training images and a depth ground truth data for the training images. The techniques further include obtaining a feature tensor (FT) in view of the FVs and the depth distributions, and processing the obtained FTs, using a second NN, to identify one or more objects depicted in the images.

Object identification in bird's-eye view reference frame with explicit depth estimation co-training

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network using contrastive learning. One of the methods includes obtaining a network input representing an environment; processing the network input using a first subnetwork of the neural network to generate a respective embedding for each location in the environment; processing the embeddings for each location in the environment using a second subnetwork of the neural network to generate a respective object prediction for each location; determining, for each of a plurality of pairs of the plurality of locations in the environment, whether the respective object predictions of the pair of locations characterize the same possible object or different possible objects; computing a respective contrastive loss value for each of the plurality of pairs of locations; and updating values for a plurality of parameters of the first subnetwork using the computed contrastive loss values.

Contrastive learning for object detection

Systems and techniques for pedestrian path predictions are disclosed herein. For example, an environment, features of the environment, and pedestrians within the environment may be identified. Models for the pedestrians may be generated based on features of the environment. A model may be indicative of goals of a corresponding pedestrian and predicted paths for the corresponding pedestrian. Pedestrian path predictions for the pedestrians may be determined based on corresponding predicted paths. A pedestrian path prediction may be indicative of a probability that the corresponding pedestrian will travel a corresponding predicted path. Pedestrian path predictions may be rendered for the predicted paths, such as using different colors or different display aspects, thereby enabling a driver of a vehicle to be presented with information indicative of where a pedestrian is likely to travel.

Pedestrian path predictions

Growing community of inventors

Los Angeles, CA, United States of America

Vasiliy Igorevich Karasev