Waymo LLC

Jiyang Gao

Congcong Li

Dragomir Dimitrov Anguelov

Hang Zhao

Yi Shen

Chen Sun

Zijian Guo

Pei Sun

Benjamin Sapp

Yue Shen

Yu Ouyang

Yu Zhang

Vijay Vasudevan

Junhua Mao

Jiquan Ngiam

Yin Zhou

Yuning Chai

Cordelia Luise Schmid

Tian Lan

Alexander Patrick McCauley

Alper Ayvaci

Ruichi Yu

Xu Chen

Kang Li

Henrik Kretzschmar

Xin Huang

Ruizhongtai Qi

Pengchong Jin

Dillon Cower

Zhishuai Zhang

Martin Nikoltchev

Guan Sun

Yukai Liu

Kyriacos Christoforos Shiarlis

Longlong Jing

Xiaowei Li

The described aspects and implementations enable online lane estimation and tracking in autonomous vehicle and driver assistance applications. In one implementation, disclosed is a method and a system to perform the method that includes receiving perception data indicative of driving lanes and reference objects in a driving environment of a vehicle, identifying target destinations corresponding to the driving lanes, determining candidate driving paths connecting a starting location with one of the target destinations, selecting a target driving path from the candidate driving paths based on geometric properties of each candidate driving path and a spatial relationship of a respective candidate driving path to the reference objects, and causing a motion of the vehicle to be modified based on the target driving path.

Online lane estimation and tracking in autonomous vehicle and driving assistance applications

A system obtains scene context data characterizing the environment. The scene context data includes data that characterizes a trajectory of an agent in a vicinity of a vehicle up to a current time point. The system identifies a plurality of initial target locations, and generates, for each of a plurality of target locations that each corresponds to one of the initial target locations, a respective predicted likelihood score that represents a likelihood that the target location will be an intended final location for a future trajectory of the agent. For each target location in a first subset of the target locations, the system generates a predicted future trajectory for the agent given that the target location is the intended final location for the future trajectory. The system further selects, as likely future trajectories of the agent, one or more of the predicted future trajectories.

Agent trajectory prediction using target locations

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting three-dimensional object locations from images. One of the methods includes obtaining a sequence of images that comprises, at each of a plurality of time steps, a respective image that was captured by a camera at the time step; generating, for each image in the sequence, respective pseudo-lidar features of a respective pseudo-lidar representation of a region in the image that has been determined to depict a first object; generating, for a particular image at a particular time step in the sequence, image patch features of the region in the particular image that has been determined to depict the first object; and generating, from the respective pseudo-lidar features and the image patch features, a prediction that characterizes a location of the first object in a three-dimensional coordinate system at the particular time step in the sequence.

Three-dimensional location prediction from images

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using vectorized inputs.

Agent trajectory prediction using vectorized inputs

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for searching an autonomous vehicle sensor data repository. One of the methods includes maintaining a collection of sensor samples and one or more embeddings of each sensor sample. Each sensor sample is generated from sensor data at multiple time steps and characterizes an environment at each of the multiple time steps. Each embedding corresponds to a respective portion of the sensor sample and has been generated by an embedding neural network. A query specifying a query portion of a query sensor sample is received. A query embedding corresponding to the query portion of the query sensor sample is generated through the embedding neural network. A plurality of relevant sensor samples that have embeddings that are closest to the query embedding are identified as characterizing similar scenarios to the query portion of the query sensor sample.

Searching an autonomous vehicle sensor data repository based on context embedding

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using temporal-spatial interaction predictions.

Multi-modal multi-agent trajectory prediction

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using dynamic voxelization. When deployed within an on-board system of a vehicle, processing the point cloud data using dynamic voxelization can be used to make autonomous driving decisions for the vehicle with enhanced accuracy, for example by combining representations of point cloud data characterizing a scene from multiple views of the scene.

Processing point clouds using dynamic voxelization

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a spatio-temporal embedding of a sequence of point clouds. One of the methods includes obtaining a temporal sequence comprising a respective point cloud input corresponding to each of a plurality of time points, each point cloud input comprising point cloud data generated from sensor data captured by one or more sensors of a vehicle at the respective time point; processing each point cloud input using a first neural network to generate a respective spatial embedding that characterizes the point cloud input; and processing the spatial embeddings of the point cloud inputs using a second neural network to generate a spatio-temporal embedding that characterizes the point cloud inputs in the temporal sequence.

Spatio-temporal embeddings

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing point cloud data using spatio-temporal-interactive networks.

Spatio-temporal-interactive networks

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating vehicle intent predictions using a neural network. One of the methods includes obtaining an input characterizing one or more vehicles in an environment; generating, from the input, features of each of the vehicles; and for each of the vehicles: processing the features of the vehicle using each of a plurality of intent-specific neural networks, wherein each of the intent-specific neural networks corresponds to a respective intent from a set of intents, and wherein each intent-specific neural network is configured to process the features of the vehicle to generate an output for the corresponding intent.

Vehicle intent prediction neural network

Growing community of inventors

San Jose, CA, United States of America

Jiyang Gao