Waymo LLC

Benjamin Sapp

Dragomir Dimitrov Anguelov

Yuning Chai

Khaled Refaat

Jiyang Gao

Hang Zhao

Congcong Li

Mayank Bansal

Chen Sun

Reza Mahjourian

Cordelia Luise Schmid

Tian Lan

Yi Shen

Yue Shen

Alexander Patrick McCauley

Kai Ding

Balakrishnan Varadarajan

Wei Chai

Mingxing Tan

Pei Sun

Xin Huang

Benjamin James Caine

Andre Liang Cornman

Jinkyu Kim

Liting Sun

Kratarth Goel

Scott Morgan Ettinger

Ekaterina Igorevna Tolstaya

Avikalp Srivastava

Hee Jin Jeong

Rami Al-Rfou

Nigamaa Nayakanti

Aurick Qikun Zhou

Rebecca Dawn Roelofs

Kyriacos Christoforos Shiarlis

Ahmed Said Mohammed Hefny

Carlton Macdonald Downey

Methods, systems, and apparatus for generating trajectory predictions for one or more target agents. In one aspect, a system comprises one or more computers configured to obtain scene context data characterizing a scene in an environment at a current time point, where the scene includes multiple agents that include a target agent and one or more context agents, and the scene context data includes respective context data for each of multiple different modalities of context data. The one or more computers then generate an encoded representation of the scene in the environment that includes one or more embeddings and process the encoded representation of the scene context data using a decoder neural network to generate a trajectory prediction output for the target agent that predicts a future trajectory of the target after the current time point.

Trajectory prediction using efficient attention neural networks

Methods, systems, and apparatus for predicting future trajectories of agents in an environment. In one aspect, a system comprises one or more computers configured to receive a data set comprising multiple training examples. The training examples include scene data comprising respective agent data for multiple agents and a ground truth trajectory for a target agent that represents ground truth motion of the target agent after a corresponding time point. The one or more computers obtain data identifying one or more of the multiple agents as non-causal agents for each training example. A non-causal agent is an agent whose states do not cause the ground truth trajectory for the target agent to change. The one or more computers generate a respective modified training example from each of the multiple training examples.

Robust behavior prediction neural networks through non-causal agent based augmentation

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using context-sensitive fusion.

Agent trajectory prediction using context-sensitive fusion

A system obtains scene context data characterizing the environment. The scene context data includes data that characterizes a trajectory of an agent in a vicinity of a vehicle up to a current time point. The system identifies a plurality of initial target locations, and generates, for each of a plurality of target locations that each corresponds to one of the initial target locations, a respective predicted likelihood score that represents a likelihood that the target location will be an intended final location for a future trajectory of the agent. For each target location in a first subset of the target locations, the system generates a predicted future trajectory for the agent given that the target location is the intended final location for the future trajectory. The system further selects, as likely future trajectories of the agent, one or more of the predicted future trajectories.

Agent trajectory prediction using target locations

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using candidate future intents.

Agent trajectory prediction using dynamic candidate future intents

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for predicting the future movement of agents in an environment. In particular, the future movement is predicted through occupancy flow fields that specify, for each future time point in a sequence of future time points and for each agent type in a set of one or more agent types: an occupancy prediction for the future time step that specifies, for each grid cell, an occupancy likelihood that any agent of the agent type will occupy the grid cell at the future time point, and a motion flow prediction that specifies, for each grid cell, a motion vector that represents predicted motion of agents of the agent type within the grid cell at the future time point.

Predicting the future movement of agents in an environment using occupancy flow fields

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using anchor trajectories.

Agent trajectory prediction using anchor trajectories

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for agent trajectory prediction using temporal-spatial interaction predictions.

Multi-modal multi-agent trajectory prediction

Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for performing a conditional behavior prediction for one or more agents. The system obtains context data characterizing an environment. The context data includes data characterizing a plurality of agents, including a query agent and one or more target agents, in the environment at a current time point. The system further obtains data identifying a planned future trajectory for the query agent after the current time point, and for each target agent in the set, processes the context data and the data identifying the planned future trajectory using a first neural network to generate a conditional trajectory prediction output that defines a conditional probability distribution over possible future trajectories of the target agent after the current time point given that the query agent follows the planned future trajectory for the query agent after the current time point.

Conditional agent trajectory prediction

Growing community of inventors

Marina Del Rey, CA, United States of America

Benjamin Sapp