Trajectory prediction for autonomous vehicles using attention mechanism

Abstract

Techniques are discussed herein for using a machine learning attention mechanism to predict movements, states, and/or trajectories of agents in various environments. In various examples, a prediction component of an autonomous vehicle may analyze sensor data to determine, for individual agents in the environment, unique sets of additional objects that are relevant to predicting the subsequent movements of the individual agents. For a particular agent, the prediction component may determine the relative positions and/or states between the agent and the associated set of relevant objects for the agent, and may use an attention mechanism to determine an object interaction vector including weighted attention scores for each additional object relative to the agent. Object interaction vectors may be generated for any number of agents and/or any number of timesteps to determine predicted agent movements and to forecast subsequent driving scenes within the environment.