Behavior modeling based on relative position data and absolute yaw data

Abstract

Techniques for performing prediction based on relative position data and/or absolute yaw data are described herein. A vehicle may detect an object in an environment. The vehicle may generate an embedding associated with the object and input the embedding into a machine learned model. The machine learned model may add the absolute yaw of the object to the embedding, generate a rotation matrix based on the pose of the object, and apply the rotation matrix to the embedding. Based on modifying the embedding (or generating a modified embedding), the attention layer of the machine learned model may perform attention on the modified embedding and respond by outputting output data. The vehicle may rotate the output data (or generate rotated output data) which may be used by one or more machine learned models to predict object behavior, generate vehicle actions, etc.