Honda Motor Co., Ltd.

Toyota Research Institute, Inc.

Jiachen Li

Chiho Choi

Haiming Gang

Hengbo Ma

Kikuo Fujimura

David Francis Isele

Srikanth Malla

Blake Warren Wulfe

Akira Kanehara

Jin Ge

Harshayu Girase

Mykel John Kochenderfer

Xiaobai Ma

A system and method for providing long term and key intentions for trajectory prediction that include receiving image data and LiDAR data associated with RGB images and LiDAR point clouds that are associated with a surrounding environment of an ego agent and processing a long term and key intentions for trajectory prediction dataset (LOKI dataset) that is utilized to complete joint trajectory and intention prediction for heterogeneous traffic agents. The system and method also include encoding a past observation history of each of the heterogeneous traffic agents and sampling a respective goal. The system and method further include decoding and predicting future trajectories associated with each of the heterogeneous traffic agents based on data included within the LOKI dataset, the encoded past observation history, and the respective goal.

System and method for providing long term and key intentions for trajectory prediction

Object identification may be provided herein. A feature extractor may extract a first set of visual features, extract a second set of visual features, concatenate the first set of visual features, the second set of visual features, and a set of bounding box information, determine a number of object features and a global feature for a scene, and receive ego-vehicle feature information associated with an ego-vehicle. An object classifier may receive the number of object features, the global feature, and the ego-vehicle feature information, generate relational features with respect to relationships between each of the number of objects from the scene, and classify each of the number of objects from the scene based on the number of object features, the relational features, the global feature, the ego-vehicle feature information, and an intention of the ego-vehicle.

Autonomous vehicle object classifier method and system

A system and method for completing continual multi-agent trajectory forecasting with a graph-based conditional generative memory system that include receiving data associated with a surrounding location of an ego agent and inputting the data associated with the surrounding location of the ego agent to at least one episodic memory buffer and processing scene graphs associated with the surrounding location of the ego agent that are associated with the plurality of time steps. The system and method additionally include aggregating the data associated with the surrounding location of the ego agent associated with the plurality of time steps into mixed data and training a generative memory and a predictor with the mixed data. The system and method further include predicting future trajectories associated with traffic agents that are located within the surrounding location of the ego agent based on the training of the generative memory and the predictor.

System and method for completing continual multi-agent trajectory forecasting

The systems and methods herein utilize interactive Gaussian processes for crowd navigation. For example, an encoder receives sensor data and context information. The encoder also extracts interaction patterns from observed trajectories from the sensor data and context information. The encoder further generates a static latent interaction graph for a first time step based on the interaction patterns. A recurrent module generates a distribution of time dependent static latent interaction graphs iteratively from the first time step for a series of time steps based on the static latent interaction graph. The series of time steps are separated by a re-encoding gap. The decoder generates multi-modal distribution of future states based on the distribution of time dependent static latent interaction graphs.

Systems and methods for heterogeneous multi-agent multi-modal trajectory prediction with evolving interaction graphs

Systems and methods for path planning with latent state inference and spatial-temporal relationships are provided. In one embodiment, a system includes an inference module, a policy module, a graphical representation module, and a planning module. The inference module receives sensor data associated with a plurality of agents. The inference module maps the sensor data to a latent state distribution to identify latent states of the plurality of agents. The latent states identify agents as cooperative or aggressive. The policy module predicts future trajectories of the plurality of agents at a given time based on sensor data and the latent states of the plurality of agents. The graphical representation module generates a graphical representation based on the sensor data and a graphical representation neural network. The planning module generates a motion plan for the ego agent based on the predicted future trajectories and the graphical representation.

Systems and methods for path planning with latent state inference and graphical relationships

Systems and methods are provided for implementing hybrid prediction. Hybrid prediction integrates two deep learning based trajectory prediction approaches: grid-based approaches and graph-based approaches. Hybrid prediction techniques can achieve enhanced performance by combining the grid and graph approaches in a manner that incorporates appropriate inductive biases for different elements of a high-dimensional space. A hybrid prediction framework processor can generate trajectory predictions relating to movement of agents in a surrounding environment based on a prediction model generating using hybrid prediction. Trajectory predictions output from the hybrid prediction framework processor can be used to control an autonomous vehicle. For example, the autonomous vehicle can perform safety-aware and autonomous operations to avoid oncoming objects, based on the trajectory predictions.

Growing community of inventors

Albany, CA, United States of America

Jiachen Li