Nvidia Corporation

Honda Motor Co., Ltd.

Xitong Yang

Jan Kautz

Xiaodong Yang

Ming-Yu Liu

Fanyi Xiao

Teruhisa Misu

Yi-Ting Chen

Sifei Liu

Ahmed Taha

Larry Davis

Iterative prediction systems and methods for the task of action detection process an inputted sequence of video frames to generate an output of both action tubes and respective action labels, wherein the action tubes comprise a sequence of bounding boxes on each video frame. An iterative predictor processes large offsets between the bounding boxes and the ground-truth.

Iterative spatio-temporal action detection in video

There are numerous features in video that can be detected using computer-based systems, such as objects and/or motion. The detection of these features, and in particular the detection of motion, has many useful applications, such as action recognition, activity detection, object tracking, etc. The present disclosure provides a neural network that learns motion from unlabeled video frames. In particular, the neural network uses the unlabeled video frames to perform self-supervised hierarchical motion learning. The present disclosure also describes how the learned motion can be used in video action recognition.

Self-supervised hierarchical motion learning for video action recognition

Multi-modal data representing driving events and corresponding actions related to the driving events can be obtained and used to train a neural network at least in part by using a triplet loss computed for the driving events as a regression loss to determine an embedding of driving event data. In some cases, using the trained neural network, a retrieval request for an input driving event and corresponding action can be processed by determining, from the neural network, one or more similar driving events or corresponding actions in the multi-modal data.

Systems for modeling uncertainty in multi-modal retrieval and methods thereof

Growing community of inventors

Greenbelt, MD, United States of America

Xitong Yang