Leland Stanford Junior University

Toyota Research Institute, Inc.

Ehsan Adeli-Mosabbeb

Adrien David Gaidon

Kuan-Hui Lee

Juan Carlos Niebles Duque

Juan Carlos Niebles

Karttikeya Mangalam

Bingbin Liu

De-An Huang

Haoye Cai

Zhangjie Cao

Boxiao Pan

Kuan Lee

Kathleen Poston

Mandy Lu

Many embodiments of the invention include systems and methods for evaluating motion from a video, the method includes identifying a target individual in a set of one or more frames in a video, analyzing the set of frames to determine a set of pose parameters, generating a 3D body mesh based on the pose parameters, identifying joint positions for the target individual in the set of frames based on the generated 3D body mesh, predicting a motion evaluation score based on the identified join positions, providing an output based on the motion evaluation score.

Systems and methods for estimation of Parkinson's Disease gait impairment severity from videos using MDS-UPDRS

A method for scene perception using video captioning based on a spatio-temporal graph model is described. The method includes decomposing the spatio-temporal graph model of a scene in input video into a spatial graph and a temporal graph. The method also includes modeling a two branch framework having an object branch and a scene branch according to the spatial graph and the temporal graph to learn object interactions between the object branch and the scene branch. The method further includes transferring the learned object interactions from the object branch to the scene branch as privileged information. The method also includes captioning the scene by aligning language logits from the object branch and the scene branch according to the learned object interactions.

Spatio-temporal graph for video captioning with knowledge distillation

A system and method for predicting pedestrian intent is provided. A prediction circuit comprising a plurality of gated recurrent units (GRUB) receives a sequence of images captured by a camera. The prediction circuit parses each frame of the sequence of images to identify one or more pedestrians and one or more objects. Using the parsed data, the prediction circuit generates a pedestrian-centric spatiotemporal graph, the parsed data comprising one or more identified pedestrians and one or more identified object. The prediction circuit uses the pedestrian-centric graph to determine a probability of one or more pedestrians crossing a street for each frame of the sequence of images.

Spatiotemporal relationship reasoning for pedestrian intent prediction

A method for predicting spatial positions of several key points on a human body in the near future in an egocentric setting is described. The method includes generating a frame-level supervision for human poses. The method also includes suppressing noise and filling missing joints of the human body using a pose completion module. The method further includes splitting the poses into a global stream and a local stream. Furthermore, the method includes combining the global stream and the local stream to forecast future human locomotion.

Disentangling human dynamics for pedestrian locomotion forecasting with noisy supervision

Growing community of inventors

Mountain View, CA, United States of America

Ehsan Adeli-Mosabbeb