The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Mar. 14, 2023
Filed:
Mar. 23, 2020
Toyota Research Institute, Inc.;
The Board of Trustees of the Leland Stanford Junior University, Stanford, CA (US);
Boxiao Pan, Palo Alto, CA (US);
Haoye Cai, Palo Alto, CA (US);
De-An Huang, Stanford, CA (US);
Kuan-Hui Lee, San Jose, CA (US);
Adrien David Gaidon, San Jose, CA (US);
Ehsan Adeli-Mosabbeb, Mountain View, CA (US);
Juan Carlos Niebles Duque, Mountain View, CA (US);
TOYOTA RESEARCH INSTITUTE, INC., Los Altos, CA (US);
THE BOARD OF TRUSTEES OF THE LELAND STANFORD JUNIOR UNIVERSITY, Stanford, CA (US);
Abstract
A method for scene perception using video captioning based on a spatio-temporal graph model is described. The method includes decomposing the spatio-temporal graph model of a scene in input video into a spatial graph and a temporal graph. The method also includes modeling a two branch framework having an object branch and a scene branch according to the spatial graph and the temporal graph to learn object interactions between the object branch and the scene branch. The method further includes transferring the learned object interactions from the object branch to the scene branch as privileged information. The method also includes captioning the scene by aligning language logits from the object branch and the scene branch according to the learned object interactions.