Qualcomm Incorporated

Other

Zhenyang Li

Cornelis Gerardus Maria Snoek

Efstratios Gavves

Mihir Jain

Amir Ghodrati

Kirill Gavrilyuk

A method of pixel-wise localization of an actor and an action in a sequence of frames includes receiving a natural language query describing the action and the actor. The method also includes receiving the sequence of frames. The method further includes localizing the action and the actor in the sequence of frames based on the natural language query.

Spatio-temporal action and actor localization

A method of predicting action labels for a video stream includes receiving the video stream and calculating an optical flow of consecutive frames of the video stream. An attention map is generated from the current frame of the video stream and the calculated optical flow. An action label is predicted for the current frame based on the optical flow, a previous hidden state and the attention map.

Recurrent networks with motion-based attention for video understanding

A method of processing data within a convolutional attention recurrent neural network (RNN) includes generating a current multi-dimensional attention map. The current multi-dimensional attention map indicates areas of interest in a first frame from a sequence of spatio-temporal data. The method further includes receiving a multi-dimensional feature map. The method also includes convolving the current multi-dimensional attention map and the multi-dimensional feature map to obtain a multi-dimensional hidden state and a next multi-dimensional attention map. The method identifies a class of interest in the first frame based on the multi-dimensional hidden state and training data.

Growing community of inventors

Amsterdam, Netherlands

Zhenyang Li