Huawei Technologies Co., Limited

Niamul Quader

Juwei Lu

Peng Dai

Wei Li

Deepak Sridhar

Chao Xing

Md Ibrahim Khalil

Srikanth Muralidharan

Yaoxin Li

Arnab Kumar Mondal

Systems, methods, and computer media of processing a video are disclosed. An example method may include: receiving a plurality of video frames of a video; generating a plurality of first input features based on the plurality of video frames; generating a plurality of second input features based on reversing a temporal order of the plurality of first input features; generating a first set of joint attention features based on the plurality of first input features; generating a second set of joint attention features based on the plurality of second input features; and concatenating the first set of joint attention features and the second set of joint attention features to generate a final set of joint attention features.

Systems, methods and computer media for joint attention video processing

Methods and systems are described for performing video retrieval together with video grounding. A word-based query for a video is and encoded into a query representation using a trained query encoder. One or more similar video representations are identified, from a plurality of video representations that are similar to the query representation. Each similar video representation represents a respective relevant video. A grounding is generated for each relevant video by forward propagating each respective similar video representation together with the query representation through a trained grounding module. The relevant videos or identifiers of the relevant videos are outputted together with the grounding generated for each relevant video.

Systems and methods for video retrieval and grounding

An adaptive action recognizer for video that performs multiscale spatiotemporal decomposition of video to generate lower complexity video. The adaptive action recognizer has a number of processing pathways, one for each level of video complexity with each processing pathway having a different computational cost. The adaptive action recognizer applies a decision making scheme that encourages using low average computational costs while retaining high accuracy.

Adaptive action recognizer for video

Methods and systems for updating the weights of a set of convolution kernels of a convolutional layer of a neural network are described. A set of convolution kernels having attention-infused weights is generated by using an attention mechanism based on characteristics of the weights. For example, a set of location-based attention multipliers is applied to weights in the set of convolution kernels, a magnitude-based attention function is applied to the weights in the set of convolution kernels, or both. An output activation map is generated using the set of convolution kernels with attention-infused weights. A loss for the neural network is computed, and the gradient is back propagated to update the attention-infused weights of the convolution kernels.

Growing community of inventors

Toronto, Canada

Niamul Quader