Huawei Technologies Co., Limited

Juwei Lu

Peng Dai

Wei Li

Wei Zhou

Deepak Sridhar

Jin Tang

Jianpeng Xu

Xiaofei Wu

Varshanth Ravindra Rao

Niamul Quader

Yannick Verdie

Sayem Mohammad Siam

Md Ibrahim Khalil

Songcen Xu

Gaganpreet Singh

Yuanhao Yu

Sachi Mizobuchi

Taslim Arefin Khan

Ghazaleh Saniee-Monfared

Mona Hosseinkhani Loorak

Rafael Veras Guimaraes

Xiu Yi

Srikanth Muralidharan

Zihao Yang

Hanwen Liang

Bharath Sekar

Ruiwen Li

Zheng Liu

Yang Wang

Richard Patrick Wildes

Steven George McDonagh

Zhixiang Chi

Wentao Liu

Rasoul Mohammadi Nasiri

Chao Xing

Xiaolin Wu

Shuhao Li

Zi Hao Yang

Ali Ghobadzadeh

Xin Ding

Sidharth Singla

Qiong Zhang

Yaoxin Li

Yanhui Guo

Uzair Ahmad

Arnab Kumar Mondal

Methods and devices for machine vision-based selection of content are described. One or more hands are detected in a current frame of video data. A respective fingertip location is determined for each of up to two of the detected hands. A content selection gesture is determined corresponding to the up to two detected hands. Selected content is extracted, as indicated by the content selection gesture and based on the up to two fingertip locations. The device may be a smartphone, a tablet, a laptop, a smart light device, a reader device, etc.

Devices and methods for gesture-based selection

Systems and methods for temporal action localization of video data are described. A feature representation extracted from video data has a temporal dimension and a spatial dimension. The feature representation is self-aligned in the spatial dimension. Spatial multi-sampling is performed to obtain a plurality of sparse samples of the self-aligned representation along the spatial dimension, and the multi-sampled representation is fused with the self-aligned representation. Attention-based context information aggregation is applied on the fused representation to obtain a spatially refined representation. Local temporal information aggregation is applied on the self-aligned representation to obtain a temporally refined representation. Action localization is performed on a concatenation of the spatially refined representation and the temporally refined representation.

Methods and systems for temporal action localization of video data

Method and devices for training a keypoint estimation network are described. In each training iteration, synthetic images are generated by a generator, each synthetic image being assigned respective assigned keypoints by the generator. Using a prior-iteration of the keypoint estimation network, a set of predicted keypoints is obtained for each synthetic image. Based on an error score between the predicted keypoints and the assigned keypoints, poor quality synthetic images are discarded. The remaining synthetic images, together with real world images, are used to train an updated keypoint estimation network. The performance of the updated keypoint estimation network is validated, and the training iterations are performed until a convergence criteria is satisfied.

Methods, devices, and computer readable media for training a keypoint estimation network using cGAN-based data augmentation

Methods and devices are described for computer vision-based gesture detection. From a frame of image data, extracted locations of keypoints of a detected hand are obtained. The extracted locations are normalized to obtain normalized features. The normalized features are processed using a trained decision tree ensemble to generate a probability of a valid gesture for the detected hand. The generated probability is compared with a defined decision threshold to generate a binary classification to classify the detected hand as a valid gesture or invalid gesture.

Devices and methods for single or multi-user gesture detection using computer vision

Methods, systems, and media for training deep neural networks for cross-domain few-shot classification are described. The methods comprise an encoder and a decoder of a deep neural network. The training of the autoencoder comprises two training stages. For each iteration in the first training stage, a batch of data samples from the source dataset are sampled and fed to the encoder to generate a plurality of source feature maps, then determining a first training stage loss, which updates the autoencoder's parameters. For each iteration in the second training stage, the novel dataset is split into a support set and a query set. The support set is fed to the encoder to determine a prototype for each class label. The query set is also fed to the encoder to calculate a query set metric classification loss. The query set metric classification loss updates the autoencoder's parameters.

Methods and systems for cross-domain few-shot classification

Methods and systems for estimation of a 3D hand pose are disclosed. A 2D image containing a detected hand is processed using a U-net network to obtain a global feature vector and a heatmap for the keypoints of the hand. Information from the global feature vector and the heatmap are concatenated to obtain a set of input tokens that are processed using a transformer encoder to obtain a first set of 2D keypoints representing estimated 2D locations of the keypoints in a first view. The first set of 2D keypoints are inputted as a query to a transformer decoder, to obtain a second set of 2D keypoints representing estimated 2D locations of the keypoints in a second view. The first and second sets of 2D keypoints are aggregated to output the set of estimated 3D keypoints.

Methods and systems for 3D hand pose estimation from RGB images

Methods and systems for gesture-based control of a device are described. An input frame is processed to determine a location of a distinguishing anatomical feature in the input frame. A virtual gesture-space is defined based on the location of the distinguishing anatomical feature, the virtual gesture-space being a defined space for detecting a gesture input. The input frame is processed in only the virtual gesture-space, to detect and track a hand. Using information generated from detecting and tracking the at least one hand, a gesture class is determined for the at least one hand. The device may be a smart television, a smart phone, a tablet, etc.

Methods and systems for hand gesture-based control of a device

Methods and systems for gesture-based control of a device are described. A virtual gesture-space is determined in a received input frame. The virtual gesture-space is associated with a primary user from a ranked user list of users. The received input frame is processed in only the virtual gesture-space, to detect and track a hand. Using a hand bounding box generated by detecting and tracking the hand, gesture classification is performed to determine a gesture input associated with the hand. A command input associated with the determined gesture input is processed. The device may be a smart television, a smart phone, a tablet, etc.

Methods, systems, and media for image searching are described. Images comprising one query image and a plurality of candidate images are received. For each candidate image, a first model similarity measure from an output of a first model configured for scene classification to perceive scenes in the images is determined. Further, for each candidate image of the plurality of candidate images, a second model similarity measure from the output of a second model configured for attribute classification to perceive attributes in the images is determined. For each candidate image of the plurality of candidate images, a similarity agglomerate index of a weighted aggregate of the first model similarity measure and the second model similarity measure is computed. The plurality of candidate images based on the respective similarity agglomerate index of each candidate image are ranked and a first ranked candidate images corresponding to the searched images are generated.

Methods, systems, and media for image searching

Methods, devices and processor-readable media for an integrated teacher-student machine learning system. One or more teacher-student modules are trained as part of the teacher neural network training. Each student sub-network uses a portion of the teacher neural network to generate an intermediate feature map, then provides the intermediate feature map to a student sub-network to generate inferences. The student sub-network may use a feature enhancement block to map the intermediate feature map to a subsequent feature map. A compression block may be used to compress intermediate feature map data for transmission in some embodiments.

Methods, devices and media providing an integrated teacher-student system

Systems, methods, and computer media of processing a video are disclosed. An example method may include: receiving a plurality of video frames of a video; generating a plurality of first input features based on the plurality of video frames; generating a plurality of second input features based on reversing a temporal order of the plurality of first input features; generating a first set of joint attention features based on the plurality of first input features; generating a second set of joint attention features based on the plurality of second input features; and concatenating the first set of joint attention features and the second set of joint attention features to generate a final set of joint attention features.

Systems, methods and computer media for joint attention video processing

Methods, devices, and processor-readable media for adjusting the control-display gain of a gesture-controlled device are described. Adjusting the control-display gain may facilitate user interaction with content or UI elements rendered on a display screen of the gesture-controlled device. The control-display gain may be adjusted based on a property of how a mid-air dragging gesture is being performed by a user's hand. The property may be the location of the gesture, the orientation of the hand performing the gesture, or the velocity of the gesture. A hand that becomes stationary for a threshold time period while performing the dragging gesture may adjust the control-display gain to a different level. Control-display gain may be set to a different value based on the current velocity of the hand performing the gesture. The control-display gain levels may be selected from a continuous range of values or a set of discrete values. Devices for performing the methods are described.

Method and device for adjusting the control-display gain of a gesture controlled electronic device

Methods and systems for determining a surface color of a target surface under an environment with an environmental light source. A plurality of images of the target surface are captured as the target surface is illuminated with a variable intensity, constant color light source and a constant intensity, constant color environmental light source, wherein the intensity of the light source on the target surface is varied by a known amount between the capturing of the images. A color feature tensor, independent of the environmental light source, is extracted from the image data, and used to infer a surface color of the target surface.

Method, system, and device for color measurement of a surface

A method, device and computer-readable medium for generating a super-resolution version of a compressed video stream. By leveraging the motion information and residual information in compressed video streams, described examples are able to skip the time-consuming motion-estimation step for most frames and make the most use of the SR results of key frames. A key frame SR module generates SR versions of I-frames and other key frames of a compressed video stream using techniques similar to existing multi-frame approaches to VSR. A non-key frame SR module generates SR version of the non-key inter frames between these key frames by making use of motion information and residual information used to encode the inter frames in the compressed video stream.

Method, device, and medium for generating super-resolution video

Methods and systems are described for performing video retrieval together with video grounding. A word-based query for a video is and encoded into a query representation using a trained query encoder. One or more similar video representations are identified, from a plurality of video representations that are similar to the query representation. Each similar video representation represents a respective relevant video. A grounding is generated for each relevant video by forward propagating each respective similar video representation together with the query representation through a trained grounding module. The relevant videos or identifiers of the relevant videos are outputted together with the grounding generated for each relevant video.

Systems and methods for video retrieval and grounding

Growing community of inventors

North York, Canada

Juwei Lu