Qualcomm Incorporated

Huawei Device Co., Ltd.

Xin Li

Yingyong Qi

Jiancheng Lyu

Lei Wang

Ning Bi

Shuxue Quan

Bolan Jiang

Yunke Pan

Chun-Ting Huang

Techniques are provided for image processing. For instance, a process can include obtaining an image; extracting a first set of features at a first scale resolution; extracting a second set of features at a second scale resolution (lower than the first scale resolution); performing a self-attention transform to generate similarity scores for the second set of features; adding the similarity scores to the second set of features to generate a first feature extractor output; up-sampling the first feature extractor output to generate a second feature extractor output; adding the second feature extractor output to the first set of features to generate a third feature extractor output; receiving an instance query; performing a cross-attention transform on the instance query and the first feature extractor output to generate a set of weights; and matrix multiplying the set of weights and the third feature extractor output to generate instance masks.

Convolution and transformer-based image segmentation

Methods and systems of frame based image segmentation are provided. For example, a method for feature object tracking between frames of video data is provided. The method comprises receiving a first frame of video data, extracting a mask feature for each of one or more objects of the first frame, adjusting the first frame by applying each initial mask and corresponding identification to a respective object of the first frame, and outputting the adjusted first frame. The method further comprises tracking the one or more objects in one or more consecutive frames. The tracking comprises extracting a masked feature for each of one or more objects in the consecutive frame, adjusting the consecutive frame by applying each initial mask and corresponding identification for the consecutive frame to the respective object of the one or more objects of the consecutive frame, and outputting the adjusted consecutive frame.

Frame-based video segmentation

Systems and techniques are provided for performing video-based activity recognition. For example, a process can include extracting, using a first machine learning model, first one or more features from a first frame and second one or more features from a second frame. The first one or more features and the second one or more features are associated with a person driving a vehicle. The process can include processing, using a second machine learning model, the first one or more features and the second one or more features. The process can include determining, based on processing of the first one or more features and the second one or more features using the second machine learning model, at least one activity associated with the person driving the vehicle.

Video-based activity recognition

Provided are systems, methods, and computer-readable medium for operating a neural network. In various implementations, the neural network can receive an input image that includes an object to be identified. The neural network can generate a plurality of initial feature maps using a convolution layers, wherein a first initial feature maps is generated using the input image. The neural network can generate an up-sampled feature map using a de-convolution layer that takes an initial feature map as input, where the up-sampled feature map has a same resolution as the previous initial feature map. The neural network can combine the up-sampled feature map and the previous initial feature map, and use the combined feature map to more accurate identify the object.

Combining convolution and deconvolution for object detection

Mobile phone

Growing community of inventors

San Diego, CA, United States of America

Xin Li