Baidu USA LLC

Baidu.com Times Technology (Beijing) Co. Ltd.

Huawei Technologies Co., Limited

Future Wei Technologies, Inc.

Baopu Li

Yingze Bao

Zhihong Pan

Zhiyu Cheng

Jun Yang

Yanwen Fan

Yiwei Zhao

Hsuchun Cheng

Shuo Huang

Gang Zhang

Dongliang He

Wenhao Wu

Wenyou Sun

Juan Chen

Yuchen Bian

Min-Hung Chen

Qiuling Suo

Tianwei Lin

Teng Xi

Image super-resolution (SR) refers to the process of recovering high-resolution (HR) images from low-resolution (LR) inputs. Blind image SR is a more challenging task which involves unknown blurring kernels and characterizes the degradation process from HR to LR. In the present disclosure, embodiments of a variational autoencoder (VAE) are leveraged to train a kernel autoencoder for more accurate degradation representation and more efficient kernel estimation. In one or more embodiments, a kernel-agnostic loss is used to learn more robust kernel features in the latent space from LR inputs without using ground-truth kernel references. In addition, attention-based adaptive pooling is introduced to improve kernel estimation accuracy, and spatially non-uniform kernel features are passed into SR restoration resulting in additional kernel estimation error tolerance. Extensive experiments on synthetic and real-world images show that embodiments of the presented model outperform state-of-the-art methods significantly with the peak signal-to-noise ratio (PSNR) raised considerably.

Robust and efficient blind super-resolution using variational kernel autoencoder

Model pruning is used to trim large neural networks, like convolutional neural networks (CNNs), to reduce computation overheads. Existing model pruning methods mainly rely on heuristics rules or local relationships of CNN layers. A novel hypernetwork based on graph neural network is disclosed for generating and evaluating pruned networks. A graph is first constructed according to information flow of channels and layers in a CNN network, with channels and layers represented as nodes and information flows represented as edges. A graph neural network is applied to aggregate both local and global dependencies across all channels and layers of the CNN network, resulting in informative node embeddings. With such embeddings, pruned CNN networks including their architectures and weights may be effectively generated and evaluated.

Automatic channel pruning via graph neural network based hypernetwork

Network architecture search (NAS) received a lot of attention. The supernet-based differentiable approach is popular because it can effectively share the weights and lead to more efficient search. However, the mismatch between the architecture and weights caused by weight sharing still exists. Moreover, the coupling effects among different operators are also neglected. To alleviate these problems, embodiments of an effective NAS methodology by similarity-based operator ranking are presented herein. With the aim of approximating each layer's output in the supernet, a similarity-based operator ranking based on statistical random comparison is used. In one or more embodiments, then the operator that possibly causes the least change to feature distribution discrepancy is pruned. In one or more embodiments, a fair sampling process may be used to mitigate the operators' Matthew effect that happened frequently in previous supernet approaches.

Neural architecture search via similarity-based operator ranking

Embodiments of this disclosure provide techniques used in a location sensing service and a timeline service implemented in a host device for providing location history. In particular, the location sensing service may passively collect location coordinates of the device using requests made by third party applications. The location sensing service may receive a signal comprising an intent for a location request in response to a change in the user activity or in response to a change in a cellular identification of the device. The device may determine that a time elapsed from a previous location request and a time corresponding with receiving one or more of the signals exceed a time threshold and, based thereon, request a location coordinate of the device.

Multimodal location sensing on a mobile phone

Deep neural networks (DNN) model quantization may be used to reduce storage and computation burdens by decreasing the bit width. Presented herein are novel cursor-based adaptive quantization embodiments. In embodiments, a multiple bits quantization mechanism is formulated as a differentiable architecture search (DAS) process with a continuous cursor that represents a possible quantization bit. In embodiments, the cursor-based DAS adaptively searches for a quantization bit for each layer. The DAS process may be accelerated via an alternative approximate optimization process, which is designed for mixed quantization scheme of a DNN model. In embodiments, a new loss function is used in the search process to simultaneously optimize accuracy and parameter size of the model. In a quantization step, the closest two integers to the cursor may be adopted as the bits to quantize the DNN together to reduce the quantization noise and avoid the local convergence problem.

Cursor-based adaptive quantization for deep neural networks

Described herein are embodiments of a deep residual network dedicated to color filter array mosaic patterns. A mosaic stride convolution layer is introduced to match the mosaic pattern of a multispectral filter arrays (MSFA) or a color filter array raw image. Embodiments of a data augmentation using MSFA shifting and dynamic noise are applied to make the model robust to different noise levels. Embodiments of network optimization criteria may be created by using the noise standard deviation to normalize the Lloss function. Comprehensive experiments demonstrate that embodiments of the disclosed deep residual network outperform the state-of-the-art denoising algorithms in MSFA field.

Deep residual network for color filter array image denoising

A method for information processing for accelerating neural network training. The method includes: acquiring a neural network corresponding to a deep learning task; and performing iterations of iterative training on the neural network based on a training data set. The training data set includes task data corresponding to the deep learning task. The iterative training includes: processing the task data in the training data set using a current neural network, and determining, based on a processing result of the neural network on the task data in a current iterative training, prediction loss of the current iterative training; determining a learning rate and a momentum in the current iterative training; and updating weight parameters of the current neural network by gradient descent based on a preset weight decay, and the learning rate, the momentum, and the prediction loss in the current iterative training. This method achieves efficient and low-cost deep learning-based neural network training.

Method, electronic device and computer readable medium for information processing for accelerating neural network training

Described herein are systems and embodiments for multispectral image demosaicking using deep panchromatic image guided residual interpolation. Embodiments of a ResNet-based deep learning model are disclosed to reconstruct the full-resolution panchromatic image from multispectral filter array (MSFA) mosaic image. In one or more embodiments, the reconstructed deep panchromatic image (DPI) is deployed as the guide to recover the full-resolution multispectral image using a two-pass guided residual interpolation methodology. Experiment results demonstrate that the disclosed method embodiments outperform some state-of-the-art conventional and deep learning demosaicking methods both qualitatively and quantitatively.

Systems and methods for multispectral image demosaicking using deep panchromatic image guided residual interpolation

Presented herein are embodiments of a training deep learning models. In one or more embodiments, a compact deep learning model comprises fewer layers, which require fewer floating-point operations (FLOPs). Presented herein are also embodiments of a new learning rate function, which can adaptively change the learning rate between two linear functions. In one or more embodiments, combinations of half-precision floating point format training together with larger batch size in the training process may also be employed to aid the training process.

Deep learning model embodiments and training embodiments for faster training

Embodiments herein treat the action segmentation as a domain adaption (DA) problem and reduce the domain discrepancy by performing unsupervised DA with auxiliary unlabeled videos. In one or more embodiments, to reduce domain discrepancy for both the spatial and temporal directions, embodiments of a Mixed Temporal Domain Adaptation (MTDA) approach are presented to jointly align frame-level and video-level embedded feature spaces across domains, and, in one or more embodiments, further integrate with a domain attention mechanism to focus on aligning the frame-level features with higher domain discrepancy, leading to more effective domain adaptation. Comprehensive experiment results validate that embodiments outperform previous state-of-the-art methods. Embodiments can adapt models effectively by using auxiliary unlabeled videos, leading to further applications of large-scale problems, such as video surveillance and human activity analysis.

Video action segmentation by mixed temporal domain adaption

The disclosure relates to technology for location-based services, and in particular, to geofencing. A computing device generates multiple circular shaped geofences to cover a geographic region defined by a polygon shaped geofence. The multiple circular shaped geofences are monitored to detect a current location of user equipment entering a boundary of any of the circular shaped geofences. Based on the detection, determining that the current location of the user equipment is within the polygon shaped geofence. A location based service is notified that the user equipment has entered the polygon shaped geofence.

Growing community of inventors

Santa Clara, CA, United States of America

Baopu Li