Hong Kong Applied Science and Technology Research Institute Company, Limited

King Hung Chiu

Luhong Liang

Peng Luo

Xuejiao Liu

Kwok Wai Hung

Wai Keung Cheung

Chao Shi

Tim Ka Lung Wong

Edmund Y Lam

A bit-depth optimization engine reduces the hardware cost of a neural network. When training data is applied to a neural network during training routines, accuracy cost and hardware costs are generated. A hardware complexity cost generator generates costs for weights near bit-depth steps where the number of binary bits required to represent a weight decreases, such as from 2to 2&#x2212;1, where one less binary bit is required. Gradients are generated from costs for each weight, and weights near bit-depth steps are easily selected since they have a large gradient, while weights far away from a bit-depth step have near-zero gradients. The selected weights are reduced during optimization. Over many cycles of optimization, a low-bit-depth neural network is generated that uses fewer binary bits per weight, resulting in lower hardware costs when the low-bit-depth neural network is manufactured on an Application-Specific Integrated Circuit (ASIC).

Method and system for bit-depth reduction in artificial neural networks

An image processor generates a Super-Resolution (SR) frame by upscaling. A Human Visual Preference Model (HVPM) helps detect random texture regions, where visual artifacts and errors are tolerated to allow for more image details, and immaculate regions having flat areas, corners, or regular structures, where details may be sacrificed to prevent annoying visual artifacts that seem to stand out more. A regularity or isotropic measurement is generated for each input pixel. More regular and less anisotropic regions are mapped as immaculate regions. Higher weights for blurring, smoothing, or blending from a single frame source are assigned for immaculate regions to reduce the likelihood of generated artifacts. In the random texture regions, multiple frames are used as sources for blending, and sharpening is increased to enhance details, but more artifacts are likely. These artifacts are more easily tolerated by humans in the random texture regions than in the regular-structure immaculate regions.

Visual-experience-optimized super-resolution frame generator

A frame-rate converter reduces halo artifacts along edges of moving objects. Halo artifacts occur on interpolated frames where a moving object covers and uncovers pixels along its edges. Motion estimation among three original frames produces hybrid direction motion vectors that are bi-directional for background and objects, but are unidirectional for covered and uncovered regions, since motion vectors with large matching errors are deleted. Covered regions in the interpolated frame are detected as intersecting only a forward but no backward hybrid motion vector. Bi-directional motion estimation from the hybrid motion vectors of two original frames produces refined motion vectors for the interpolated frame. Refined motion vectors in the covered regions are deleted and replaced with hybrid motion vectors from the original frames. Hybrid motion vectors from the original frames are assigned to the critical covered regions rather than using interpolated vectors in the covered regions, reducing halo artifacts.

Halo reduction in frame-rate-conversion using hybrid bi-directional motion vectors for occlusion/disocclusion detection

Embodiments of the present invention include apparatuses, systems and methods for multi-patch based super-resolution from a single video frame. Such embodiments include a scale-invariant self-similarity (SiSS) based super-resolution method. Instead of searching HR examples in a database or in LR image, the present embodiments may select the patches according to the SiSS characteristics of the patch itself, so that the computational complexity of the method may be reduced because there is not any search involved. To solve the problem of lack of relevant examples in natural images, the present embodiments may employ multi-shaped and multi-sized patches in HR image reconstruction. Additionally, embodiments may include steps for a hybrid weighing method for suppressing artifacts. Advantageously, certain embodiments of the method may be 10&#x2dc;1,000 times faster than the example based SR approaches using patch searching and can achieve comparable HR image quality.

Growing community of inventors

Hong Kong, China

King Hung Chiu