Hong Kong Applied Science and Technology Research Institute Company, Limited

Hong Kong Applied Science and Technology Research

Institute Company Limited

Yan Huo

Tao Li

Hailiang Li

King Ngi Ngan

Ka Man Cheng

Hoi Ming Wong

Jie Dong

Lu Wang

Hon Wah Wong

Chi Keung Fong

Gwo Giun Lee

Chun-Fu Chen

Zhibin Lei

Hing-Mo Lam

Wai-Kuen Cham

Chunhui Cui

Ka Lung Tim Wong

Wai Hong Wong

Ho Yu Wong

Zhihan Wang

Miaohui Hao

Xingbo Guan

Zhi-Song Liu

Wan-Chi Siu

Li-Wen Wang

Oscar Au

Thomas H Y Pun

Yi Yang

Lu Fang

Oscar Chi-Lim Au

Wai Lam Hui

Yui Lam Chan

Wan Chi Siu

Yan Ho Kam

Ciao-Siang Siao

An image-guided diffusion network has two Convolution Neural Networks (CNNs). A RGB image and an IR image are concatenated with a Gaussian noise image and input to a denoising neural network that merges information from the RGB and IR images as noise is removed over many iterations. Then an enhancement neural network up-samples for Super Resolution (SR) and convolutes to generate a condition vector that controls Global Feature Modulation (GFM) at three convolution layers to generate a SRGFM enhanced fusion image. Timesteps are embedded using adaptive group normalization blocks within Adaptive Bottleneck Residual (ABR) blocks in the denoising network, which is a UNet having many levels of ABRs, and in the enhancement network before feature modulation. Global image features are detected by triple convoluting the image input to the enhancement network to generate the condition vector that controls feature modulation blocks at three layers of convolution.

Diffusion-based multiple-modality image fusion

A lossless data compressor prevents normalization overruns on-the-fly as symbol occurrence counts are rounded to generate symbol frequencies, allowing an encoding table generator to generate encoding table entries without waiting for the symbol frequency table to finish filling. Rounding errors are accumulated as symbols are normalized and compensated for by reducing a symbol frequency when the symbol frequency is at least 2 and the accumulated errors have exceeded a threshold. The symbol frequency is also reduced when the number of remaining states in the encoding table is insufficient for a number of remaining unprocessed symbols and states for a current encoding table entry. Since error compensation occurs as symbols are being normalized, encoding table generation is not forced to wait for all symbols in the block to be processed, reducing latency. Three pipeline stages can operate on three input blocks: symbol counting, normalization/error compensation/encoding table generation, and data encoding.

Hardware implementation of frequency table generation for Asymmetric-Numeral-System-based data compression

A data-compression analyzer can rapidly make a binary decision to compress or not compress an input data block or can use a slower neural network to predict the block's compression ratio with a regression model. A Concentration Value (CV) that is the sum of the squares of the frequencies and a Number of Zero (NZ) symbols are calculated from an un-sorted symbol frequency table. A rapid decision to compress is signaled when their product CV*NZ exceeds a horizontal threshold THH. During training, CV*NZ is plotted as a function of compression ratio C % for many training data blocks. Different test values of THH are applied to the plot to determine true and false positive rates that are plotted as a Receiver Operating Characteristic (ROC) curve. The point on the ROC curve having the largest Youden index is selected as the optimum THH for use in future binary decisions.

Predicting compression ratio of data with compressible decision

A train-linking lossless data compressor examines a block of data and uses a same coder to generate a same code when all data values in the input block are identical. When the input data is not all the same value, then a Gaussian coder, a Laplace coder, and a delta coder are activated in parallel. The three compressed code lengths are compared and the smallest code length is output as the compressed code when it is smaller than a copy code length. The copy code is a tag followed by copying all the data in the input block. When the smallest of the three compressed code lengths is larger than the copy code length, the file is not compressible, and the copy code is output. No frequency table is required so latency is low. The delta coder subtracts data values from an average value of the last data block.

Train-linking lossless compressor of numeric values

A neural network receives mono-color pixels in a Bayer pattern from an image sensor and interpolates pixels to generate full-color RGB pixels while also enlightening the image for better detail. A Back-Projection (BP) layer is added to each contracting layer in a U-net convolution neural network where feature depth is increased as pixels are downsampled, while a BP channel shrink layer is added to each expansion layer where feature depth is reduced to upsample and increase pixel resolution. Convolution layers in the BP layer lighten and darken images to generate an error that is then lightened and added to a weighted lightened input. The neural network learns the best weightings to correct for noise and error in the lightening process in these BP layers. Noise is reduced further by repeating a convolution and leaky Rectified Linear Unit (lrelu) in series for the first convolution in the BP layer that performs lightening.

Neural-network for raw low-light image enhancement

This invention discloses a method for object tracking, including determination of an area scaling ratio of the object in a video image sequence. In one embodiment, a centroid of the object is determined. One or more directed straight lines are selected, each passing through the centroid, extending from an end of the object's boundary to an opposite end thereof, and having a direction that is upward. A length scaling ratio for each directed straight line is determined by: determining a motion vector for each selected pixel on the line; computing a scalar component of the motion vector projected onto the line; estimating a change of the line's length according to the scalar components obtained for all pixels; and determining the length scaling ratio according to the change of the line's length. The area scaling ratio is computed based on the length scaling ratios for all directed straight lines.

Scale changing detection and scaling ratio determination by using motion information

Method and apparatus for providing a fast and accurate video coding process are disclosed. After checking the coding history of certain coded video frame units of a video, the order of the inter prediction and the intra prediction is adaptively exchanged for each coding video frame unit of an inter frame. Furthermore, the computations for coding modes in the latter part of the computation order are selectively skipped so as to speed up the coding process without degrading the video quality.

Method and apparatus for coding mode selection

This invention relates to method and apparatus for subpixel-based down-sampling. This invention implements an adaptive filterbased on edge detection, which removes visible color fringing artifacts while efficiently retaining sharpness.

Method and apparatus for subpixel-based down-sampling

The present invention relates to method and apparatus for improving video quality. The present invention provides a unified loop filter including: a pixel determining unit which determines the type of a pixel based on boundary strength; a similarity transforming unit which transforms a nonlinear filter into a nonlinear similarity-ordered statistics filter; and an integrating unit which integrates the nonlinear similarity-ordered statistics filter with a linear image filtering portion. The unified loop filter is applicable to filter reconstructed frames when an encoder or a decoder is processing a video signal.

Method and apparatus for improving video quality by utilizing a unified loop filter

The invention is used in video coding. Systems, apparatuses and methods for processing an order-16 integer transform from an order-8 transform are provided. The order-16 transform method involves expanding an order-8 transform by generating an order-16 integer matrix and a scaling matrix.

Method and device for order-16 integer transform from order-8 integer cosine transform

The presently claimed invention adaptively selects a local de-interlacing method according to information from a compressed video bitstream. According to one embodiment, successive images from a video bitstream are first reconstructed. Syntax elements, selected from a macroblock type, a macroblock partition, a motion vector, a distance from a reference frame, the existence of non-zero transform coefficients and the distribution of transform coefficients, are then extracted from this bitstream. Based on the syntax elements, a de-interlacing algorithm is decided from an algorithm set for each image region in a video frame. Algorithms in the algorithm set include motion compensation, edge-based line averaging, and line averaging. Each image region is thereby interpolated using the de-interlacing algorithm and complementary motion compensation. The complementary motion compensation de-interlaces the image region using an inverse of motion vectors extracted from a bitstream clip representing neighboring frames.

Method and apparatus of de-interlacing video

A method for use in video compression is disclosed. In particular, the claimed invention relates to a method of more efficient fractional-pixel interpolation in two steps by a fixed filter () and an adaptive filter () for fractional-pixel motion compensation.

Method for motion compensation

Some representative embodiments are directed to systems and methods for compressing a data set. In one embodiment, a method comprises receiving a frame of data to be encoded, generating a residual frame that represents a difference between the received frame and one or several reference frames, performing a respective sum of absolute differences (SAD) calculation for each block within the residual frame, and applying a transform function to each data value within the residual frame, wherein the transform function is at least a function of a SAD value calculated for the block containing the respective data value.

Growing community of inventors

Shenzhen, China

Yan Huo