City University of Hong Kong

Dongjie Ye

Sam Tak Wu Kwong

Shiqi Wang

Zhangkai Ni

Haifeng Guo

A method for adaptive reconstruction of a compressively sensed data. The method contains the steps of receiving sensed data; conducting an initial reconstruction to the sensed data to obtain a plurality of first reconstruction patches; by a reconstruction module, conducting a progressive reconstruction to the sensed data to obtain a plurality of second reconstruction patches; summing the plurality of second reconstruction patches with the a plurality of first reconstruction patches to obtain final patches; and merging the final patches to obtain a reconstructed data. The progressive reconstruction further contains concatenating transformer features and convolution features to obtain the second reconstruction patches. The invention provides a hybrid network for adaptive sampling and reconstruction of CS, which integrates the advantages of leveraging both detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning.

Convolution and transformer based compressive sensing

There is provided a computer-implemented method for learned video compression, which includes processing a current frame (x) and previously decoded frame ({circumflex over (x)}) of a video data using a motion estimation model to estimate a motion vector (v) for every pixel, compressing the motion vector (v) and reconstructing the motion vector (v) to a reconstructed motion vector ({circumflex over (v)}), applying an enhanced context mining (ECM) model to obtain enhanced context ({umlaut over (C)}) from the reconstructed motion vector ({circumflex over (v)}) and previously decoded frame feature (x̆), compressing the current frame (x) with the assistance of the enhanced context ({umlaut over (C)}) to obtain a reconstructed frame ({circumflex over (x)}'), and providing the reconstructed frame ({circumflex over (x)}′) to a post-enhancement backend network to obtain a high-resolution frame ({circumflex over (x)}).

Growing community of inventors

Hong Kong, China

Dongjie Ye