Peking University

Tencent Technology (shenzhen) Company Limited

Shenzhen China Star Optoelectronics Technology Co., Ltd.

Orange

Shenzhen China Star Optoelectronics Semiconductor Display Technology Co., Ltd.

Ronggang Wang

Wen Gao

Zhenyu Wang

Wenmin Wang

Ge Li

Shengfu Dong

Ying Li

Kui Fan

Hui Zhao

Tiejun Huang

Xiubao Jiang

Yueming Wang

Hao Li

Yang Zhao

Lin Li

Xinxin Zhang

Xi Xie

Mengdi Fan

Siwei Ma

Yufeng Jin

Jing Lv

Ming-Jong Jou

Shen-sian Syu

Jingchang Chen

Yihao Zhang

Peilei Dong

Ruonan Zhang

Guisen Xu

Mingmin Zhen

Chengzhou Tang

Xianghao Zang

Lei Chen

Xunan Mao

Chenchen Gu

Sheng Wang

Yufan Deng

Chao Wang

Mingjong Jou

ShenSian Syu

Hongbin Cao

Zhihao Li

Shen Zhang

Hao Lv

Jie Wan

Haoxian Zhang

Hongmeng Song

Liang Han

Chuanxin Tang

Jianbo Jiao

Chunbiao Zhu

Bocheng Xin

Chenxia Li

Yongbing Zhang

Chenchen Yu

Quanzhan Zheng

Lu Ran

A method and a device for encoding or decoding based on an inter-frame prediction. The method includes steps of: determining a temporal motion vector prediction value of a to-be-processed coding unit, where the temporal motion vector prediction value is a temporal motion vector prediction value of a sub-block, a temporal motion vector of which is obtainable through prediction, in sub-blocks adjacent to the to-be-processed coding unit and/or sub-blocks in the to-be-processed coding unit; determining a motion vector residual prediction value of the to-be-processed coding unit according to the temporal motion vector prediction value; determining a motion vector of a sub-block in the to-be-processed coding unit according to the temporal motion vector prediction value and the motion vector residual prediction value and performing a motion compensation according to the motion vector of the sub-block in the to-be-processed coding unit to determine a prediction block of the to-be-processed coding unit.

Method and device for encoding or decoding based on inter-frame prediction

A method for deriving motion vector is provided, this method includes: obtaining a space domain motion vector prediction and a time domain motion vector prediction of adjacent blocks of a coding unit in a predetermined direction; performing a filtering operation on the space domain motion vector and the time domain motion vector prediction to obtain the space domain motion vector prediction and the time domain motion vector prediction of the filtered adjacent blocks; determining, according to a predetermined inter-frame prediction mode, reference motion vectors of a current block in four side directions by using the space domain motion vector prediction and the time domain motion vector prediction of the filtered adjacent blocks and a coordinate position of the current block in the coding unit; and deriving motion vectors of the current block according to the reference motion vectors and the coordinate position of the current block in the coding unit.

Method for deriving motion vector, and electronic device of current block in coding unit

Embodiments of the present disclosure provide a method and device for selecting a context model of a quantized coefficient end flag. The method comprises: obtaining a scanning position POS of a non-zero coefficient corresponding to current quantized coefficient end flag in a specific scanning order; wherein the scanning position POS is a subscript of the non-zero coefficient in the scanning order; configuring a first context model array, and using a fixed value as the base to calculate the logarithmic value of the value of the scanning position POS plus 1, and according to the logarithmic value, selecting a first context model from the first context model array; and using the first context model to encode or decode a binary symbol of the current quantized coefficient end flag. According to the technical solution of the present application, it is able to improve encoding and decoding efficiency for a quantized coefficient end flag, thereby further improve the efficiency of video encoding and decoding.

Method and device for selecting context model of quantization coefficient end flag bit

An embodiment of the present description provides a decoding method and device for a quantization block as well as an electronic device. The method comprises: determining the scanning order of quantization coefficients in the quantization block, initializing the scanning position of the quantization coefficients, and decoding a run-length; adding a subscript value of the current scanning position and the value of the run-length to obtain the scanning position of nonzero coefficients, decoding the values of the nonzero coefficients; judging whether the subscript value of the nonzero coefficients is the maximum subscript value, if yes, ending the decoding of the quantization block; if not, judging whether there are undecoded nonzero coefficients, and if there are no undecoded nonzero coefficients, ending the decoding of the quantization block; if there are undecoded nonzero coefficients, adding 1 to the subscript value of the decoded nonzero coefficients to obtain an updated current scanning position, taking the current scanning position as the initial scanning position to decode the next run-length until the decoding of the quantization block is completed. The technical solution of the present application can improve the decoding efficiency of quantization block.

Decoding method and device for quantization block, video decoding method, and electronic device

A method, system, device and computer-readable storage medium for inverse quantization. The method comprises: determining an initial weighted inverse quantization matrix, wherein, the initial weighted inverse quantization matrix is the same as the quantized block in size; setting some matrix elements in the initial weighted inverse quantization matrix to zero to obtain a weighted inverse quantization matrix, wherein, determining the matrix elements that need to be zeroed according to the size of the quantized block; weighted inverse quantizing the quantized coefficients in the quantized block to generate corresponding inverse transform coefficients, wherein, the value of the matrix element corresponding to the position of the quantized coefficient in the weighted inverse quantization matrix is used as a weight coefficient of the weighted inverse quantization.

Method, system, device, and computer-readable storage medium for inverse quantization

Embodiments of the present application provide a method and a device of inverse tone mapping and an electronic device. The method includes: obtaining one or more low dynamic range images; performing a decomposition operation to the low dynamic range image to acquire a detail layer and a basic layer of the low dynamic range image; restoring the detail layer and the basic layer by using a predetermined first restoration network and a second restoration network to acquire restored detail layer and basic layer; and adjusting the restored detail layer and basic layer by using a predetermined fusion network to acquire an adjusted high dynamic range image. With the technical solution of the present application, the conversion from a low dynamic range image to a high dynamic range image can be more robustly completed without complicated parameter settings.

Method and device of inverse tone mapping and electronic device

An embodiment of the present disclosure discloses an intra-prediction-based encoding and decoding method, apparatus and filter, and belongs to a video coding technical field. The method comprises: acquiring reconstructed pixels adjacent to a to-be-predicted prediction block; determining corresponding reconstructed pixels adjacent to the to-be-predicted prediction block among the reconstructed pixels as reference pixels for filtering, based on prediction modes of the to-be-predicted prediction block; based on the reference pixels, using a filter corresponding to prediction modes of the to-be-predicted prediction block to filter interim prediction values of the to-be-predicted prediction block, to obtain new pixel values of the to-be-predicted prediction block, wherein the interim prediction values are obtained based on other intra prediction methods; performing a CLIP operation on the new pixel values of the to-be-predicted prediction block, to obtain a first prediction value of the to-be-predicted prediction block. By the method provided in the specification, the accuracy of intra prediction may be improved, and coding efficiency may be improved.

Intra-prediction-based encoding and decoding method, apparatus and filter

The present application discloses a cross-media retrieval method based on deep semantic space, which includes a feature generation stage and a semantic space learning stage. In the feature generation stage, a CNN visual feature vector and an LSTM language description vector of an image are generated by simulating a perception process of a person for the image; and topic information about a text is explored by using an LDA topic model, thus extracting an LDA text topic vector. In the semantic space learning phase, a training set image is trained to obtain a four-layer Multi-Sensory Fusion Deep Neural Network, and a training set text is trained to obtain a three-layer text semantic network, respectively. Finally, a test image and a text are respectively mapped to an isomorphic semantic space by using two networks, so as to realize cross-media retrieval. The disclosed method can significantly improve the performance of cross-media retrieval.

Cross-media retrieval method based on deep semantic space

Disclosed is a boundary filtering method for intra prediction, relating to the video encoding technology filed. Whether boundary filtering is performed on an intra prediction block or not is adaptively selected by means of a rate distortion optimization decision; during filtering, a filter coefficient exponentially attenuated relative to distance to boundary is adopted to perform filtering on the first N rows or the first N columns of the intra prediction block by means of an intra prediction block filter, and different filtering strengths are used according to different sizes of the prediction blocks. Therefore, the boundary distortion problem of intra prediction block is solved, the intra prediction precision is improved, and the encoding efficiency of intra prediction block is increased; and the practicability and the robustness of the boundary filtering technology are improved.

Boundary filtering method for intra prediction

A video action detection method based on a convolutional neural network (CNN) is disclosed in the field of computer vision recognition technologies. A temporal-spatial pyramid pooling layer is added to a network structure, which eliminates limitations on input by a network, speeds up training and detection, and improves performance of video action classification and time location. The disclosed convolutional neural network includes a convolutional layer, a common pooling layer, a temporal-spatial pyramid pooling layer and a full connection layer. The outputs of the convolutional neural network include a category classification output layer and a time localization calculation result output layer. The disclosed method does not require down-sampling to obtain video clips of different durations, but instead utilizes direct input of the whole video at once, improving efficiency. Moreover, the network is trained by using video clips of the same frequency without increasing differences within a category, thus reducing the learning burden of the network, achieving faster model convergence and better detection.

Video action detection method based on convolutional neural network

The embodiment of the present specification discloses an image synthesis method, apparatus and device for free-viewpoint. The method includes: correcting a first depth map based on a first color map and the first depth map inputted, wherein the first color map includes a left image and a right image, and the first depth map includes a left image and a right image; forward-projecting the second depth map to a virtual viewpoint position based on the pose of a virtual viewpoint to obtain a third depth map, where the third depth map is a projection map located at the virtual viewpoint position; back-projecting the left and right color maps closest to the virtual viewpoint position in the first color map to the virtual viewpoint position based on the third depth map to obtain a second color map, where the second color map is a projection view of the first color map at the virtual viewpoint position; correcting the second color map by using an optical flow algorithm to obtain a third color map, where the optical flow algorithm implements aligning the left and right images of the second color map; performing weighting calculation and blendering on the third color map to obtain a composite image.

Image synthesis method, apparatus and device for free-viewpoint

The present specification discloses a method, apparatus, and device for video frame interpolation. The method of embodiment of the present specification comprises: acquiring a video frame training sample, wherein the video frame training sample includes an even number of consecutive video frames and a first key frame, and the first key frame is an intermediate frame of the even number of consecutive video frames; constructing a pyramid deep learning model, wherein each level of the pyramid deep learning model being used to generate intermediate frames of different resolutions has a plurality of convolutional neural network layers; inputting the even number of consecutive video frames to the pyramid deep learning model to generate a second key frame; modifying the pyramid deep learning model according to the second key frame and the first key frame to generate a modified pyramid deep learning model; inputting a plurality of video frames to be processed into the modified pyramid deep learning model to generate an intermediate frame of the plurality of video frames. The invention fully exploits the spatio-temporal domain information between multi-frame video frames, and adopts a pyramid refinement strategy to effectively estimate the motion information and the occlusion region, thereby greatly improving the quality of the intermediate frame.

Method, apparatus, and device for video frame interpolation

A method and a device for MCMC framework-based sub-hypergraph matching are provided. Matching of object features is performed by constructing sub-hypergraphs. In a large number of actual images and videos, objects vary constantly, and contain various noise points as well as other interference factors, which makes image object matching and searching very difficult. Perform object feature matching by representing the appearance and positions of objects by sub-hypergraphs allows for faster and more accurate image matching. Furthermore, a sub-hypergraph has several advantages over a graph or a hypergraph: on one hand, a sub-hypergraph has more geometric information (e.g. angle transformation, rotation, scale, etc.) than a graph, and has a lower degree of difficulty and better extensibility than a hypergraph. On the other hand, the disclosed method and device have stronger capabilities to resist interference and good robustness, and are adaptable to more complex settings, especially with outliers.

MCMC framework-based sub-hypergraph matching method and device

The present application provides methods, systems, devices and computer-readable mediums for encoding and decoding transform. A method of the present application comprises: performing transform-encoding on a prediction residual block separately using a plurality of transform matrices, to obtain transformed residual blocks; based on the transformed residual block, determining, by a rate-distortion optimization decision, a transform matrix or a combination of transform matrices matching the residual characteristics of the prediction residual block from the plurality of transform matrices, and determining a transformed residual block to be outputted in a bitstream. Compared with the prior art, the method of the embodiments of the present invention performs a residual transform using transform matrices that are more closely matched with the residual characteristic, thereby improving the expression of the residual signal and improving the coding efficiency of the residual block.

Methods, systems, devices and computer-readable mediums for encoding and decoding transform matrices matching residual characteristics of prediction residual block

The present application provides a method and a device of encoding and decoding based on free viewpoint, and relates to the technical field of video encoding. The method includes: generating a planar splicing image and splice information based on multiple single-viewpoint videos at a server side; generating a planar splicing video based on the planar splicing image; generating camera side information of the planar splicing video based on camera side information existing in the multiple single-viewpoint videos; and encoding the planar splicing video, the splice information and the camera side information of the planar splicing video to generate a planar splicing video bit stream, and decoding a planar splicing video bit stream to acquire a virtual viewpoint according to viewpoint information of a viewer at client side.

Method and device of encoding and decoding based on free viewpoint

Growing community of inventors

Shenzhen, China

Ronggang Wang