Mediatek Corporation

Neuchips Corporation

Ping Chao

Yung-Chang Chang

Chi-Cheng Ju

Chih-Ming Wang

Youn-Long Lin

Chao-Yang Kao

Huei-Min Lin

Han-Liang Chou

Tsu-Ming Liu

Tzu-Der Chuang

Ching-Yeh Chen

Yu-Wen Huang

Chun-Chia Chen

Yu-Chen Sun

Tung-Hsing Wu

Chia-Yun Cheng

Ting-An Lin

Wan-Yu Chen

Hsin-Hao Chung

Chuang-Chi Chiou

Ping-yao Wang

Wei-Gen Wu

Chung Hsien Lee

Kung-Tsun Yang

A harmonic densely connecting method includes an input step, a plurality of layer operation steps and an output step. The input step is for storing an original input tensor of the block into a memory. Each of the layer operation steps includes a layer-input tensor concatenating step and a convolution operation step. The layer-input tensor concatenating step is for selecting at least one layer-input element tensor of a layer-input set from the memory according to an input connection rule. When a number of the at least one layer-input element tensor is greater than 1, concatenating all of the layer-input element tensors and producing a layer-input tensor. The convolution operation step is for calculating a convolution operation to produce at least one result tensor and then storing the at least one result tensor into the memory. The output step is for outputting a block output.

Harmonic densely connecting method of block of convolutional neural network model and system thereof, and non-transitory tangible computer readable recording medium

A feature map caching method of a convolutional neural network includes a connection analyzing step and a plurality of layer operation steps. The connection analyzing step is for analyzing a network to establish a convolutional neural network connection list. The convolutional neural network connection list includes a plurality of tensors and a plurality of layer operation coefficients. Each of the layer operation coefficients includes a step index, at least one input operand label and an output operand label. The step index as a processing order for the layer operation step. At least one of the layer operation steps is for flushing at least one of the tensors in a cache according to a distance between the at least one of the layer operation steps and a future layer operation step of the layer operation steps. The distance is calculated according to the convolutional neural network connection list.

Feature map caching method of convolutional neural network and system thereof

A memory-adaptive processing method for a convolutional neural network includes a feature map counting step, a size relation counting step and a convolution calculating step. The feature map counting step is for counting a number of a plurality of input channels of a plurality of input feature maps, an input feature map tile size, a number of a plurality of output channels of a plurality of output feature maps and an output feature map tile size for a convolutional layer operation. The size relation counting step is for obtaining a cache free space size in a feature map cache and counting a size relation. The convolution calculating step is for performing the convolutional layer operation with the input feature maps to produce the output feature maps according to a memory-adaptive processing technique, and the memory-adaptive processing technique includes a dividing step and an output-group-first processing step.

Memory-adaptive processing method for convolutional neural network

A method and a circuit for adaptive loop filtering in a video coding system are described. The method can include receiving a block of samples generated from a previous-stage filter circuit in a filter pipeline, the block of samples being one of multiple blocks included in a current picture, performing, in parallel, adaptive loop filter (ALF) processing for multiple target samples in the block of samples, while the previous-stage filter circuit is simultaneously processing another block in the current picture, storing, in a buffer, first samples each having a filter input area defined by a filter shape that includes at least one sample which has not been received, and storing, in the buffer, second samples included in the filter input areas of the first samples.

High efficiency adaptive loop filter processing for video coding

A memory-adaptive processing method for a convolutional neural network includes a feature map counting step, a size relation counting step and a convolution calculating step. The feature map counting step is for counting a plurality of input channels of an input feature map tile and a plurality of output channels of an output feature map tile for a convolutional layer operation of the convolutional neural network. The size relation counting step is for obtaining a cache free space size in a feature map cache and counting a size relation among a total input size, a total output size and the cache free space size of the feature map cache. The convolution calculating step is for performing the convolutional layer operation according to a memory-adaptive processing technique.

Memory-adaptive processing method for convolutional neural network and system thereof

Aspects of the disclosure provide a method for merging compressed access units according to compression rates and/or positions of the respective compressed access units. The method can include receiving a sequence of compressed access units corresponding to a sequence of raw access units partitioned from an image or a video frame and corresponding to a sequence of memory spaces in a frame buffer, determining a merged access unit including at least two consecutive compressed access units based on compression rates and/or positions of the sequence of compressed access units. The merged access unit is to be stored in the frame buffer with a reduced gap between the at least two consecutive compressed access units compared with storing the at least two consecutive compressed access units in corresponding memory spaces in the sequence of memory spaces.

Merged access units in frame buffer compression

A method and apparatus of sharing an on-chip buffer or cache memory for a video coding system using coding modes including Inter prediction mode or Intra Block Copy (IntraBC) mode are disclosed. At least partial pre-deblocking reconstructed video data of a current picture is stored in an on-chip buffer or cache memory. If the current block is coded using IntraBC mode, the pre-deblocking reconstructed video data of the current picture stored in the on-chip buffer or cache memory are used to derive IntraBC prediction for the current block. In some embodiments, if the current block is coded using Inter prediction mode, Inter reference video data from the previous picture stored in the on-chip buffer or cache memory are used to derive Inter prediction for the current block. In another embodiment, the motion compensation/motion estimation unit is shared by the two modes.

Method and apparatus for resource sharing between intra block copy mode and inter prediction mode in video coding systems

A method for storing data of an image frame into a frame buffer includes at least the following steps: dividing the image frame into a plurality of access units, each having at least one encoding unit, wherein each encoding unit is a unit for data compression; and performing the data compression upon each encoding unit of the image frame, and generating an output bitstream to the frame buffer based on a data compression result of the encoding unit. A processing result of each access unit includes each output bitstream of the at least one encoding unit included in the access unit; a plurality of processing results of the access units are stored in a plurality of storage spaces allocated in the frame buffer, respectively; and a size of each of the storage spaces is equal to a size of a corresponding access unit.

Method and apparatus for accessing compressed data and/or uncompressed data of image frame in frame buffer

A method and apparatus for loop filter processing of reconstructed video data for a video coding system are disclosed. The system receives reconstructed video data for an image unit. The loop filter processing is applied to reconstructed pixels above a deblocking boundary of the current CTU. In order to reduce line buffer requirement and/or to reduce loop filter switching for image units, the sample adaptive offset (SAO) parameter boundary and spatial-loop-filter restricted boundary for the luma and chroma components are determined by global consideration. In one embodiment, the SAO parameter boundary and the spatial-loop-filter restricted boundary are aligned for the luma and chroma components respectively. In another embodiment, the SAO parameter boundary and the spatial-loop-filter restricted boundary for the luma and chroma components are all aligned.

Method and apparatus of loop filters for efficient hardware implementation

A data arrangement method includes following steps: obtaining pixel data of a plurality of first N-bit pixels of a picture; and storing the obtained pixel data of the first N-bit pixels in a plurality of M-bit storage units of a first buffer according to a block-based scan order of the picture. The picture includes a plurality of data blocks, and the block-based scan order includes a raster-scan order for the data blocks. At least one of the M-bit storage units is filled with part of the obtained pixel data of the first N-bit pixels, M and N are positive integers, M is not divisible by N, and the first N-bit pixels include at least one pixel divided into a first part stored in one of the M-bit storage units in the first buffer and a second part stored in another of the M-bit storage units in the first buffer.

Method and apparatus for arranging pixels of picture in storage units each having storage size not divisible by pixel size

A method and apparatus for applying DF processing and SAO processing to reconstructed video data are disclosed. The DF processing is applied to a current access element of reconstructed video data to generate DF output data and the deblocking status is determined while applying the DF processing. Status-dependent SAO processing is applied to one or more pixels of the DF output data according to the deblocking status. The status-dependent SAO processing comprises SAO processing, partial SAO processing, and no SAO processing. The SAO starting time for SAO processing is between the DF-output starting time and ending time for the current block. The DF starting time of a next block can be earlier than the SAO ending time of the current block by a period oft, where t is smaller than time difference between the DF-output starting time and the DF starting time of the next block.

Method and apparatus for video processing incorporating deblocking and sample adaptive offset

A video processing system includes a data buffer and a storage controller. The data buffer is shared between a plurality of in-loop filters, wherein not all of the in-loop filters comply with a same video standard. The storage controller controls data access of the data buffer, wherein for each in-loop filter granted to access the data buffer, the data buffer stores a partial data of a picture processed by the in-loop filter. Another video processing system includes a storage device and a storage controller. The storage controller adaptively determines a size of a storage space according to a tile partition setting of a picture to be processed by an in-loop filter, and controls the storage device to allocate the storage space to serve as a data buffer for storing data of the in-loop filter.

Video processing system with shared/configurable in-loop filter data buffer architecture and related video processing method thereof

A method and apparatus for SAO (sample adaptive offset) processing in a video decoder are disclosed. Embodiments according to the present invention reduce the required line buffer associated with the SAO processing. According to one embodiment, tri-level comparison results for one deblocked pixel row or column of the image unit are determined according to SAO type of the deblocked pixel row or column. The tri-level comparison results are stored in a buffer and the tri-level comparison results are read back from the buffer for SAO processing of the subsequent row or column from a subsequent image unit. The comparison results correspond to 'larger', 'equal' and &#x201c;smaller&#x201d; states. The comparison results can be stored more efficiently.

Growing community of inventors

Taipei, Taiwan

Ping Chao