Nokia Technologies Oy

Hamed Rezazadegan Tavakoli

Emre Baris Aksu

Miska Matias Hannuksela

Honglei Zhang

Francesco Cricrì

Francesco Cricri

Nannan Zou

Nam Le

Jani Lainema

Goutham Rangu

Iraj Saniee

Joachim Wabnig

Homayun Afrabandpey

Apparatuses, methods, and computer programs for compressing a neural network are disclosed. An apparatus includes at least one processor; and at least one non-transitory memory including computer program code, the memory and the computer program code configured to, with the at least one processor, cause the apparatus to: receive information from a second device, where the information comprises at least one parameter configured to be used for compression of a neural network, where the at least one parameter is in regard to at least one first aspect or task of the neural network; and compress the neural network, where the neural network is compressed based, at least partially, upon the at least one parameter received from the second device. The apparatus may also receive a compressed neural network from the second device, and further compress the compressed neural network based on the information.

High-level syntax for priority signaling in neural network compression

The embodiments relate to a method for encoding two or more tensors. The method comprises processing the two or more tensors having respective dimensions so that the dimensions of said two or more sensors have the same number (); identifying which axis of each individual tensor is swappable to result in concatenable tensors around an axis of concatenation (); reshaping the tensors so that the dimensions are modified based on the swapped axis (); concatenating the tensors around the axis of concatenation to result in concatenated tensor (); compressing the concatenated tensor (); generating syntax structures for carrying concatenation and axis swapping information (); and generating a bitstream by combining the syntax structures and the compressed concatenated tensor (). The embodiments also relate to a method for decoding, and to apparatuses for implementing the methods.

Method, an apparatus and a computer program product for neural network compression

The embodiments relate to an apparatus and a method for encoding and decoding. The encoding method comprises receiving () visual data; converting () the visual data into a latent representation comprising channels; quantizing () the latent representation; processing () the channels of the quantized latent representation (x) according to a predefined channel dependency mode by generating () a mixed representation (y) comprising predicted values and original values from the quantized latent representation; generating () a mask representation (m) indicating which elements in the mixed representation are with original values; estimating () parameters (q) for a probability distribution function or a probability distribution of elements with the predicted values; encoding () elements with the predicted values using the original values from the quantized latent representation and the estimated probability distribution function for these elements into a bitstream; updating () a mixed representation (y) with the original values from the quantized latent representation (x) and the mask representation (m) accordingly; repeating () the processing for all elements.

Method, an apparatus and a computer program product for video encoding and video decoding

An apparatus comprising: at least one processor; and at least one non-transitory memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receive a signal, the signal comprising a sparse signal; perform residual coding on the signal; perform predictive coding on the signal; determine a residual, the residual comprising a residual of the signal and a base signal or a residual of an approximation and the base signal, the approximation being an approximation of the signal; and determine whether to transmit the residual or the signal over a communication channel.

Predictive and residual coding of sparse signals for weight update compression

In example embodiments, an apparatus, a method, and a computer program product are provided. The apparatus comprises at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform determine a processing order of building blocks to encode or decode a media item; and determine a number of processing steps required to encode or decode the media item; wherein the processing order of building blocks and the number of processing steps are determined based on a content of the media item by using a guided probability model based on a neural network.

Guided probability model for compressed representation of neural networks

Various embodiments provide an apparatus, a method, and a computer program product. 1. An apparatus incudes at least one processor; and at least one non-transitory memory includes computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to: perform an overfitting operation, at an encoder side, to obtain an overfitted probability model, wherein overfitting comprises one or more training operations applied to a probability model, wherein one or more parameters of the probability model are trained; use the overfitted probability model to provide probability estimates to a lossless codec or a substantially lossless codec for encoding data or a portion of the data; and signal information to a decoder on whether to perform the overfitting operation at the decoder side.

Apparatus method and computer program product for probability model overfitting

In example embodiments, an apparatus, a method, and a computer program product are provided. An example apparatus include processing circuitry; and at least one memory including computer program code, the at least one memory and the computer program code configured to, with the processing circuitry, cause the apparatus at least to: overfit a neural network on each media item, from a batch of media items, for a number of iterations to obtain an overfitted neural network model for the each media item; evaluate the overfitted neural network model on the each media item to obtain evaluation errors; and update parameters of the neural network to be based on the evaluation errors.

Apparatus, method and computer program product for optimizing parameters of a compressed representation of a neural network

An apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to: estimate an importance of parameters of a neural network based on a graph diffusion process over at least one layer of the neural network; determine the parameters of the neural network that are suitable for pruning or sparsification; remove neurons of the neural network to prune or sparsify the neural network; and provide at least one syntax element for signaling the pruned or sparsified neural network over a communication channel, wherein the at least one syntax element comprises at least one neural network representation syntax element.

Graph diffusion for structured pruning of neural networks

A method is provided for defining a metadata box of a neural network representation (NNR) item data, wherein the NNR item data comprises an NNR bitstream; and defining an association between the NNR item data and an NNR configuration by using a configuration item property, wherein the NNR configuration item property comprises information about stored NNR item data. Corresponding apparatuses and computer program products are also provided.

Carriage and signaling of neural network representations

Example embodiments provide a system for training a data coding pipeline including a feature extractor neural network, an encoder neural network, and a decoder neural network configured to reconstruct input data based on encoded features. A plurality of losses corresponding to different tasks may be determined for the coding pipeline. Tasks may be performed based on an output of the coding pipeline. A weight update may be determined for at least a subset of the coding pipeline based on the plurality of losses. The weight update may be configured to reduce a number of iterations for fine-tuning the coding pipeline for one of the tasks. This enables faster adaptation of the coding pipeline for one of the tasks after deployment of the coding pipeline. Apparatuses, methods, and computer programs are disclosed. Apparatuses, methods, and computer programs are disclosed.

Training a data coding system comprising a feature extractor neural network

A method, an apparatus, and a computer program product are provided. An example method includes defining an enhancement message comprising at least one of the following: an identifying number for identifying a post-processing filter; a mode identity (idc) field used of indicating association of a post-processing filter with the identifying number; a flag for specifying the enhancement message being used for a current layer; and the payload byte comprising a bitstream; and using the enhancement message for at least one of specifying a neural network that is used as a post-processing filter or cancelling a use of a previous post-processing filter with the same identifying number.

High-level syntax for signaling neural networks within a media bitstream

An example method is provided to include receiving a media bitstream comprising one or more media units and a first enhancement information message, wherein the first enhancement information message comprises at least two independently parsable structures, a first independently parsable structure comprising information about at least one purpose of one or more neural networks (NNs) to be applied to the one or more media units, and a second independently parsable structure comprising or identifying one or more neural networks; decoding the one or more media units; and using the one or more neural networks to enhance or filter one or more frames of the decoded the one or more media units, based on the at least one purpose. An example method includes. Corresponding apparatuses and computer program products are also provided.

In example embodiments, an apparatus, a method, and a computer program product are provided. The apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: encode or decode a high-level bitstream syntax for at least one neural network; wherein the high-level bitstream syntax comprises at least one information unit, wherein the at least one information unit comprises syntax definitions for the at least one neural network or a portion of the at least one neural network; and wherein a serialized bitstream comprises one or more of the at least one information units.

High level syntax and carriage for compressed representation of neural networks

Data may be encoded to minimize distortion after decoding, but the quality required for presentation of the decoded data to a machine and the quality required for presentation to a human may be different. To accommodate different quality requirements, video data may be encoded to produce a first set of encoded data and a second set of encoded data, where the first set may be decoded for use by one of a machine consumer or a human consumer, and a combination of the first set and the second set may be decoded for use by the other of a machine consumer or a human consumer. The first and second set may be produced with a neural encoder and a neural decoder, and/or may be produced with the use of prediction and transform neural network modules. A human-targeted structure and a machine-targeted structure may produce the sets of encoded data.

Cascaded prediction-transform approach for mixed machine-human targeted video coding

An apparatus includes circuitry configured to: partition an input tensor into one or more block tensors; partition at least one of the block tensors into one or more continuation bands, the one or more continuation bands being associated with a caching counter having a value; store the one or more continuation bands in a cache managed using a cache manager; retrieve, prior to a convolution or pooling operation on a current block tensor, the one or more continuation bands of a previous block tensor from the cache that are adjacent to a current block tensor; concatenate the retrieved continuation bands with the current block tensor; apply the convolution or pooling operation on the current block tensor after the concatenation; decrease the respective caching counter value of the retrieved continuation bands; and clear the continuation bands from the cache when its respective caching counter reaches a value of zero.

Growing community of inventors

Espoo, Finland

Hamed Rezazadegan Tavakoli