Samsung Electronics Co., Ltd.

Dolby Laboratories Licensing Corporation

Georgios Georgiadis

Weiran Deng

Ning Xu

Kimball Darr Thurston, Iii

Joseph H Hassoun

Ali Shafiee Ardestani

James E Crenshaw

Hamzah Ahmed Ali Abdelaziz

Hui Chen

Alexander Partin

Jun Fang

David Philip Lloyd Thorsley

Kimball Darr Thurston Iii

A technique to prune weights of a neural network using an analytic threshold function h(w) provides a neural network having weights that have been optimally pruned. The neural network includes a plurality of layers in which each layer includes a set of weights w associated with the layer that enhance a speed performance of the neural network, an accuracy of the neural network, or a combination thereof. Each set of weights is based on a cost function C that has been minimized by back-propagating an output of the neural network in response to input training data. The cost function C is also minimized based on a derivative of the cost function C with respect to a first parameter of the analytic threshold function h(w) and on a derivative of the cost function C with respect to a second parameter of the analytic threshold function h(w).

Self-pruning neural networks for weight parameter reduction

In some embodiments, a method of quantizing an artificial neural network includes dividing a quantization range for a tensor of the artificial neural network into a first region and a second region, and quantizing values of the tensor in the first region separately from values of the tensor in the second region. In some embodiments, linear or nonlinear quantization are applied to values of the tensor in the first region and the second region. In some embodiments, the method includes locating a breakpoint between the first region and the second region by substantially minimizing an expected quantization error over at least a portion of the quantization range. In some embodiments, the expected quantization error is minimized by solving analytically and/or searching numerically.

Piecewise quantization for neural networks

A system and a method provide compression and decompression of weights of a layer of a neural network. For compression, the values of the weights are pruned and the weights of a layer are configured as a tensor having a tensor size of H&#xd7;W&#xd7;C in which H represents a height of the tensor, W represents a width of the tensor, and C represents a number of channels of the tensor. The tensor is formatted into at least one block of values. Each block is encoded independently from other blocks of the tensor using at least one lossless compression mode. For decoding, each block is decoded independently from other blocks using at least one decompression mode corresponding to the at least one compression mode used to compress the block; and deformatted into a tensor having the size of H&#xd7;W&#xd7;C.

Lossless compression of neural network weights

A system and a method generate a neural network that includes at least one layer having weights and output feature maps that have been jointly pruned and quantized. The weights of the layer are pruned using an analytic threshold function. Each weight remaining after pruning is quantized based on a weighted average of a quantization and dequantization of the weight for all quantization levels to form quantized weights for the layer. Output feature maps of the layer are generated based on the quantized weights of the layer. Each output feature map of the layer is quantized based on a weighted average of a quantization and dequantization of the output feature map for all quantization levels. Parameters of the analytic threshold function, the weighted average of all quantization levels of the weights and the weighted average of each output feature map of the layer are updated using a cost function.

Jointly pruning and quantizing deep neural networks

A system and method for pruning. A neural network includes a plurality of long short-term memory cells, each of which includes an input having a weight matrix W, an input gate having a weight matrix W, a forget gate having a weight matrix W, and an output gate having a weight matrix W. In some embodiments, after initial training, one or more of the weight matrices W, W, and Ware pruned, and the weight matrix Wis left unchanged. The neural network is then retrained, the pruned weights being constrained to remain zero during retraining.

Accelerating long short-term memory networks via selective pruning

Color trim data is used in an approximation function to approximate one or more non-linear transformations of image data in an image processing pipeline. The color trim data is derived in one embodiment through a back projection on a colorist system, and the color trim data is used at the time of rendering an image on a display management system.

Color image modification with approximation function

Methods and systems for enhancing motion estimation are disclosed for stereoscopic video sequences. The motion vector parameters and disparity values for consecutive frames may be calculated to estimate motion in a coherent manner between two channels, such as a left channel and a right channel in stereoscopic images. Occlusion handling is also applied to improve the estimation.

Growing community of inventors

Burbank, CA, United States of America

Georgios Georgiadis