Qualcomm Incorporated

Tijmen Pieter Frederik Blankevoort

Markus Nagel

Marinus Willem Van Baalen

Christos Louizos

Babak Ehteshami Bejnordi

Matthias Reisser

Jilei Hou

Ying Wang

Chirag Sureshbhai Patel

Erich Plondke

Kambiz Azarian Yazdi

Max Welling

Amirhossein Habibian

Jin Won Lee

Davide Abati

Yadong Lu

Rana Ali Amjad

Yash Sanjay Bhalgat

Jonathan Dewitt Wolfe

Jakub Mikolaj Tomczak

Andrey Kuzmin

Andrii Skliar

Yawei Li

Bert Moons

Eric Wayne Mahurin

Brian Kahne

A processor-implemented method includes bit shifting a binary representation of a neural network parameter. The neural network parameter has fewer bits, b, than a number of hardware bits, B, supported by hardware that processes the neural network parameter. The bit shifting effectively multiplies the neural network parameter by 2. The method also includes dividing a quantization scale by 2to obtain an updated quantization scale. The method further includes quantizing the bit shifted binary representation with the updated quantization scale to obtain a value for the neural network parameter.

Simulated low bit-width quantization using bit shifted neural network parameters

Certain aspects of the present disclosure provide techniques and apparatus for processing a video stream using a machine learning model. An example method generally includes generating a first group of tokens from a first frame of the video stream and a second group of tokens from a second frame of the video stream. A first set of tokens associated with features to be reused from the first frame and a second set of tokens associated with features to be computed from the second frame are identified based on a comparison of tokens from the first group of tokens to corresponding tokens in the second group of tokens. A feature output is generated for portions of the second frame corresponding to the second set of tokens. Features associated with the first set of tokens are combined with the generated feature output into a representation of the second frame.

Processing video content using gated transformer neural networks

A method for an artificial neural network includes receiving an input. A quantization threshold is determined based on the input, or a characteristic or type of the input. Neural network values, such as weights or activations, of one or more layers of the artificial neural network are quantized according to the quantization threshold. The artificial neural network generates an output based on the quantized neural network values.

Variable quantization for neural networks

Various embodiments include methods and devices for joint mixed-precision quantization and structured pruning. Embodiments may include determining whether a plurality of gates of quantization and pruning gates are selected for combination, and in response to determining that the plurality of gates are selected for combination, iteratively for each successive gate of the plurality of gates selected for combination quantizing a residual error of a quantized tensor to a scale of a next bit-width producing a residual error quantized tensor in which the next bit-width increases for each successive iteration, and adding the quantized tensor and the residual error quantized tensor producing a next quantized tensor in which the next quantized tensor has the next bit-width, and in which the next quantized tensor is the quantized tensor for a successive iteration.

Bayesian bits joint mixed-precision quantization and structured pruning using decomposed quantization and Bayesian gates

Various aspects provide methods for learning, such as continual learning, that support task-incremental learning using a multi-head classification architecture. Various aspects may enable conditional computing to support multi-head classification. Various aspects provide methods for learning, such as continual learning, that support class-incremental learning using a single-head classification architecture. Various aspects may enable conditional computing to support single-head classification by predicting the task associated with a given test input and selecting an associated classification head based at least in part on the task prediction.

Conditional computation for continual learning

Various embodiments include methods and neural network computing devices implementing the methods for performing quantization in neural networks. Various embodiments may include equalizing ranges of weight tensors or output channel weights within a first layer of the neural network by scaling each of the output channel weights of the first layer by a corresponding scaling factor, and scaling each of a second adjacent layer's corresponding input channel weights by applying an inverse of the corresponding scaling factor to the input channel weights. The corresponding scaling factor may be determined using a black-box optimizer on a quantization error metric or based on heuristics, equalization of dynamic ranges, equalization of range extrema (minima or maxima), differential learning using straight through estimator (STE) methods and a local or global loss, or using an error metric for the quantization error and a black-box optimizer that minimizes the error metric with respect to the scaling.

Systems and methods of cross layer rescaling for improved quantization performance

A method for compressing a deep neural network includes determining a pruning ratio for a channel and a mixed-precision quantization bit-width based on an operational budget of a device implementing the deep neural network. The method further includes quantizing a weight parameter of the deep neural network and/or an activation parameter of the deep neural network based on the quantization bit-width. The method also includes pruning the channel of the deep neural network based on the pruning ratio.

Joint pruning and quantization scheme for deep neural networks

A method for pruning weights of an artificial neural network based on a learned threshold includes determining a pruning threshold for pruning a first set of pre-trained weights of multiple pre-trained weights based on a function of a classification loss and a regularization loss. Weights are pruned from the first set of pre-trained weights when a first value of the weight is less than the pruning threshold. A second set of pre-trained weights of the multiple pre-trained weights is fine-tuned or adjusted in response to a second value of each pre-trained weight in the second set of pre-trained weights being greater than the pruning threshold.

Learned threshold pruning for deep neural networks

Various embodiments include methods and neural network computing devices implementing the methods, for generating an approximation neural network. Various embodiments may include performing approximation operations on a weights tensor associated with a layer of a neural network to generate an approximation weights tensor, determining an expected output error of the layer in the neural network due to the approximation weights tensor, subtracting the expected output error from a bias parameter of the layer to determine an adjusted bias parameter and substituting the adjusted bias parameter for the bias parameter in the layer. Such operations may be performed for one or more layers in a neural network to produce an approximation version of the neural network for execution on a resource limited processor.

Analytic and empirical correction of biased error introduced by approximation methods

A method for quantizing a neural network includes modeling noise of parameters of the neural network. The method also includes assigning grid values to each realization of the parameters according to a concrete distribution that depends on a local fixed-point quantization grid and the modeled noise and. The method further includes computing a fixed-point value representing parameters of a hard fixed-point quantized neural network.

Growing community of inventors

Amsterdam, Netherlands

Tijmen Pieter Frederik Blankevoort