Moffett Technologies Co., Limited

Moffett International Co., Limited

Wei Wang

Enxu Yan

Zhibin Xiao

Yong Lu

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for balanced-weight sparse convolution processing. An exemplary method comprises: obtaining an input tensor and a plurality of filters at a layer within a neural network; segmenting the input tensor into a plurality of sub-tensors; dividing a channel dimension of each of the plurality of filters into a plurality of channel groups; pruning each of the plurality of filters so that each of the plurality of channel groups of each filter comprises a same number of non-zero weights; segmenting each of the plurality of filters into a plurality of the sub-filters according to the plurality of channel groups; and assigning the plurality of sub-tensors and the plurality of sub-filters to a plurality of processors for parallel convolution processing.

Method and system for balanced-weight sparse convolution processing

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for hierarchical weight-sparse convolution processing are described. An exemplary method comprises: obtaining an input tensor and a filter at a convolution layer of a neural network; segmenting the filter into a plurality of sub-filters; generating a hierarchical bit representation of the filter representing a plurality of non-zero weights in the filter, wherein the hierarchical bit representation comprises a first layer, the first layer comprising a plurality of bits respectively corresponding to the plurality of sub-filters in the filter, each of the plurality of bits indicating whether the corresponding sub-filter includes at least one non-zero weight; and performing multiply-and-accumulate (MAC) operations based on the hierarchical bit representation of the filter and the input tensor.

Method and system for hierarchical weight-sparse convolution processing

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for balanced-weight sparse convolution processing. An exemplary method comprises: obtaining an input tensor and a plurality of filters; segmenting the input tensor into a plurality of sub-tensors according to a number of available processors; segmenting each of the plurality of filters into one or more sub-filters to obtain a plurality of the sub-filters; respectively assigning the plurality of sub-tensors and sub-filters to the available processors; for each of the available processors, iterating each of the index-value pairs and identifying, based on the index-value pair, a corresponding input value in the assigned sub-tensor to perform a multiply-and-accumulate (MAC) operation to generate a partial sum; reassigning the plurality of sub-filters to the available processors by rotating the plurality of sub-filters among the plurality of processors; and accumulating a plurality of the partial sums to obtain an output tensor.

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for hierarchical weight-sparse convolution processing are described. An exemplary method comprises: obtaining an input tensor and a plurality of filters at a convolution layer of a neural network; segmenting the input tensor into a plurality of sub-tensors and assigning the plurality of sub-tensors to a plurality of processors; generating, for each of the plurality of filters, a hierarchical bit representation of a plurality of non-zero weights in the filter, wherein the hierarchical bit representation comprises a plurality of bits indicating whether a sub-filter has at least one non-zero weight, and a plurality of key-value pairs corresponding to the plurality of non-zero weights in the filter; identifying, based on the hierarchical bit representation, one or more of the plurality of non-zero weights and corresponding input values from the assigned sub-tensor to perform multiply-and-accumulate (MAC) operations.

Systems, methods and computer-readable medium for (i) accelerating the inference speed of a deep neural network (DNN), and (ii) compressing the vector representations produced by the DNN out of a variety of input data, such as image, audio, video and text. A method embodiment takes as inputs a neural network architecture and a task-dependent loss function, measuring how well a neural network performs on a training data set, and outputs a deep neural network with sparse neuron activations. The invented procedure augments an existing training objective function of a DNN with regularization terms that encourage sparse activation of neurons, and compresses the DNN by solving the optimization problem with a variety of algorithms. The present disclosure also shows how to utilize the sparsity of activations during the inference of DNNs so the number of arithmetic operations can be reduced proportionately, and how to use the sparse representations produced by the DNNs to build an efficient search engine.

Growing community of inventors

Los Altos, CA, United States of America

Wei Wang