Huawei Technologies Co., Limited

Applied Materials, Inc.

Wei Wang

Shan Liu

Jianle Chen

Wei Jiang

Haoping Yu

Jiafeng Zhu

Bharath Swaminathan

Zhan Ma

Meng Xu

Xian Wang

Jie Shen

Wei Wei

The disclosure relates to technology for generating a compressed neural network. A weight tensor is received from a neural network to be compressed, and it is reordered to be compressed to have an inner two-dimensional (2D) shape and a 2D sparse bitmap. A layered structure is generated that represents the reordered weight tensor, and the reordered weight tensor is divided into a group of coefficients (GOCs). An encoding mode is selected to generate a quantized reordered weight tensor using one of a codebook or direct quantization, and a column swapped quantized reordered weigh tensor is generated. A compressed neural network is formed by encoding and the compressed representation of the neural network is transmitted to a target system for decompression.

Generating a compressed representation of a neural network with proficient inference speed and power consumption

A first aspect relates to a computer-implemented method for performing model compression. The method includes compressing a machine learning (ML) network model comprising a multiple layer structure to produce a compressed ML network model. The compressed ML network model maintains the multiple layer structure of the ML network model. The method generates a model file for the compressed ML network model. The model file includes the compressed ML network model and decoding information for enabling the ML network model to be decompressed and executed layer-by-layer.

Machine learning network model compression

Growing community of inventors

Santa Clara, CA, United States of America

Wei Wang