The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Feb. 25, 2025
Filed:
May. 17, 2021
Marziehsadat Tahaei, Montreal, CA;
Ali Ghodsi, Waterloo, CA;
Mehdi Rezagholizadeh, Montreal, CA;
Vahid Partovi Nia, Montreal, CA;
Marziehsadat Tahaei, Montreal, CA;
Ali Ghodsi, Waterloo, CA;
Mehdi Rezagholizadeh, Montreal, CA;
Vahid Partovi Nia, Montreal, CA;
HUAWEI TECHNOLOGIES CO., LTD., Shenzhen, CN;
Abstract
Methods and systems for compressing a neural network (NN) which performs an inference task and for performing computations of a Kronecker layer of a Kronecker NN are described. Data samples are obtained from a training dataset. The input data of the data samples are inputted into a trained NN to generate NN predictions for the input data. Further, the input data are inputted into a Kronecker NN to generate Kronecker NN predictions for the input data. Two losses are computed: a knowledge distillation loss, based on outputs generated by a layer of the NN and a corresponding Kronecker layer of the Kronecker NN and a loss for Kronecker layer, based on the Kronecker NN predictions and ground-truth labels of the data samples. The two losses are combined into a total loss, which is propagated through the Kronecker NN to adjust values of learnable parameters of the Kronecker NN.