The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Aug. 30, 2022
Filed:
Aug. 16, 2021
Moffett Technologies Co., Limited, Central, HK;
Enxu Yan, Los Altos, CA (US);
MOFFETT INTERNATIONAL CO., LIMITED, Hong Kong, HK;
Abstract
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing neural network training, are described. The method may include: during a forward propagation at a current layer of a neural network, generating, based on a sparse input tensor and a sparse weight tensor of the current layer, a dense output tensor, and sparsifying the dense output tensor to obtain a sparse output tensor; during a backward propagation at the current layer of the neural network: determining a first sparse derivative tensor based on the sparse output tensor, obtaining a dense derivative tensor based on the first sparse derivative tensor and the sparse weight tensor of the current layer, and sparsifying the dense derivative tensor to obtain a second sparse derivative tensor; and training weight tensors of the neural network based on the first sparse derivative tensor and the second sparse derivative tensor.