The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Nov. 02, 2021
Filed:
Dec. 15, 2017
International Business Machines Corporation, Armonk, NY (US);
Tung D. Le, Ichikawa, JP;
Haruki Imai, Yokohama, JP;
Taro Sekiyama, Urayasu, JP;
Yasushi Negishi, Tokyo, JP;
INTERNATIONAL BUSINESS MACHINES CORPORATION, Armonk, NY (US);
Abstract
A computer-implemented method, computer program product, and computer processing system are provided for accelerating neural network data parallel training in multiple graphics processing units (GPUs) using at least one central processing unit (CPU). The method includes forming a set of chunks. Each of the chunks includes a respective group of neural network layers other than a last layer. The method further includes performing one or more chunk-wise synchronization operations during a backward phase of the neural network data parallel training, by each of the multiple GPUs and the at least one CPU.