The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
May. 13, 2025
Filed:
Jun. 15, 2017
Nvidia Corporation, Santa Clara, CA (US);
Boris Ginsburg, Santa Clara, CA (US);
Sergei Nikolaev, Santa Clara, CA (US);
Ahmad Kiswani, Santa Clara, CA (US);
Hao Wu, Hanzhou, CN;
Amir Gholaminejad, Santa Clara, CA (US);
Slawomir Kierat, Mountain View, CA (US);
Michael Houston, Saratoga, CA (US);
Alex Fit-Florea, Belmont, CA (US);
NVIDIA Corporation, Santa Clara, CA (US);
Abstract
Aspects of the present invention are directed to computer-implemented techniques for improving the training of artificial neural networks using a reduced precision (e.g., float16) data format. Embodiments of the present invention rescale tensor values prior to performing matrix operations (such as matrix multiplication or matrix addition) to prevent overflow and underflow. To preserve accuracy throughout the performance of the matrix operations, the scale factors are defined using a novel data format to represent tensors, wherein a matrix is represented by the tuple X, where X=(a, v[.]), wherein a is a float scale factor and v[.] are scaled values stored in the float16 format. The value of any element X[i] according to this data format would be equal to a*v[i].