The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Nov. 26, 2019
Filed:
Dec. 29, 2016
Initializing and learning rate adjustment for rectifier linear unit based artificial neural networks
Amazon Technologies, Inc., Seattle, WA (US);
Ayyavu Madhavaraj, Bangalore, IN;
Sri Venkata Surya Siva Rama Krishna Garimella, Bangalore, IN;
Amazon Technologies, Inc., Seattle, WA (US);
Abstract
A data processing technique uses an Artificial Neural Network (ANN) with Rectifier Linear Units (ReLU) to yield improve accuracy in a runtime task, for example, in processing audio-based data acquired by a speech-enabled device. The technique includes a first aspect that relates to initialization of the ANN weights to initially yield a high fraction of positive outputs from the ReLU. These weights are then modified using an iterative procedure in which the weights are incrementally updated. A second aspect relates to controlling the size of the incremental updates (a 'learning rate') during the iterations of training according to a variance of the weights at each layer.