The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Aug. 04, 2026
Filed:
Oct. 05, 2022
Google Llc, Mountain View, CA (US);
Venkata S. Bhojanapalli, New York, NY (US);
Andreas Veit, New York, NY (US);
Ayan Chakrabarti, New York, NY (US);
Frederick Liu, Bellevue, WA (US);
Himanshu Jain, New York, NY (US);
Michal Lukasik, New York, NY (US);
Sanjiv Kumar, Jericho, NY (US);
Yin-Wen Chang, New York, NY (US);
GOOGLE LLC, Mountain View, CA (US);
Abstract
Provided are systems and methods that improve the computational efficiency of Transformers or other attention-based neural networks or machine learning models by re-using a number of attention scores between layers and/or heads of the model. To reduce the computational cost of self-attention-based models while achieving comparable or even superior results, example aspects of the present disclosure propose a novel architecture that reuses attention scores computed in one layer in one or multiple subsequent layers.