The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jan. 06, 2026
Filed:
Feb. 04, 2022
Google Llc, Mountain View, CA (US);
Zhe Zhao, Palo Alto, CA (US);
Maheswaran Sathiamoorthy, Santa Clara, CA (US);
Lichan Hong, Mountain View, CA (US);
Yihua Chen, Sunnyvale, CA (US);
Ed Huai-Hsin Chi, Palo Alto, CA (US);
Aakanksha Chowdhery, Mountain View, CA (US);
Hussein Hazimeh, Cambridge, MA (US);
Google LLC, Mountain View, CA (US);
Abstract
A system including a main neural network for performing one or more machine learning tasks on a network input to generate one or more network outputs. The main neural network includes a Mixture of Experts (MoE) subnetwork that includes a plurality of expert neural networks and a gating subsystem. The gating subsystem is configured to: apply a softmax function to a set of gating parameters having learned values to generate a respective softmax score for each of one or more of the plurality of expert neural networks; determine a respective weight for each of the one or more of the plurality of expert neural networks; select a proper subset of the plurality of expert neural networks; and combine the respective expert outputs generated by the one or more expert neural networks in the proper subset to generate one or more MoE outputs.