The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Sep. 06, 2022
Filed:
Sep. 30, 2019
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Bharadwaj Pudipeddi, San Jose, CA (US);
Marc Tremblay, Bellevue, WA (US);
Sujeeth Subramanya Bharadwaj, Milpitas, CA (US);
Devangkumar Patel, Freemont, CA (US);
Jinwen Xi, Sunnyvale, CA (US);
Maral Mesmakhosroshahi, Sunnyvale, CA (US);
MICROSOFT TECHNOLOGY LICENSING, LLC, Redmond, WA (US);
Abstract
Methods, systems, apparatuses, and computer program products are described herein that enable execution of a large AI model on a memory-constrained target device that is communicatively connected to a parameter server, which stores a master copy of the AI model. The AI model may be dissected into smaller portions (e.g., layers or sub-layers), and each portion may be executed as efficiently as possible on the target device. After execution of one portion of the AI model is finished, another portion of the AI model may be downloaded and executed at the target device. To improve efficiency, the input samples may be divided into microbatches, and a plurality of microbatches executing in sequential order may form a minibatch. The size of the group of microbatches or minibatch can be adjusted to reduce the communication overhead. Multi-level parallel parameters reduction may be performed at the parameter server and the target device.