The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
May. 20, 2025
Filed:
Jun. 10, 2022
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Adit Krishnan, Mountain View, CA (US);
Ji Li, San Jose, CA (US);
Yixuan Wei, Beijing, CN;
Xiaozhi Yu, San Jose, CA (US);
Han Hu, Beijing, CN;
Qi Dai, Beijing, CN;
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
A data processing system implements a dynamic knowledge distillation process including dividing training data into a plurality of batches of samples and distilling a student model from a teacher model using an iterative knowledge distillation. The process includes instantiating an instance of the teacher model and the student model in a memory of the data processing system and obtaining a respective batch of training data from the plurality of batches of samples in the memory. The process includes training the teacher and student models using each of the samples in the respective batch of the training data, evaluating the performance of the student model compared with the performance of the teacher model, and providing feedback to student model to adjust the behavior of the student model based on the performance of the student model.