The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jan. 20, 2026
Filed:
Dec. 25, 2020
Microsoft Technology Licensing, Llc, Redmond, WA (US);
Jian Jiao, Bellevue, WA (US);
Yeyun Gong, Beijing, CN;
Nan Duan, Beijing, CN;
Weizhu Chen, Bellevue, WA (US);
Kewen Tang, Bellevue, WA (US);
Qiang Lou, Bellevue, WA (US);
Ruofei Zhang, Sunnyvale, CA (US);
Yu Yan, Bellevue, WA (US);
Jiusheng Chen, Bellevue, WA (US);
Microsoft Technology Licensing, LLC, Redmond, WA (US);
Abstract
Systems and methods are provided for training a data model based on training data. The training includes pre-training and fine-tuning the data model based on a combination of an autoregressive (AR) model and a non-autoregressive (NAR) model. Training data may be received and encoded into streams of tokens. A pre-trainer during decoding generates a continuum of data structures of the AR and NAR combined model including a main stream and a series of predicting streams. Masked tokens in predicting streams reference or attend to one or more preceding tokens in the main stream or the preceding predicting streams. A fine-tuner selects streams to generate a trained model according to a target data model. The target data model is determined based on balancing an accuracy constraint and an efficiency constraint for predicting tokens. The decoder acts as abridge between the AR and NAR models in generating a trained data model.