Microsoft Technology Licensing, LLC

Wei Xiong

Xuedong D Huang

Jianfeng Gao

Nanshan Zeng

Sriram Srinivasan

Bin Wang

Arash Ghanaie-Sichanie

Pengcheng He

Heiko Rahmel

Li-Juan Qin

Song Wang

Yujia Xie

Yuantao Wang

Yanling Xiong

Lesly Sadiht Miculicich Werlen

Systems and methods are provided for training and using a novel unified language foundation model. An encoder-decoder natural language model is obtained and various training data is obtained and used for training. The training process integrates a combination of replaced token detection, corrupted span reconstruction, and disentangled attention methodologies to produce a unified encoder-decoder model. The trained model is trained for performing both natural language understanding (NLU) tasks and natural language generation (NLG) tasks. Attention applied to the model is applied discretely to segmented chunks of encoded data during processing to improve the efficiency of applying attention by the model.

Pre-training a unified natural language model with corrupted span and replaced token detection

The disclosure herein describes training a text processing model to generate model output text data using input text data and a sentence count. A training data entry including input text data and output text data is obtained. A sentence count of the output text data is determined, and the output text data is labeled with a sentence count label and a sentence number label. Model output text data is generated with a text processing model using the input text data and determined sentence count as input data. Loss data associated with a difference between the generated model output text data and the labeled output text data is used to adjust the text processing model, which enables the model to be trained to produce output text data with a target sentence count in a computationally efficient manner.

Length-controlled text generation using a text processing model

Systems, methods, and computer-readable storage devices are disclosed for generating smart notes for a meeting based on participant actions and machine learning. One method including: receiving meeting data from a plurality of participant devices participating in an online meeting; continuously generating text data based on the received audio data from each participant device of the plurality of participant devices; iteratively performing the following steps until receiving meeting data for the meeting has ended, the steps including: receiving an indication that a predefined action has occurred on the first participating device; generating a participant segment of the meeting data for at least the first participant device from a first predetermined time before when the predefined action occurred to when the predefined action occurred; determining whether the receiving meeting data of the meeting has ended; and generating a summary of the meeting.

Systems, methods, and computer-readable storage device for generating notes for a meeting based on participant actions and machine learning

Growing community of inventors

Bellevue, WA, United States of America

Wei Xiong