Beijing Baidu Netcom Science Technology Co., Ltd

Tongyang Liu

Shu Wang

Yong Zhu

Wei Zheng

Zhifan Feng

Chunguang Chai

Wanli Chang

A method for generating a pre-trained language model, includes: obtaining sample files; obtaining typography structure information and text information of the sample files by parsing the sample files; obtaining a plurality of task models of a pre-trained language model; obtaining a trained pre-trained language model by jointly training the pre-trained language model and the plurality of task models according to the typography structure information and the text information; and generating a target pre-trained language model by fine-tuning the trained pre-trained language model according to the typography structure information and the text information.

Growing community of inventors

Beijing, China

Tongyang Liu