Microsoft Technology Licensing, LLC

Qiang Lou

Ruofei Zhang

Li Deng

Jianfeng Gao

Xiaodong He

Jian Jiao

Guowei Liu

Yelong Shen

Gregory T Buehrer

Jianchang Mao

Yunhong Zhou

Luis Gerardo Mojica De La Vega

A tagging system appends supplemental information to an original sequence of items, to produce a supplemented sequence of items. The tagging system includes a transformer-based encoder neural network that maps the supplemented sequence into hidden state information. The tagging system includes a post-processing neural network that transform the hidden state information into a tagged output sequence of items. That is, each item in the tagged output sequence includes a tag that identifies its entity class or some other characteristic. The tagging system can increase the accuracy of the tags it produces by virtue of the inclusion of the supplemental information added to each original sequence. A training system trains the tagging system to perform plural tasks, which further increases the accuracy of the tags it produces. The training system may commence training of the tagging system using a pre-trained model for the encoder neural network.

Multi-task sequence tagging with injection of supplemental information

Functionality is described herein for transforming first and second symbolic linguistic items into respective first and second continuous-valued concept vectors, using a deep learning model, such as a convolutional latent semantic model. The model is designed to capture both the local and global linguistic contexts of the linguistic items. The functionality then compares the first concept vector with the second concept vector to produce a similarity measure. More specifically, the similarity measure expresses the closeness between the first and second linguistic items in a high-level semantic space. In one case, the first linguistic item corresponds to a query, and the second linguistic item may correspond to a phrase, or a document, or a keyword, or an ad, etc. In one implementation, the convolutional latent semantic model is produced in a training phase based on click-through data.

Convolutional latent semantic models and their applications

Growing community of inventors

Bellevue, WA, United States of America

Qiang Lou