Google Inc.

Jianmo Ni

Daniel Matthew Cer

John Nham

Yun-Hsuan Sung

Zhe Dong

Siamak Shakeri

Cicero Nogueira Dos Santos

The technology employs soft knowledge prompts (KPs) to inject relevant world knowledge into language models. This includes training KPs via self-supervised learning on data from one or more knowledge bases. KPs are task independent and can function as an external memory of the language models. KPs may be entity-centric, meaning that each prompt primarily encodes information about one entity from a given knowledge base. A method includes identifying a KP in response to a received input text, concatenating that KP to a sequence of word embeddings of the input text, applying the concatenated information to a trained language model, predicting an object entity name, computing a cross-entropy loss, and updating the identified KP based on the computed cross-entropy loss.

Growing community of inventors

Santa Clara, CA, United States of America

Jianmo Ni