Zhe Dong

Title: Zhe Dong: Innovator in Language Models

Introduction

Zhe Dong is a prominent inventor based in Zurich, Switzerland. He has made significant contributions to the field of language models, particularly through his innovative patent. His work focuses on enhancing the capabilities of language models by integrating relevant world knowledge.

Latest Patents

Zhe Dong holds a patent titled "Soft knowledge prompts for language models." This technology employs soft knowledge prompts (KPs) to inject relevant world knowledge into language models. The method includes training KPs via self-supervised learning on data from one or more knowledge bases. KPs are task-independent and can function as an external memory for language models. They may be entity-centric, meaning that each prompt primarily encodes information about one entity from a given knowledge base. The process involves identifying a KP in response to a received input text, concatenating that KP to a sequence of word embeddings of the input text, applying the concatenated information to a trained language model, predicting an object entity name, computing a cross-entropy loss, and updating the identified KP based on the computed cross-entropy loss.

Career Highlights

Zhe Dong is currently employed at Google Inc., where he continues to push the boundaries of language processing technologies. His work has garnered attention for its innovative approach to integrating knowledge into language models, making them more effective and versatile.

Collaborations

Zhe Dong collaborates with notable colleagues, including Siamak Shakeri and Cicero Nogueira Dos Santos. Their combined expertise contributes to the advancement of language model technologies and the exploration of new methodologies.

Conclusion

Zhe Dong's contributions to the field of language models through his patent on soft knowledge prompts exemplify the innovative spirit of modern technology. His work at Google Inc. and collaborations with esteemed colleagues further highlight his impact on the industry.