John Nham

Title: Innovations of John Nham in Language Models

Introduction

John Nham is an accomplished inventor based in Fremont, California. He has made significant contributions to the field of language models, particularly through his innovative patent. His work focuses on enhancing the capabilities of language models by integrating relevant world knowledge.

Latest Patents

John Nham holds a patent for "Soft knowledge prompts for language models." This technology employs soft knowledge prompts (KPs) to inject relevant world knowledge into language models. The method includes training KPs via self-supervised learning on data from one or more knowledge bases. KPs are task-independent and can function as an external memory for language models. They may be entity-centric, meaning that each prompt primarily encodes information about one entity from a given knowledge base. The process involves identifying a KP in response to a received input text, concatenating that KP to a sequence of word embeddings of the input text, applying the concatenated information to a trained language model, predicting an object entity name, computing a cross-entropy loss, and updating the identified KP based on the computed cross-entropy loss.

Career Highlights

John Nham is currently employed at Google Inc., where he continues to push the boundaries of technology in language processing. His innovative approach to integrating knowledge into language models has garnered attention in the tech community.

Collaborations

Some of his notable coworkers include Siamak Shakeri and Cicero Nogueira Dos Santos, who contribute to the collaborative environment at Google Inc.

Conclusion

John Nham's work in developing soft knowledge prompts for language models represents a significant advancement in the field of artificial intelligence. His contributions are paving the way for more intelligent and context-aware language processing systems.