Siamak Shakeri

Title: Innovations of Siamak Shakeri in Language Models

Introduction

Siamak Shakeri is an accomplished inventor based in New York City, NY. He has made significant contributions to the field of language models, particularly through his innovative patent. His work focuses on enhancing the capabilities of language models by integrating relevant world knowledge.

Latest Patents

Siamak Shakeri holds a patent for "Soft knowledge prompts for language models." This technology employs soft knowledge prompts (KPs) to inject relevant world knowledge into language models. The method involves training KPs via self-supervised learning on data from one or more knowledge bases. KPs are designed to be task-independent and can function as an external memory for language models. They may be entity-centric, meaning that each prompt primarily encodes information about one entity from a given knowledge base. The process includes identifying a KP in response to a received input text, concatenating that KP to a sequence of word embeddings of the input text, applying the concatenated information to a trained language model, predicting an object entity name, computing a cross-entropy loss, and updating the identified KP based on the computed cross-entropy loss.

Career Highlights

Siamak Shakeri is currently employed at Google Inc., where he continues to push the boundaries of technology in language processing. His innovative approach has garnered attention in the tech community, showcasing his expertise in the field.

Collaborations

Siamak has collaborated with notable colleagues, including Cicero Nogueira Dos Santos and Daniel Matthew Cer. Their combined efforts contribute to the advancement of language model technologies.

Conclusion

Siamak Shakeri's work in developing soft knowledge prompts for language models represents a significant advancement in the integration of world knowledge into artificial intelligence. His contributions continue to shape the future of language processing technologies.