Bo Li

Title: Bo Li and Few-Shot Training for Transformer Language Models.

Introduction

Bo Li is an inventor based in Foster City, California. His patent work concerns techniques for training transformer-based language models, with a focus on improving how models learn from prompt-and-completion examples.

Latest Patents

Li is associated with 1 patent: “Few-shot training of transformer language model from a subset of prompt completion.” The disclosed approach uses a training dataset of prompt-and-completion examples to construct a sequence for a transformer-based language model. After a single forward pass, the process performs multiple backward passes conditioned on selected subsets of examples in that sequence. The technique is intended to improve the model’s conditional probability distribution for predicting the next token without adding training time.

Career Highlights

Li is associated with SambaNova Systems, Inc. His patent reflects work on efficient training methods for large language models and in-context few-shot learning.

Collaborations

The provided information lists Zoltan Csaki and Urmish Ajit Thakker as Li’s coworkers. Their names are included here as professional associations; no specific role in the patent is stated.

Conclusion

Bo Li’s patent describes a method for using selected prompt-and-completion examples to improve transformer language model training while avoiding additional training time.