Salesforce, Inc.

Rishabh Bhardwaj

Chu Hong Hoi

Amrita Saha

Embodiments described herein provide a soft prompt tuning technique referred to as the Vector quantized Input-contextualized Prompt (VIP). The VIP techniques has two integral properties i) instead of learning a fixed set of prompt tokens irrespective of the input, it generates a contextualized version of the soft prompts, conditional on the input text ii) it further passes the input-contextualized prompt tokens through a quantization network, inspired by Vector Quantized Transformers. The quantization network uses nearest neighbor search over a learnable codebook to train a discrete latent variable model over the prompt-space, thus generating quantized version of contextual prompt tokens. These quantized contextual prompt tokens are finally fed into the frozen language model along with the original input text.

Growing community of inventors

Singapore, Singapore

Rishabh Bhardwaj