Reliable gradient-free and likelihood-free prompt tuning

Abstract

Prompt embedding samples are drawn from a prior distribution and are passed into a pretrained model to receive a corresponding token label prediction for a batch of text data. Prompt embedding samples are accepted from a distribution of a first iteration; the accepted samples satisfy a condition of a distance function between a ground truth label and the corresponding token label prediction being less than a first tolerance. Embeddings are resampled from the accepted prompt embedding samples with probability proportional to weights and the resampled embeddings are perturbed via a perturbation kernel to obtain a new sample. The perturbed resampled embeddings are propagated through the pretrained model, and those that satisfy a condition are projected, where the second tolerance is decayed by one step per iteration. The projected resampled embeddings are concatenated with an embedding of a given input and inferencing is performed.