Large language model evaluation with enhanced interpretability by k-nearest neighbor search

Abstract

Techniques for fine-tuning free evaluation of large language models with enhanced interpretability using a debiased output probability distribution of a large language model and a probability distribution of a k-Nearest Neighbor search result are provided. In one aspect, a method for performing a downstream task with a language model includes: constructing a datastore by applying the language model to a training set; applying the language model to a prompt-applied sentence from a testing set to obtain a language model feature vector; performing a k-Nearest Neighbor search of the datastore using the language model feature vector as a query vector; and interpolating a probability distribution of results from the k-Nearest Neighbor search and an output probability distribution of the language model to obtain a prediction for the downstream task.