Methods and systems for transferring knowledge from large language models to small language models using out-of-distribution feedbacks

Abstract

A method and an apparatus for transferring knowledge from a Large Language Model (LLM) to a Small Language Model (SLM) are provided. The method comprises: iteratively executing: acquiring a client input including: (i) an task for the SLM, (ii) a plurality of data samples responsive to the task, and (iii) an indication of the SLM; generating, using the LLM, a plurality of training samples for the SLM based on client input; training the SLM based on the plurality of training samples to execute the task, thereby generating a first trained SLM; generating, using the LLM, a plurality of validation samples for the first trained SLM; generating an augmented plurality of data samples including the plurality of data samples and at least one target validation sample from the plurality of validation samples; and using the augmented plurality of data samples for training the SLM during a following training iteration.