Knowledge distillation and automatic model retraining via edge device sample collection

Abstract

Techniques for machine learning (ML) model knowledge distillation and automatic retraining are described. A model adaptation controller obtains samples generated by an edge device and inference values generated based on the samples by a deployed ML model of the edge device. The model adaptation controller runs inference on the samples using a different ML model to generate inferences that can be used to determine whether the performance of the deployed ML model is lacking. If so, the model adaptation controller can retrain the deployed ML model using samples with ground truth values generated by the different ML model, resulting in a light-weight retrained model that can be provisioned to the edge device. This retraining process may be performed iteratively to automatically improve and adapt the ML model running at the edge device.