Automatic onboard validation of a newly trained vehicle machine learning model

Abstract

A vehicle system receives indication of a newly trained machine learning model designated for validation. The system load a copy of the model into shadow execution hardware, capable of background execution of the model and subscribes to one or more data topics to which input data for the model, gathered by a vehicle data gathering process, is published. The system executes the model in the background as the vehicle travels, using data published to the data topics and benchmarks output from the model to determine whether the model outperforms a prior version of the model, that represents the model prior to the model being newly trained, based on relative performance of both models compared to performance expectations defined in a configuration file stored by the vehicle. Also, the system, responsive to the model outperforming the prior version of the model based on the performance expectations defined by the configuration file, validates the model as suitable for deployment.