Methods and apparatus for natural language based scenario discovery to train a machine learning model for a driving system

Abstract

A method for classifying environmental image frames for vehicles, using a universal iterative classification model, is presented. The method combines text-based querying, active machine-learning models, and user input to form an end-to-end automatic flow for sourcing video frames captured by sensors on a vehicle. An index of frames is used and continuously populated with data for new frames, with each frame scored on its likelihood of containing a representation of a driving scenario of interest. Each iteration of the classification model produces a classification result that is predicted to belong to the scenario of interest. Binary labels can be applied to the results. Subsequent iterative training of classification models can be performed using updated training sets containing previously labeled classification results, to improve precision and accuracy in classifying image data to a driving scenario.