Machine learning model development and optimization process that ensures performance validation and data sufficiency for regulatory approval

Abstract

Machine learning model development and optimization tools are provided that ensure performance validation and data sufficiency for regulatory approval. According to an embodiment, a computer implemented method can comprise training a machine learning model to perform an inferencing task on an initial set of data samples included in a sample population. In various embodiments, the model can include a medical AI model. The method further comprises determining, by the system, subgroup performance measures for subgroups of the data samples respectively associated with different metadata factors, wherein the subgroup performance measures reflect performance accuracy of the machine learning model with respect to the subgroups. The method further comprises determining, by the system, whether the machine learning model meets an acceptable level of performance for deployment in a field environment based on whether the subgroup performance measures respectively satisfy a threshold subgroup performance measure.