International Business Machines Corporation

Umit M Cakmak

Lukasz Grzegorz Cmielowski

Wojciech Sobala

Marek Oszajec

Andrzej Jan Wrobel

Pawel Slowikowski

Embodiments of the present invention disclose a method, computer program product, and system for mitigating machine learning performance digression due to insufficient test data availability. A set of data is received, wherein the received set of data is parsed into a set of training data and a set of test data. A trained model is generated and the trained model is applying to the set of test data. A first set of performance values of the tested trained model are recorded and, if above a threshold, associated with a performance baseline value. A set of modified test data is generated and the trained model is applied to the set of modified test data. A second set of performance values are recorded and a performance difference value is calculated based on the performance baseline value and second set of recorded performance values. A table of results is generated, for display.

Assessment of machine learning performance with limited test data

A processor may acquire a trained predictive computational model from a database. The processor may apply a trained reduced complexity model to the trained predictive computational model. The trained reduced complexity model may be associated with the trained predictive computational model. The processor may select at least one metric. The processor may determine a quality indicator related to the at least one metric by identifying the type of the at least one metric, evaluating the output of the trained predictive computational model in relation to the type of the at least one metric, and generating, based on the evaluation of the trained predictive computational model, a threshold associated with the at least one metric. The processor may determine the accuracy of the trained predictive computational model based on the quality indicator.

Automatically determining accuracy of a predictive model

A current data set with multiple records is fed into a data analysis model. The current data set is limited to data occurring in a current time window with a predetermined window size. The model is run on the current data set and a current data prediction result is generated. Limited historical data sets having multiple records are selected. Each record has values for several features. Each historical data set is limited to data occurring in a historical time window having the window size. A historical class label distribution is determined for the historical data sets and an upper and lower control limit are determined for the historical data sets using the historical class label distribution. A current class label distribution having a mean value is determined for the current prediction result. An alert is provided when the mean value is not between the upper and lower control limit.

Prediction quality assessment

A method for training a deep learning algorithm using N-dimensional data sets may be provided. Each data set comprises a plurality of N&#x2212;1-dimensional data sets. The method comprises selecting a batch size and assembling an equally sized training batch. The samples are selected to be evenly distributed within said respective N-dimensional data sets. The method comprises also starting from a predetermined offset number, wherein the number of samples is equal to the selected batch size number, and feeding said training batches of N&#x2212;1-dimensional samples into a deep learning algorithm for the training. Upon the training resulting in a learning rate that is below a predetermined level, selecting a different offset number for at least one of said N-dimensional data sets, and going back to the step of assembling. Upon the training resulting in a learning rate that is equal or higher than said predetermined level, the method stops.

Growing community of inventors

Cracow, Poland

Umit M Cakmak