International Business Machines Corporation

Sattwati Kundu

Geetha Adinarayan

Mansoor Ahmed

Nair Raghunath Eledath

Diptikalyan Saha

Vitobha Munigala

Samiulla Zakir Hussain Shaikh

Embodiments of the present invention provide computer-implemented methods, computer program products and computer systems. Embodiments of the present invention can, in response to receiving information, generate a data profile for a model that includes metadata for data requirements, model specific requirements, and data quality metrics. Embodiments of the present invention can generate one or more perturbations for training data associated with the received information and validate at least one perturbation of the one or more perturbations of training data as relevant test data based, at least in part on context associated with the model. Embodiments of the present invention can then generate one or more test scenarios based on the at least one validated perturbation and varying hyperparameters of the model and generate a test report based on an execution of at least one generated test scenario of the generated one or more test scenarios.

Testing models in data pipeline

A system, computer program product, and method are presented for providing replacement data for data in a time series data stream that has issues indicative of errors, where the data issues and the replacement data are related to one or more KPIs. The method includes determining one or more predicted replacement values for potentially erroneous data instances in the time series data stream. The method further includes resolving the potentially erroneous data instances with one predicted replacement value of the one or more predicted replacement values in the time series data stream.

Data quality-based computations for KPIs derived from time-series data

Methods, systems and computer readable media are provided for configuring machine learning systems to automatically and dynamically select a machine learning model based on statistical profiling of received data to improve machine learning applications for high variance data. Data is received from a system in operation. A profile is computed for the received data. A database comprising a plurality of stored profiles for a dataset is accessed, wherein each stored profile corresponds to a distinct pattern identified in the dataset. The computed profile is compared to each of the stored profiles to determine whether the computed profile matches one or more of the stored profiles. When one or more stored profiles match the computed profile, a matching profile is selected by the machine learning system. The received data is processed using a ML model associated with the matching profile.

Dynamically applying machine learning models from profiling received data

A system, computer program product, and method are presented for providing confidence values for replacement data for data that has issues indicative of errors, where the data issues, the replacement data, and confidence values are related to one or more KPIs. The method includes identifying one or more potentially erroneous data instances and determining one or more predicted replacement values for the potentially erroneous data instances. The method further includes determining a confidence value for each predicted replacement value and resolving the one or more potentially erroneous data instances with one predicted replacement value of the one or more predicted replacement values. The method also includes generating an explanatory basis for the resolution of the one or more potentially erroneous data instances.

Data quality-based confidence computations for KPIs derived from time-series data

A computer-implemented method, system and computer program product for identifying anomalies in data during a data outage. An anomaly detection model is built using data received from a sensor at a characterized granularity. Once a period of service occurs following a data outage, a quantum of missing data during the data outage is identified using predictive modeling if the data during the data outage is not available at the granularity in which the anomaly detection model is built. The identified quantum of missing data is retrofitted into a predicted pattern during the data outage and the analytics are then re-run on the retrofitted quantum of missing data in the predicted pattern to identify anomalies during the data outage. In this manner, anomalies in data, such as data from sensor readings, can be identified during the data outage thereby enabling the model to provide more accurate predictions of anomalies occurring during the data outage.

Growing community of inventors

Bengaluru, India

Sattwati Kundu