International Business Machines Corporation

Shrey Shrivastava

Dhavalkumar C Patel

Arun Kwangil Iyengar

Nianjun Zhou

Anuradha Bhamidipaty

Emmanuel Yashchin

Dinesh C Verma

Jayant R Kalagnanam

Chandrasekhara K Reddy

Jeffrey Willoughby

Yuanchen Hu

Shuxin Lin

Methods and systems to provide a form of probabilistic labeling to associate an outage with a disturbance, which could itself be either known based on the available data or unknown. In the latter case, labeling is especially challenging, as it necessitates the discovery of the disturbance. One approach incorporates a statistical change-point analysis to time-series events that correspond to service tickets in the relevant geographic sub-regions. The method is calibrated to separate the regular periods from the environmental disturbance periods, under the assumption that disturbances significantly increase the rate of loss-causing events. To obtain the probability that a given loss-causing event is related to an environmental disturbance, the method leverages the difference between the rate of events expected in the absence of any disturbances (baseline) and the rate of actually observed events. In the analysis, the local disturbances are identified and estimators of their duration and magnitude are provided.

Associating disturbance events to accidents or tickets

Disclosed embodiments provide techniques for estimating imputation algorithm performance. Multiple imputer algorithms are selected, and an evaluation of how well each of the imputer algorithms can estimate the missing data is performed. Disclosed embodiments obtain an imputer candidate dataset (ICD). The imputer candidate dataset is compared to the incomplete data range, and a similarity metric is determined between the data range and the ICD. When the similarity metric exceeds a predetermined threshold, an imputer evaluation dataset (IED) is created from the ICD by removing one or more data points from the ICD. Each imputer algorithm is evaluated by applying the IED to it, and computing an imputer evaluation metric based on its performance. The multiple imputer algorithms are ranked based on the imputer evaluation metric. The best ranked imputer algorithm can then be selected for use on the incomplete data range within the measurement dataset.

Data gap mitigation

A computer implemented method of preparing process data for use in an artificial intelligence (AI) model includes collecting and storing raw data as episodic data for each episode of a process. An episode data generator assigns an episode identifier each set of episodic data. The raw data per episode is transformed into a standardized episodic data format that is usable by the AI model. Metrics are assigned to the episodic data and the episodic data is aggregated in an episode store. The data in the episode store is used by a feature extraction and learning module to extract and rank features.

Dynamic and continuous composition of features extraction and learning operation tool for episodic industrial process

A computing device, method, and system are provided of improving data quality to conserve computational resources. The computing device receives a raw dataset. One or more data quality metric goals corresponding to the received raw dataset are received. A schema of the dataset is determined. An initial set of validation nodes is identified based on the schema of the dataset. The initial set of validation nodes are executed. A next set of validation nodes are iteratively expanded and executed based on the schema of the dataset until a termination criterion is reached. A corrected dataset of the raw dataset is provided based on the iterative execution of the initial and next set of validation nodes.

Dynamic discovery and correction of data quality issues

An apparatus includes circuitry configured to maintain a record of a plurality of owners and at least one test operation owned by an owner; prompt automatically the owner in response to a failure of the one test operation; maintain a log of actions taken on the one test operation, and provide availability to the log of actions; update an estimated time to completion, and notify a management entity of the updated estimated time to completion; mark and prioritize an order related to the one test operation, in response to the estimated time to completion being within a threshold of a delivery date; rank the marked order with other marked orders by a risk of not being able to meet the delivery date; and notify the owner of the ranking with an urgent message, in response to the marked order failing to meet the delivery date.

Method and system for enhancing orchestration and automating communication between teams during IT systems testing

Embodiments of the invention are directed a computer-implemented method for efficiently assessing data quality metrics. A non-limiting example of the computer-implemented method includes receiving, using a processor, a plurality of updates to data points in a data stream. The processor is further used to provide a plurality of data quality metrics (DQMs), and to maintain information on how much the plurality of DQMs are changing over time. The processor also maintains information on computational overhead for the plurality of DQMs, and also updates data quality information based on the maintained information.

Efficient real-time data quality analysis

Embodiments of the invention are directed a computer-implemented method for assessing data quality. A non-limiting example of the computer-implemented method includes using a processor to receive a plurality of updates to data points in a data stream. The processor is further used to compute instances of a data quality metric (DQM) from the data points in the data stream. The instances of the DQM are configured to differentiate the data points in the data stream by time and assign a higher weight to the instances of the DQM computed from more recent data points in the data stream. The instances of the DQM are continuously updated as more of the data points are received by the processor while limiting cycles of the processor consumed by updating the instances of the DQM to a threshold.

Growing community of inventors

White Plains, NY, United States of America

Shrey Shrivastava