International Business Machines Corporation

Si Er Han

Jing James Xu

Xue Ying Zhang

Ji Hui Yang

Xiao Ming Ma

Jun Wang

Lei Gao

Jing Xu

A Peng Zhang

Wen Pei Yu

Bei Chen

Dakuo Wang

Chuang Gan

Steven G Barbee

Jun Song Wang

Chun Lei Xu

Dhavalkumar C Patel

Syed Yousaf Shah

Jin Wang

Rui Wang

Peter Daniel Kirchner

Horst Cornelius Samulowitz

Long H Vu

Saket Sathe

Gregory Bramble

Xuan-Hong Dang

Sheng Yan Sun

Kai Lin Li

Lingfei Wu

Lei Tian

Petros Zerfos

Jia Wang

Xing Xing Shen

Xu Zhao

Pei-Yi Lin

Di Ling Chen

Xing Wei

Zhan Peng Huo

Wesley M Gifford

Hong Wei Sun

Yan Xa Zhou

Chong Liu

Yi Shao

Arunima Chaudhary

Abel Nicolas Valente

Jin Rong Wang

Daniel Karl I Weidele

Sharon Chen

Yi Wang

Yang Y Yang

John Dillon Eversman

Voranouth Supadulya

Dian Bj Wang

Jacqueline Cheng

Xuye Liu

Long Vu

Wei Shan Ws Kang

Micky Chiang

Hua Xa Li

Jing Jia

Shi Bin Liu

Hao Zheng

Data segmentation, analysis, and security is provided. An analysis of a set of generated data evolution paths corresponding to a set of data collected over a defined span of time is performed. A behavior trend is determined based on analysis of the set of generated data evolution paths. A set of action steps is performed automatically based on the determined behavior trend.

Taking action based on data evolution

A method, system, and computer program product for incremental machine learning for a parametric machine learning model are disclosed. The method may include processing samples comprising historical samples and new samples with an existing parametric machine learning model to obtain at least one prediction residual of each of the samples, wherein the existing parametric machine learning model was trained based on the historical samples. The method may further include clustering the samples based on the at least one prediction residual of each of the samples and features of each of the samples. The method may further include sampling samples in each cluster to ensure that each cluster includes substantially similar number of sampled samples. The method may further include updating the existing parametric machine learning model to obtain an updated parametric machine learning model based on sampled samples in each cluster.

Incremental machine learning for a parametric machine learning model

A method, a computer system, and a computer program product for receiving a plurality of unreduced training data records (TDRs), using a first autoencoder on the plurality of unreduced TDRs to obtain a first plurality of reduced TDRs, clustering the first plurality of reduced TDRs into a plurality of clusters, wherein each first reduced TDR cluster is assigned a respective cluster label, transferring the respective cluster label from each first reduced TDR cluster to a corresponding unreduced TDR cluster, and performing stratified sampling to form K data blocks of unreduced TDRs from a plurality of unreduced TDR clusters.

Dimension reduction in the context of unsupervised learning

An example operation may include one or more of storing an original data set in memory, splitting the original data set into a subset of continuous-type data values and a subset of discrete-type data values based on variable types in the original data set, converting the subset of continuous-type data values into a second subset of discrete-type data values based on a data binning operation, generating a new subset of continuous-type data values based on the subset of continuous-type data values in the original data set, and combining a subset of discrete-type data values from a conditional contingency table within the new subset of continuous-type data values to generate a new data set.

Data generation process for multi-variable data

One or more computer processors group a plurality of predictors contained in training data into a plurality of predictor groups. The one or more computer processors create a plurality of sample sets, wherein each sample set in the plurality of sample sets contains one or more predictors selected from a respective predictor group in the plurality of predictor groups. The one or more computer processors create a cluster model for each created sample set in the plurality of created sample sets. The one or more computer processors generate a score for a record with one or more missing values utilizing at least one created cluster model of the created cluster models and at least one created sample set of the created sample sets.

Intelligent scoring of missing data records

A computer-implemented method for anomaly detection for a time series data is provided. Aspects include receiving a time series data including a plurality of sequential data points, calculating an expected next value for the time series data based on the plurality of sequential data points, and receiving an actual next value corresponding to the time series data. Aspects also include calculating an anomaly strength estimate based on the expected next value and the actual next value, identifying one of a plurality of anomaly detection pipelines based on the anomaly strength estimate and a portrait associated with each of the plurality of anomaly detection pipelines, and obtaining an anomaly prediction by inputting the time series data and the actual next value into the one of the plurality of anomaly detection pipelines.

Anomaly detection for time series data

A method for improving a query performance of a query of a database application using rewrite includes introducing a database query into a database application to obtain a query result, analyzing the database query to identify index columns and a predicate column; identifying associated columns by determining if any associations exist between the index columns and the predicate column, calculating a confidence score value for each of the associated columns responsive to the associated columns and the predicate column, generating a list of associated columns with the confidence score values, generating a rewritten query by rewriting the database query based on the list of associated columns and the confidence score values and validating the rewritten query.

Query performance improvement of a query for a database application using rewrite

A computer-implemented method for treating post-modeling data includes computing, sequentially for each category of a feature, a category importance (CI) value. The CI value is based on a model accuracy change when records of a category being examined are reassigned to a remaining set of categories of the feature according to a cumulative distribution of records among the remaining set of categories of the feature, wherein the remaining set of categories include all categories of the feature, except for the category being examined. A post-modeling category is performed to merge of each category having the CI value less than a CI value threshold.

Key category identification and visualization

An approach to time-series data point anomaly detection may be presented. Data point anomalies in time-series data can cause a cascade of incorrect predictions in a time-series data prediction model. Presented herein may be an approach to decompose a time-series training data set into elementary components, such as seasonal, trend and residual. The approach may determine one or more confidence intervals for elementary components of data points including level shift, variance, and outlier. From these confidence intervals, new data points can be analyzed and identified as anomaly data points. The approach may also prevent anomaly data points from being incorporated into a time series data prediction model, reducing prediction error in the prediction model.

Optimization of time-series anomaly detection

In an approach for post-modeling data visualization and analysis, a processor presents a first visualization of a training dataset in a first plot. Responsive to receiving a selection of a data group of the training dataset to analyze, a processor identifies three or fewer key model features of the data group of the training dataset. A processor ascertains a representative record of each key model feature of the three or fewer key model features using a Local Interpretable Model-Agnostic Explanation technique. A processor presents a second visualization of the three or fewer key model features and the representative record of each key model feature in a second plot.

Visualize data and significant records based on relationship with the model

A method, computer system, and a computer program product are provided for post-modeling feature evaluation. In one embodiment, at least at least one post model visual output and associated data is obtained that at least includes an individual conditional expectation (ICE) plot and a partial dependence (PDP) plot. Using the associated data and the plots, a Feature Importance (PI) plot is provided. A plurality of features is then determined for each PI, PDP and ICE plots to calculate at least one Interesting Value for each plot. An overall score is also calculated for each plurality of features based on the associated Interesting Values for each PDP, ICE and PI plots. At least one top feature is selected based on said scores. A final plot is then generated at least reflecting the top feature. The final plot combines the PI, PDP and ICE plots together.

Visual representation using post modeling feature evaluation

Disclosed are a computer-implemented method, a system and a computer program product for model exploration. Model feature importance of each model of a plurality of models can be obtained, the plurality of models can be grouped into a plurality of model clusters based on the model feature importance of each model, and the model feature importance can be presented by box-plot or confidence interval.

Feature importance based model optimization

Embodiments of the present disclosure relate to a method, system, and computer program product for efficient execution of a decision tree. According to the method, respective target values of a plurality of attributes of a target entity are obtained. Representations of a plurality of leaf nodes of a decision tree are obtained. Each of the representations indicates respective statistic values of a plurality of attributes of historical entities and a statistic prediction result determined from historical prediction results output at a respective one of the plurality of leaf nodes for the historical entities. Distance measures between the target entity and the plurality of leaf nodes are determined based on the target values and the statistic values indicated by the representations of the plurality of leaf nodes. A target prediction result for the target entity is determined based on the distance measures and the statistic prediction results of the historical entities.

Efficient execution of a decision tree

A method for identifying influential effects that contribute most to a status change of a target index for goal seeking analysis. The method includes generating a candidate list of significant changed predictors between the normal and abnormal status time periods in collected data, and building a plurality of regression models from the collected data. The method determines a first value (trend value or Pearson correlation value) for each of the significant changed predictors based on whether at least one of the significant changed predictors have a significant change trend using the regression models. The method obtains a second predictor importance value for each of the significant changed predictors from a single model built on all the collected data. The method generates a final predictor value for each of the significant changed predictors by combining the first value with the second predictor importance value for each of the significant changed predictors.

Identifying influential effects to be adjusted in goal seek analysis

Performing predictive analysis on running batch jobs is provided. A series of batch end time predictive models is retrieved according to a sequence of milestone jobs in a batch of jobs. Retrieved batch end time predictive models are assembled into an aggregate batch end time predictive model to increase accuracy and stability of an end time prediction of the batch of jobs. The aggregate batch end time predictive model is utilized to predict an end time of the batch of jobs during running of the batch of jobs to form a predicted end time of the batch of jobs.

Predictive analysis on running batch jobs

Growing community of inventors

Xi'an, China

Si Er Han