International Business Machines Corporation

Damir Spisic

Jing James Xu

Jing-Yun Shyr

Yannick Saillet

Albert Maier

Xueying Zhang

Fan Li

Yea Jane Chu

Sier Han

Svetlana Levitan

Daniel Jay Rope

Marc S Altshuller

Marius I Danciu

Margaret J Vais

Michael McRoberts

Raymond Wright

Neil Whitney

Graham John Wills

Xue Ying Zhang

Xing Wei

Ana C Gomez

A set of classifiable data containing a plurality of classes is ingested. A target class within the plurality of classes is determined. Using the set of classifiable data, an interactive recall rate chart is generated, and the interactive recall rate chart shows a set of target class recall rates against a set of class recall rates for the remainder of the plurality of classes. The interactive recall rate chart is presented to a user. A target class recall rate selection from the set of target class recall rates is received from the user. The set of classifiable data is reclassified, based on the target class recall rate selection.

Target class analysis heuristics

Mechanisms are provided for detecting interesting decision rules from a set of decision rules in a tree ensemble. Each tree in the tree ensemble is traversed in order to assign each individual data record from a set of data records to an identified leaf node in each tree. Predicted values are determined for the tree ensemble based on predictions provided by each leaf node to which each individual data record is assigned. Interesting sub-indices for decision rules from the set of decision rules are determined and, for each decision rule corresponding to the leaf nodes in the tree ensemble, the sub-indices are combined into interestingness index I. The decision rules are ranked corresponding to the leaf nodes in the tree ensemble according to the associated value of the interestingness index Iand a subset of the decision rules corresponding to the leaf nodes in the tree ensemble are reported.

Detecting interesting decision rules in tree ensembles

Disclosed aspects relate to data insight discovery using a clustering technique. A set of data may be compressed based on a set of proximity values with respect to a set of predictors to assemble a set of sub-clusters. A set of subgroups may be established by merging a plurality of individual sub-clusters of the set of sub-clusters using a tightness factor. A subset of the subgroups may be selected based on a selection criterion. A set of insight data which indicates a profile of the subset of the set of subgroups with respect to the set of data may be compiled for the subset of the set of subgroups.

Data insight discovery using a clustering technique

A computer-implemented method, computer program product and system for data sampling in a storage system. The storage system includes a dataset comprising records and a buffer. The dataset is scanned record-by-record to determine whether the current record belongs to a random sample. If so, then the current record may be added to a first set of records. Otherwise, at least one storage score may be calculated or determined for the current record using attribute values of the current record. Next, it may be determined whether the buffer includes available size for storing the current record. In case the buffer comprises the available size, the current record may be stored in the buffer. Otherwise, at least part of the buffer may be free up. A subsample of the dataset may be provided as a result of merging the first set of records and at least part of the buffered records.

Data sampling in a storage system

Provided are techniques for interactive adjustment of decision rules. A modified decision rule with one or more decision rule conditions is received for adjusting an original decision tree, wherein at least one of the decision rule conditions has been modified. A decision rule condition that has been modified and a corresponding decision tree node of the original decision tree are selected. Data records from a database are selected for a parent node of the corresponding decision tree node. The selected data records that match the decision rule condition are filtered. A sub-tree is generated using the filtered data records with a first splitting variable from the modified decision rule condition. An original sub-tree is replaced with the generated sub-tree and the decision rule condition to form an adjusted decision tree. The adjusted decision tree is used to predict a value of a target variable based on available predictors.

Interactive adjustment of decision rules

A method, computer program product and system for data sampling in a storage system. The storage system includes a dataset comprising records and a buffer. The dataset is scanned record-by-record to determine whether the current record belongs to a random sample. If so, then the current record may be added to a first set of records. Otherwise, at least one storage score may be calculated or determined for the current record using attribute values of the current record. Next, it may be determined whether the buffer includes available size for storing the current record. In case the buffer comprises the available size, the current record may be stored in the buffer. Otherwise, at least part of the buffer may be free up. A subsample of the dataset may be provided as a result of merging the first set of records and at least part of the buffered records.

A method for comparing predictive data models based on a predictive model search is provided. The method may include receiving a first and second portion of a set of data. The method may also include identifying a first and second variation of the second portion, wherein the first variation is different from the second variation. The method may further include generating first predictive data models based on the first variation, and second predictive data models based on the second variation. Additionally, the method may include applying a criteria to rank the first and second predictive data models based on predictive strength. The method may also include presenting a display of the ranked criteria, comprising the first portion, and a portion of the first and second predictive data models, wherein the portion of the first and second predictive data models are collectively ranked and presented according to the predictive strength.

Predictive model search by communicating comparative strength

Growing community of inventors

Chicago, IL, United States of America

Damir Spisic