International Business Machines Corporation

Toni Bollinger

Ansgar Dorneich

Christoph Lingenfelder

Oliver Schiller

Andreas Arning

James L Finnie

Friedemann Schwenkreis

Tobias Kevin Fränzel

Thomas Friedhelm Böhme

Wieland Philipp Hoffmann

Reinhold Keuler

Aspects of the invention include techniques for providing a query and data adaptive synopsis table. A non-limiting example method includes building a first synopsis table containing metadata for a database table. Information from queries executed against the database table is collected. The collected information is analyzed, responsive to a number of queries executed against the database table exceeding a query collection threshold, to identify one or more queries for which the first synopsis table was suboptimal. The method includes building, for each of the one or more queries for which the first synopsis table was suboptimal, a second synopsis table.

Query and data adaptive synopsis table

A classification model is trained and used for detecting patterns in input data. The training of the model includes retrieving a set of previously recorded input data containing a plurality of items associated with a plurality of entities and adding to each entity a known classification. Furthermore, training the model includes determining rules from the set of previously recorded input data and the known classification by associating the classification of each entity with the respective items of said entity. The training of the model further includes determining a set of rules which are applicable, aggregating the lift values of the rules determined for said entity, and predicting a classification based on the aggregated association values for each entity. The resulting aggregated lift value together with the respective entity and classification are used as input for a standard classification algorithm, where the result is a classification model.

Method for training and using a classification model with association rule models

Methods and apparatus, including computer program products, implementing and using techniques for compressing data included in several transactions. Each transaction has at least one item. A unique identifier is assigned to each different item and, if taxonomy is defined, to each different taxonomy parent. Sets of transactions are formed from the several transactions. The sets of transactions are stored using a computer data structure including: a list of identifiers of different items in the set of transactions, information indicating number of identifiers in the list, and bit field information indicating presence of the different items in the set of transactions, said bit field information being organized in accordance with the list for facilitating evaluation of patterns with respect to the set of transactions. A data structure for compressing data included in a set of transactions is also provided.

Input data structure for data mining

Methods and apparatus, including computer program products, implementing and using techniques for pattern detection in input data containing several transactions, each transaction having at least one item. Filter conditions for interesting patterns are received, and a first set of filter conditions applicable in connection with generation of candidate patterns is determined. An evaluated candidate pattern is selected as a parent candidate pattern, and evaluation information about the parent candidate pattern is maintained. Child candidate patterns are generated by extending the parent candidate pattern and taking into account the first set of filter conditions. The child candidate patterns are evaluated with respect to the input data together in sets of similar candidate patterns and based on the evaluation information about the parent candidate pattern. At least one child candidate pattern successfully passing the evaluation step is recursively used as a parent candidate pattern.

Data mining by determining patterns in input data

Methods and apparatus, including computer program products, implementing and using techniques for finding deviations in data. A set of candidate patterns is generated. A set of exception patterns that occur in the data less frequently than expected assuming statistical independence is selected from the set of candidate patterns. Data records that comply with at least one of the exception patterns are processed as exception candidates.

Data mining method for finding deviations in data

A method and system for automatically detecting deviations in a data table comprising a multitude of records and a multitude of columns. A column of the data table is selected as a classification column and a classification tree is calculated with respect to the classification column. Each edge of the classification tree is associated with a predicate. The leaf nodes of the classification tree are associated with a leaf record set comprising the subset of records for which the class predicate comprising all predicates along a path from a root node of the classification tree to the leaf nodes evaluates to TRUE. Leaf nodes are associated with a leaf label representing an expected value in the classification column for the corresponding leaf record sets. From the leaf record sets all records deviating with respect to the corresponding classification column from the leaf label are determined as deviation sets.

Growing community of inventors

Weil der Stadt, Germany

Toni Bollinger