International Business Machines Corporation

Christoph Lingenfelder

Michael Wurst

Pascal Pompey

Yannick Saillet

Liangzhao Zeng

Olivier Verscheure

Andreas Arning

Stefan Raspl

Joachim H Frank

Alexander Lang

Christoph H Sieb

Toni Bollinger

Hung-yang Chang

Ansgar Dorneich

Gregor Meyer

Iliyana P Ivanova

Felix Hamborg

Simone Zerfass

Dieter Helmut Roller

Joseph Phillip Bigus

Reinhold Geiselhart

Graham Derek Wallis

Colin Leonard Bird

Edwin Peter Dawson Pednault

Robert William Phippen

Virgil Dorin Gligor

Upendra D Chitnis

Martin Keller

Leon Gong

Sven Lorenz

Klaus Deinhart

Stefan Abraham

Janna Orechkina

Swen Wohland

Boris Charpiot

Barbara Hartel

Thilo Maier

Processing data of a data warehouse is provided and includes receiving, by a processing device, user input to create simple filter objects. Each filter object defines an ad hoc subset of a respective dimension of a dimension table of the data warehouse. User input is received to create a filtered operation object that specifies an operation and a plurality of the simple filter objects. The ad hoc subset differs from all subsets defined in the dimension table.

Processing data in a data warehouse

Techniques for iterative feature extraction using domain knowledge are provided. In one aspect, a method for feature extraction is provided. The method includes the following steps. At least one query to predict at least one future value of a given value series based on a statistical model is received. At least two predictions of the future value are produced fulfilling at least the properties of 1) each being as probable as possible given the statistical model and 2) being mutually divert (in terms of numerical distance measure). A user is queried to select one of the predictions. The user may be queried for textual annotations for the predictions. The annotations may be used to identify additional covariates to create an extended set of covariates. The extended set of covariates may be used to improve the accuracy of the statistical model.

Iterative active feature extraction

Data of a database environment, which includes hierarchy information and a matrix of values, is processed. The hierarchy information includes at least two sets of identification codes and defines at least two groups of identification codes. The matrix of values includes at least two columns of identification values. At least one simple filter object is generated based on a user input. Each simple filter object defines an ad hoc group of identification codes selected from a respective one of the sets of identification codes. A filtered operation object that specifies an operation and at least one of the simple filter objects is generated based on a user input. Each of the ad hoc groups differs from each of the groups defined by the hierarchy information.

Techniques are disclosed for determining reasons underlying insights gleaned from multi-dimensional data. In one embodiment, a contingency table is accessed that represents multiple dimensions of the data, in order to identify one or more insights. One or more dimensions, other than the represented dimensions, are evaluated to identify one or more reasons underlying a first insight of the one or more insights, and the one or more reasons are output.

Insight determination and explanation in multi-dimensional data sets

A first data mining model and a second data mining model are compared. A first data mining model Mrepresents results of a first data mining task on a first data set Dand provides a set of first prediction values. A second data mining model Mrepresents results of a second data mining task on a second data set Dand provides a set of second prediction values. A relation R is determined between said sets of prediction values. For at least a first record of an input data set, a first and second probability distribution is created based on the first and second data mining models applied to the first record. A distance measure d is calculated for said first record using the first and second probability distributions and the relation. At least one region of interest is determined based on said distance measure d.

Probabilistic data mining model comparison

A predictive analysis generates a predictive model (Padj(Y|X)) based on two separate pieces of information, The predictive analysis begins by generating a base model distribution (Pgen(Y|X)) from the original training data set (Dorig) containing tuples (x,y) of indicators (x) and corresponding labels (y). Using the 'true' distribution (Ptrue(X)) of indicators, a random data set (D') of indicator records (x) is generated reflecting this 'true' distribution (Ptrue(X)). Subsequently, the base model (Pgen(Y|X)) is applied to said random data set (D&#x2032;), thus assigning a label (y) or a distribution of labels to each indicator record (x) in said random data set (D&#x2032;) and generating an adjusted training set (Dadj). Finally, an adjusted predictive model (Padj(Y|X)) is trained based on said adjusted training set (Dadj).

Predictive modeling

A method for non-intrusive event-driven prediction of a metric in a data processing environment is provided in the illustrative embodiments. At least one set of events is observed in the data processing environment, the set of events being generated by several processes executing in the data processing environment. A subset of the set of events are tracked for an observation period, the tracking resulting in bookkeeping information about the subset of events. A pattern of events is detected in the bookkeeping information. The pattern is formed as a tuple representing a process in the several processes, the metric corresponding to the process. A prediction model is selected for the tuple. The prediction model is supplied with the tuple and executed to generate a predicted value of the metric.

Non-intrusive event-driven prediction

A process of transforming data residing in databases, such as relational databases, into forms suitable as input to data analysis tools, such as predictive modeling tools includes the steps of defining a business process problem to be solved and identifying data requirements. For example, the business process problem may relate to predicting a customer's propensity to make purchases in the future or a store's requirements for inventory in the future. In the process, a computer implemented method is used for automatically transforming data for data analysis such as predictive modeling. Database metadata that describe database tables, their interrelationships, dimensional information, fact tables and measures are accessed. A mining transformation profile is created to encapsulate aggregations and transformation on data stored in relational databases in order to convert the data to forms suitable for predictive mining tools. The mining transformation profile specifies data transformations relative to the data base metadata. Executable data transformation codes is then generated from the database metadata and the mining transformation profile. Execution of this code results in aggregation and transformation of data residing in a database for input to a data analysis tool such as a predictive modeling tool. The data transformation code can be used by, for example, the predictive modeling tool to generate an output that provides a solution to a business process problem.

System and method of transforming data for use in data analysis tools

A method, system, and computer usable program product for non-intrusive event-driven prediction of a metric in a data processing environment are provided in the illustrative embodiments. At least one set of events is observed in the data processing environment, the set of events being generated by several processes executing in the data processing environment. A subset of the set of events are tracked for an observation period, the tracking resulting in bookkeeping information about the subset of events. A pattern of events is detected in the bookkeeping information. The pattern is formed as a tuple representing a process in the several processes, the metric corresponding to the process. A prediction model is selected for the tuple. The prediction model is supplied with the tuple and executed to generate a predicted value of the metric.

A list of reference terms can be provided. Text and the list of reference terms can be broken down into tokens. At least one candidate can be generated in the text for mapping to at least one of the reference terms. Characters of the candidate can be compared to characters of the reference term according to one or more mapping rules. A confidence value of the mapping can be generated based on the comparison of characters. Candidates can be ranked according to their confidence value.

Text analysis method for finding acronyms

Methods and apparatus, including computer program products, implementing and using techniques for compressing data included in several transactions. Each transaction has at least one item. A unique identifier is assigned to each different item and, if taxonomy is defined, to each different taxonomy parent. Sets of transactions are formed from the several transactions. The sets of transactions are stored using a computer data structure including: a list of identifiers of different items in the set of transactions, information indicating number of identifiers in the list, and bit field information indicating presence of the different items in the set of transactions, said bit field information being organized in accordance with the list for facilitating evaluation of patterns with respect to the set of transactions. A data structure for compressing data included in a set of transactions is also provided.

Input data structure for data mining

Embodiments of the invention provide a method for detecting changes in behavior of authorized users of computer resources and reporting the detected changes to the relevant individuals. The method includes evaluating actions performed by each user against user behavioral models and business rules. As a result of the analysis, a subset of users may be identified and reported as having unusual or suspicious behavior. In response, the management may provide feedback indicating that the user behavior is due to the normal expected business needs or that the behavior warrants further review. The management feedback is available for use by machine learning algorithms to improve the analysis of user actions over time. Consequently, investigation of user actions regarding computer resources is facilitated and data loss is prevented more efficiently relative to the prior art approaches with only minimal disruption to the ongoing business processes.

Modeling user access to computer resources

The invention relates to a method for mapping at least one data column from a database source to at least one data column of a data target, the method comprising: defining at least one reference column of the data target and at least one database source column; performing a comparison of data contained in the data column(s) with the reference column(s); and determining mapping candidates between the data column(s) and the reference column(s).

Growing community of inventors

Herrenberg, Germany

Christoph Lingenfelder