International Business Machines Corporation

Naoki Abe

Aurelie C Lozano

John Steven Langford

Edwin Peter Dawson Pednault

David L Jensen

Edwin P D Pednault

Philip Shilung Yu

Jun Tanaka

Bishwaranjan Bhattacharjee

Charles Philippe Tresser

Sriram Kolinjivadi Padmanabhan

Kenneth Alan Goldman

Tsuyoshi Ide

Jiri Navratil

Amit Dhurandhar

Bonnie Kathryn Ray

Yan Liu

Shubir Kapoor

Shilpa N Mahatma

Srujana Merugu

Hideto Horikoshi

Richard D Lawrence

Bianca Zadrozny

Grzegorz Michal Swirszcz

Moninder Singh

Matthias Gruetzner

Fateh A Tipu

Chidanand V Apte

Tomoki Maruichi

Prem Melville

Justin Wai-chow Wong

Georgios Kollias

James J Bennett

Jinwoo Shin

Cezar Pendus

Saharon Rosset

Vincent Philip Thomas

Kimberly C Lang

Carl E Abrams

Edwin Peter Pednault

Jing Fu

Floyd S Kelly

David M Loehr

Chandan Karrem Reddy

Matthew A Hilbert

Kathleen M Troidle

Michael G Gemmell

Techniques for causal modeling. Historical feature data is received, relating to a plurality of nodes in a system. A machine learning (ML) model is generated, for a node of the plurality of nodes. The ML model is trained to predict a plurality of future feature values for the node based on at least a portion of the historical feature data. A causal graph is generated, for the plurality of nodes, using a feature selection mechanism within the ML model. The feature selection mechanism includes a regularization term encouraging sparsity of nodes in selected features in the ML model.

Non-linear causal modeling from diverse data sources

A computer-implemented method, a computer program product, and a computer system for diagnosing anomalies detected by a black-box machine learning model. A computer determines a local variance of a test sample in a test dataset, where the local variance represents uncertainty of a prediction by the black-box machine learning model. The computer initializes optimal compensations for the test sample, where the optimal compensations are optimal perturbations to test sample values of respective components of a multivariate input variable. The computer determines local gradients for the test sample. Based on the local variance and the local gradients, the computer updates the optimal compensations until convergences of the optimal compensations are reached. Using the optimal compensations, the computer diagnoses the anomalies detected by the black-box machine learning model.

Diagnosing anomalies detected by black-box machine learning models

A method for anomaly alarm consolidation includes detecting a plurality of anomalies in time-series data received from an information technology infrastructure; identifying a plurality of root-cause candidates for each of the anomalies; generating, by a scenario analysis of the anomalies, a plurality of alarms, wherein the scenario analysis predicts a plurality of future expected values of the time-series data over a plurality of historical values of the time-series data using a graphical Granger causal model and generates the alarms based on a difference between the future expected values of the time-series data and actual values of the anomalies in the time-series data; and performing a belief propagation procedure between the root-cause candidates and the alarms to determine a plurality of root-causes that collectively comprise attributed root-causes for the alarms.

Consolidating anomaly root causes and alarms using graphical granger models

A computing system initializes a first frontier to be a root of a multi-dimensional hierarchical data structure representing an entity. The system acquires first data corresponding to the first frontier. The system performs modeling on the first data to obtain a first model and a corresponding first statistic. The system expands a dimension of the first frontier. The system gathers second data corresponding to the expanded frontier. The system applies the data modeling on the second data to obtain a second model and a corresponding second statistic. The system compares the first statistic of the first model and the second statistic of the second model. The system sets the second model to be the first model in response to determining that the second model statistic is better than the first model statistic. The system outputs the first model.

Casual modeling of multi-dimensional hierarchical metric cubes

Various embodiments for maintaining security and confidentiality of data and operations within a fraud detection system. Each of these embodiments utilizes a secure architecture in which: (1) access to data is limited to only approved or authorized entities; (2) confidential details in received data can be readily identified and concealed; and (3) confidential details that have become non-confidential can be identified and exposed.

Confidential fraud detection system and method

A method of marketing optimization with respect to brand lifetime management formulates a problem of brand equity maximization utilizing Markov Decision Process (MDP) thereby casting brand equity management as a long term regard optimization problem in MDP, The marketing mix is optimized by formulating the mix as actions in MDP and, utilizing historical marketing and transaction data, aspects of the MDP are estimated.

System and method for marketing mix optimization for brand equity management

A system and method for sequential decision-making for customer relationship management includes providing customer data including stimulus-response history data, and automatically generating actionable rules based on the customer data. Further, automatically generating actionable rules may include estimating a value function using reinforcement learning.

System and method for sequential decision making for customer relationship management

Multi-class cost-sensitive boosting based on gradient boosting with 'p-norm' cost functionals' uses iterative example weighting schemes derived with respect to cost functionals, and a binary classification algorithm. Weighted sampling is iteratively applied from an expanded data set obtained by enhancing each example in the original data set with as many data points as there are possible labels for any single instance, and where each non-optimally labeled example is given the weight equaling a half times the original misclassification cost for the labeled example times the p&#x2212;1 norm of the average prediction of the current hypotheses. Each optimally labeled example is given the weight equaling the sum of the weights for all the non-optimally labeled examples for the same instance. Component classification algorithm is executed on a modified binary classification problem. A classifier hypothesis is output, which is the average of all the hypotheses output in the respective iterations.

Methods and systems for cost-sensitive boosting

A 'variable group selection' system and method in which constructs are based upon a training data set, a regression modeling module that takes into account information on groups of related predictor variables given as input and outputs a regression model with selected variable groups. Optionally, the method can be employed as a component in methods of temporal causal modeling, which are applied on a time series training data set, and output a model of causal relationships between the multiple times series in the data.

Methods and systems for variable group selection and temporal causal modeling

A method (and system) for causal modeling includes modeling a data set. The modeling includes estimating a reverse Bayesian forest for the data set and detecting outliers in a separate data set. Detecting the outliers includes applying the reverse Bayesian forest to the separate data set to obtain a probability value assigned to data points in the separate data set and identifying outliers in the separate data set by evaluating the probability value given by the reverse Bayesian forest.

Method and system for causal modeling and outlier detection

Outlier detection methods and apparatus have light computational resources requirement, especially on the storage requirement, and yet achieve a state-of-the-art predictive performance. The outlier detection problem is first reduced to that of a classification learning problem, and then selective sampling based on uncertainty of prediction is applied to further reduce the amount of data required for data analysis, resulting in enhanced predictive performance. The reduction to classification essentially consists in using the unlabeled normal data as positive examples, and randomly generated synthesized examples as negative examples. Application of selective sampling makes use of an underlying, arbitrary classification learning algorithm, the data labeled by the above procedure, and proceeds iteratively. Each iteration consisting of selection of a smaller sub-sample from the input data, training of the underlying classification algorithm with the selected data, and storing the classifier output by the classification algorithm. The selection is done by essentially choosing examples that are harder to classify with the classifiers obtained in the preceding iterations. The final output hypothesis is a voting function of the classifiers obtained in the iterations of the above procedure.

Resource-light method and apparatus for outlier detection

Feature importance information available in a predictive model with correlation information among the variables is presented to facilitate more flexible choices of actions by business managers. The displayed feature importance information combines feature importance information available in a predictive model with correlational information among the variables. The displayed feature importance information may be presented as a network structure among the variables as a graph, and regression coefficients of the variables indicated on the corresponding nodes in the graph. To generate the display, a regression engine is called on a set of training data that outputs importance measures for the explanatory variables for predicting the target variable. A graphical model structural learning module is called that outputs a graph on the explanatory variables of the above regression problem representing the correlational structure among them. The feature importance measure, output by the regression engine, is displayed for each node in the graph, as an attribute, such as color, size, texture, etc, of that node in the graph output by the graphical model structural learning module.

Method and apparatus for presenting feature importance in predictive modeling

Methods for multi-class cost-sensitive learning are based on iterative example weighting schemes and solve multi-class cost-sensitive learning problems using a binary classification algorithm. One of the methods works by iteratively applying weighted sampling from an expanded data set, which is obtained by enhancing each example in the original data set with as many data points as there are possible labels for any single instance, using a weighting scheme which gives each labeled example the weight specified as the difference between the average cost on that instance by the averaged hypotheses from the iterations so far and the misclassification cost associated with the label in the labeled example in question. It then calls the component classification algorithm on a modified binary classification problem in which each example is itself already a labeled pair, and its (meta) label is 1 or 0 depending on whether the example weight in the above weighting scheme is positive or negative, respectively. It then finally outputs a classifier hypothesis which is the average of all the hypotheses output in the respective iterations.

Methods for multi-class cost-sensitive learning

The present invention employs data processing systems to handle debt collection by formulation the collections process as a Markov Decision Process with constrained resources, thus making it possible automatically to generate an optimal collections policy with respect to maximizing long-term expected return throughout the course of a collections process, subject to constraints on the available resources possibly in multiple organizations. This is accomplished by coupling data modeling and resource optimization within the constrained Markov Decision Process formulation and generating optimized rules based on constrained reinforcement learning process comprising applied on the basis of past historical data.

Method and system for debt collection optimization

Growing community of inventors

Rye, NY, United States of America

Naoki Abe