Fair Isaac Corporation

Shafi Ur Rahman

Scott Michael Zoldi

Amit Kiran Sowani

Ashish Kumar Gupta

Michael K Tyler

Eeshan Malhotra

Zhenyu Yan

Gerald Fahner

Nitin Basant

Manmeet Kaur

Sambandan Murugan

Moiz Saifee

Nitan Basant

Robin P

Rakhi Agrawal

A method includes determining, by a trained machine learning model, a score based at least on one or more latent features. The method also includes monitoring the determining of the score by the trained machine learning model. The monitoring includes determining one or more production statistics associated with the one or more latent features, derived variables and input data elements, and accessing one or more reference assets persisted on a model governance blockchain. The one or more reference assets includes one or more reference statistics and a threshold indicating a deviation between the one or more production statistics and the one or more reference statistics. The method also includes generating an alert based on the one or more production statistics associated with the one or more latent features meeting the threshold. Related methods and articles of manufacture are also disclosed.

Blockchain-based model governance and auditable monitoring of machine learning models

Systems for improving security of a computer-implemented artificial intelligence by monitoring one or more transactions received by the machine learning decision model; receiving a first score generated by the machine learning decision model in association with a first transaction; identifying the first transaction as belonging to a first class, in response to the first score being lower than a certain score threshold and the first transaction having a low occurrence likelihood; receiving a second score in association with the first transaction based on one or more adversarial latent features associated with the first transaction as detectable by an adversary detection model; and determining at least one adversarial latent transaction feature being exploited by the first transaction, in response to determining that the second score falls above the certain score threshold.

Overly optimistic data patterns and learned adversarial latent features

A system and method for analyzing coverage, bias and model explanations in large dimensional modeling data includes discretizing three or more variables of a dataset to generate a discretized phase space represented as a grid of a plurality of cells, the dataset comprising a plurality of records, each record of the plurality of records having a value and a unique identifier (ID). A grid transformation is applied to each record in the dataset to assign each record to a cell of the plurality of cells of the grid according to the grid transformation. A grid index is generated to reference each cell using a discretized feature vector. A grid storage for storing the records assigned to each cell of the grid is then created. The grid storage using the ID of each record as a reference to each record and the discretized feature vector as a key to each cell.

Method and apparatus for analyzing coverage, bias, and model explanations in large dimensional modeling data

Systems, methods and computer program products for improving security of artificial intelligence systems. The system comprising processors for monitoring one or more transactions received by a machine learning decision model to determine a first score associated with a first transaction. The first transaction may be identified as likely adversarial, in response to the first score being lower than a certain score threshold and the first transaction having a low occurrence likelihood. A second score may be generated in association with the first transaction based on one or more adversarial latent features associated with the first transaction. At least one adversarial latent feature may be detected as being exploited by the first transaction, in response to determining that the second score falls above the certain score threshold. Accordingly, an abnormal volume of activations of adversarial latent features spanning across a plurality of transactions scored may be detected and blocked.

A system and method for constructing an improved computing model that preserves use rights for data utilized by the model. A first dataset is accessed to build a computing model. The first data set is subject to terminable usage rights provisions. A portion of the first dataset is sampled to generate a second dataset. Vectors present in the first dataset and the second dataset are discretized. In response to determine that the usage rights associated with the primary dataset have been terminated, a coverage depletion for the second dataset is computed based on the usage rights termination associated with the first dataset. An estimated mean time to coverage failure for the first model based on the depletion coverage is determined for the second dataset. One or more data points are removed from the first dataset due to the termination of usage rights.

Building resilient models to address dynamic customer data use rights

Computer-implemented methods and systems for quantifying appropriate machine learning model complexity corresponding to training dataset are provided. The method comprises monitoring, using one or more processors, N observed variables, vthrough v, of a training dataset for a machine learning model; translating the N observed variables into m equisized bin indexes which generate mpossible equisized hypercells to estimate a fundamental dimensionality for the dataset; generating one or more samples by assigning a record in the dataset with numbers j through k as set id; generating a merged sample Si, for one or more values of the set id i, where i goes from j to k; and computing a fractal dimension of the equisized hypercube phase space based on count of cells with data coverage of at least one data point.

Latent feature dimensionality bounds for robust machine learning on high dimensional datasets

A model governance framework uses a shared ledger on the back of a blockchain. The solution tracks various analytic tracking documents (ATDs) and associated assets, like requirements and sprints, through various stages of an ATD lifecycle. Data schema and data distributions are also tracked. The decision models, corresponding variables and execution codes are also tracked. Existing variables and execution codes are made available via a preexisting asset ATD for reuse. Various transactions provide mechanism for accessing and manipulating the various assets through a recorded ledger of events and approvals. A system provides tracking of the approvals and the approvers of all model assets that are touched by any participant, and further provides access control and security for multi-user access. An application layer provides graphical access to the various aspects of the blockchain. Event notification provides the backbone for interfacing with various external systems like email servers and version control systems.

Blockchain for data and model governance

Computer-implemented systems and methods for selecting a first neural network model from a set of neural network models for a first dataset, the first neural network model having a set of predictor variables and a second dataset comprising a plurality of datapoints mapped into a multi-dimensional grid that defines one or more neighborhood data regions; applying the first neural network model on the first dataset to generate a model score for one or more datapoints in the second dataset, the model score representing an optimal fit of input predictor variables to a target variable for the set of variables of the first neural network model.

Density based confidence measures of neural networks for reliable predictions

In transactional systems where past transactions can have impact on the current score of a machine learning based decision model, the transactions that are most responsible for the score and the associated reasons are determined by the transactional system. A system and method identifies such past transactions that maximally impact the current score and allow for a more effective understanding of the scores generated by a model in a transactional system and explanation of specific transactions for automated decisioning, to explain the scores in terms of past transactions. Further an existing instance-based explanation system is used to identify the reasons for the score, and how the identified transactions influence these reasons. A combination of impact on score and impact on reasons determines the most impactful past transaction with respect to the most recent score being explained.

Temporal explanations of machine learning model outcomes

Systems and methods that use a neural network architecture for extracting interpretable relationships among predictive input variables. This leads to neural network models that are interpretable and explainable. More importantly, these systems and methods lead to discovering new interpretable variables that are functions of predictive input variables, which in turn can be extracted as new features and utilized in other types of interpretable models, like scorecards (fraud score, etc.), but with higher predictive power than conventional systems and methods.

System and method for generating explainable latent features of machine learning models

Data for a plurality of entities that can be offered a plurality of products can be obtained. The data can include categorical data and numeric data. Based on business constraints, some of all of the data can be selected. The selected data can be converted to another set of numeric data, wherein the categorical values are converted to numeric values. Dimensions of the converted data can be reduced to generate another set of data. Based on this another set of data, clusters of entities can be formed. The products can be grouped by assigning a unique product identifier of each product to a corresponding cluster. This grouping of products can be used by a predictive model to predict a likelihood of an entity to purchase a particular product in a future time period. Related methods, apparatus, systems, techniques and articles are also described.

Data-driven product grouping

A system and method for learning and associating reliability and confidence corresponding to a model's predictions by examining the support associated with datapoints in the variable phase space in terms of data coverage, and their impact on the weights distribution. The approach disclosed herein examines the impact of minor perturbations on a small fraction of the training exemplars in the variable phase space on the weights to understand whether the weights remain unperturbed or change significantly.

The current subject matter relates to generation of relevant real-time offers based on global positioning system (GPS) data of an individual. A mobile device of an individual can record the GPS data of the individual. The mobile device can be connected to a central system. The central system can receive the recorded GPS data. The central system can predict, by using a trained predictive model and based on transaction history of the individual and the GPS data, categories of likely purchases by the individual. The central system can generate or reproduce offers from merchants of the predicted categories that are located within a threshold distance from a current location of the individual. The central system can send the generated offers to the mobile device that can display the generated offers in real-time. Other applications can include improving relevance of batch offers and/or real-time offers based on a recent purchase trigger.

Provision of relevant offers based on GPS data

The state of a system is determined in which data sets are generated that include a plurality of data instances representing states of one or more components of a computer system. The data instances generated by one or more data set sources that are configured to output a data instance in response to a trigger associated with the one or more components. The data instances are normalized by the application of one or more rules. The data instances from individual data set sources are separately collated to generate groups of time-specific collated data instances. State types may be assigned to each of the collated data instance groups. Distributions of state-types across the groups may be determined and a list of infrequent state-types may be generated based on the determined distributions of state-types across the groups.

Growing community of inventors

San Diego, CA, United States of America

Shafi Ur Rahman