Fair Isaac Corporation

Other

Fair Issac Corporation

Scott Michael Zoldi

Shafi Ur Rahman

Matthew Kennel

Steven Wu

Yuting Jia

Hua Li

Larry Peranich

Uwe Mayer

Li Hsiang Sun

Robert D Holmes

Joseph F Murray

Maria Edna Derderian

Jehangir Athwal

Alex Barclay

Dirk Englund

Dimpy Pathria

Tim Roach

Justin Lawyer

Jun Zhang

Shane De Zilwa

Michael P Balon

Jeremy Mamer Schmitt

Anthony Vaiciulis

Xinwei Xue

Heming Xu

Yonghui Chen

Gerald Fahner

Gregory Gancarz

Liang Wang

Ilya Levner

Daniel Shoham

Min Shao

Radu Drossu

David Griegel

Michael Urban

Gordon Cameron

Sridhar Maddipati

Fei Chen

Jim Coggeshall

Prodip Hore

Ron Martin

Jenny (Guofeng) Zhang

Qing Liu

Jeffrey D Carlson

Xiang Yang Li

Dayne B Freitag

Matthias Blume

Shailesh Kumar

Qing Lin

Richard Rohwer

Amit Kiran Sowani

Jianjun Xie

Chahm An

Stuart Clarkson Wells

Kiyoung Yang

Vesselin Diev

Richard Calmbach

Sidakpreet Singh Chawla

Julie A Haag

Maziar Yaesoubi

Keerthi Kancherla

Yuchen Chen

Surjit Singh

Todd Smith

Hila Hashemi

James Coggeshall

Todd Alan Smith

Sajama

Ujjwal Kumar

Xinwai Xue

Alexei Betin

Gabriel Paul Hopkins

Matthew Bochner Kennel

A method includes determining, by a trained machine learning model, a score based at least on one or more latent features. The method also includes monitoring the determining of the score by the trained machine learning model. The monitoring includes determining one or more production statistics associated with the one or more latent features, derived variables and input data elements, and accessing one or more reference assets persisted on a model governance blockchain. The one or more reference assets includes one or more reference statistics and a threshold indicating a deviation between the one or more production statistics and the one or more reference statistics. The method also includes generating an alert based on the one or more production statistics associated with the one or more latent features meeting the threshold. Related methods and articles of manufacture are also disclosed.

Blockchain-based model governance and auditable monitoring of machine learning models

Systems, methods and products for quantitative translation of design requirements into a machine learning framework for training a classification model. A plurality of auxiliary tasks associated with a plurality of auxiliary task models are specified. The plurality of auxiliary task models are concurrently trained on the auxiliary tasks to generate one or more latent features learned by the plurality of auxiliary task models. The one or more latent features may be transferred from the plurality of auxiliary task models to augment a latent feature space of a target task for the classification model. Contribution levels of the transferred one or more latent features are adjusted based on design requirements for the target task for the classification model. First and second contribution levels are specified for respective first and second sets of auxiliary task latent features being quantified and enforced.

Constrained designed and guided latent feature space contributions to machine learning models

Computer-implemented method and systems to improve training and performance of artificial intelligence (AI) systems having one or more machine learning models stored in one or more data storage mediums connected in at least one computing network is provided. The method comprises receiving student model scores, generated by a student machine learning model stored in a data storage medium, the student machine learning model having a primary loss function; receiving teacher scores provided by one or more analytic resources, the teacher scores being provided based on known results and behavior of pre-existing machine learning models used for accomplishing a first series of classification objectives; transforming the teacher scores into transformed teacher scores.

Rank distillation for training supervised machine learning models

Computer-implemented machines, systems and methods for providing insights about uncertainty of a machine learning model. A method includes determining an uncertainty value associated with a first machine learning model output of a first machine learning model. The method further includes generating a confidence interval for the first machine learning model output associated with an input. The method further includes switching, responsive to the uncertainty value satisfying a threshold, from the first machine learning model to a second machine learning model, the second machine learning model generating a second machine learning model output. The method further includes generating the second machine learning model. The method further includes providing, responsive to the switching, the machine learning output, the uncertainty value, the confidence interval, and the second machine learning output to a user interface.

Machine learning uncertainty quantification and modification

A predictive analytics system and method in the setting of multi-class classification are disclosed, for identifying systematic changes in an evaluation dataset processed by a fraud-detection model by examining the time series histories of an ensemble of entities such as accounts. The ensemble of entities is examined and processed both individually and in aggregate, via a set of features determined previously using a distinct training dataset. The specific set of features in question may be calculated from the entity's time series history, and may or may not be used by the model to perform the classification. Certain properties of the detected changes are measured and used to improve the efficacy of the predictive model.

Method for real-time enhancement of a predictive algorithm by a novel measurement of concept drift using algorithmically-generated features

Systems and methods for training a machine learning model implemented over a network configured to represent the machine learning model are provided. At least one or more directed edges connect the one or more nodes an edge representing a connection between a first node and a second node, the second node computing an activation depending on the values of activations on first nodes and values associated with the connections, the connection being either conforming or non-conforming. The machine learning model may be trained by iteratively adjusting parameters w and b, respectively associated with weights and biases associated with edges connecting computational nodes. Connections between nodes may be sparsified by adjusting the parameter w to a first value for non-conforming connections during the training phase to reduce complexity of the connections among the plurality of nodes, or to ensure the input-output function of the network adheres to additional constraints.

Training artificial neural networks with constraints

Systems and methods for providing insights about a machine learning model are provided. The method includes, using training data to train the machine learning model to learn patterns to determine whether data associated with an event provides an indication that the event belongs to a certain class from among a plurality of classes, evaluating one or more features of the machine learning model to produce a data set pairing observed scores S and a set of predictive input variables V, and constructing at least one data-driven estimator based on an explanatory statistic, the estimator being represented in a computationally efficient form and packaged with the machine learning model and utilized to provide a definition of explainability for a score generated by the machine learning model.

Attributing reasons to predictive model scores with local mutual information

A method includes generating a plurality of binary feature maps containing a set of feature map values including a first binary value and/or a second binary value, by at least converting each input value of a set of input values of a plurality of input feature vectors to the first binary value when the corresponding input value is the zero value or the second binary value when the corresponding input value is the non-zero value. The method includes segmenting the plurality of binary feature maps into a plurality of segments representing behavior profiles. Each segment includes at least one subsegment in which the set of feature map values is the same for all binary feature maps in the at least one subsegment. The method includes predicting, based on a segment of the plurality of segments, a specific outcome. Related methods and articles of manufacture are also disclosed.

Segmentation using zero value features in machine learning

Systems for improving security of a computer-implemented artificial intelligence by monitoring one or more transactions received by the machine learning decision model; receiving a first score generated by the machine learning decision model in association with a first transaction; identifying the first transaction as belonging to a first class, in response to the first score being lower than a certain score threshold and the first transaction having a low occurrence likelihood; receiving a second score in association with the first transaction based on one or more adversarial latent features associated with the first transaction as detectable by an adversary detection model; and determining at least one adversarial latent transaction feature being exploited by the first transaction, in response to determining that the second score falls above the certain score threshold.

Overly optimistic data patterns and learned adversarial latent features

Computer-implemented machines, systems and methods for providing insights about misalignment in a latent space of a machine learning model. A method includes initializing a second weight matrix of a second artificial neural network based on a first weight matrix from a first artificial neural network. The method further includes applying transfer learning between the first artificial neural network and the second artificial neural network. The method further includes comparing the first latent space with the second latent space. The method further includes determining, responsive to the comparing, a first score indicating alignment of the first latent space and the second latent space. The method further includes determining, and responsive to the first score satisfying a threshold, an appropriateness of the machine learning model.

Latent-space misalignment measure of responsible AI for machine learning models

A system and method for analyzing coverage, bias and model explanations in large dimensional modeling data includes discretizing three or more variables of a dataset to generate a discretized phase space represented as a grid of a plurality of cells, the dataset comprising a plurality of records, each record of the plurality of records having a value and a unique identifier (ID). A grid transformation is applied to each record in the dataset to assign each record to a cell of the plurality of cells of the grid according to the grid transformation. A grid index is generated to reference each cell using a discretized feature vector. A grid storage for storing the records assigned to each cell of the grid is then created. The grid storage using the ID of each record as a reference to each record and the discretized feature vector as a key to each cell.

Method and apparatus for analyzing coverage, bias, and model explanations in large dimensional modeling data

Attributing reasons to predictive model scores

A diagnostic system for model governance is presented. The diagnostic system includes an auto-encoder to monitor model suitability for both supervised and unsupervised models. When applied to unsupervised models, the diagnostic system can provide a reliable indication on model degradation and recommendation on model rebuild. When applied to supervised models, the diagnostic system can determine the most appropriate model for the client based on a reconstruction error of a trained auto-encoder for each associated model. An auto-encoder can determine outliers among subpopulations of consumers, as well as support model go-live inspections.

Auto-encoder enhanced self-diagnostic components for model monitoring

Systems, methods and computer program products for improving security of artificial intelligence systems. The system comprising processors for monitoring one or more transactions received by a machine learning decision model to determine a first score associated with a first transaction. The first transaction may be identified as likely adversarial, in response to the first score being lower than a certain score threshold and the first transaction having a low occurrence likelihood. A second score may be generated in association with the first transaction based on one or more adversarial latent features associated with the first transaction. At least one adversarial latent feature may be detected as being exploited by the first transaction, in response to determining that the second score falls above the certain score threshold. Accordingly, an abnormal volume of activations of adversarial latent features spanning across a plurality of transactions scored may be detected and blocked.

Systems and methods for generating concise explanations of scored observations that strike good, and computationally efficient, trade-offs between rank-ordering performance and explainability of scored observations are disclosed. The systems and methods described herein for explaining scored observations are based on a framework of partial dependence functions (PDFs), multi-layered neural networks (MNNs), and Latent Explanations Neural Network Scoring (LENNS).

Fast automatic explanation of scored observations

Growing community of inventors

San Diego, CA, United States of America

Scott Michael Zoldi