Northwestern University

Zestfinance, Inc.

Perfect Market, Inc.

Jerome Louis Budzik

Lawrence A Birnbaum

Kristian J Hammond

Sean Javad Kamkar

John Wickens Lamb Merrill

Sanjay Chand Sood

Andrew Joseph Crossen

Jose Efrain Valentin

Mark Frederick Eberstein

Feng Li

John Candido

Douglas C Merrill

John Joseph Beahan, Jr

Yachen Yan

Esfandiar Alizadeh

Derek Wilcox

Ozan Sayin

Jiahuan He

Benjamin Anthony Solecki

Michael Edward Ruberry

Geoffrey Michael Ward

Vishwaesh Rajiv

Evan George Kriminger

Armen Avedis Donigian

Michael Egan Van Veen

Qiumeng Yu

Adam Kleinman

Bojan Tunguz

Ruochen Zha

Lingzhi Du

Saar Yalov

David Sheehan

Melanie Eunique DeBruin

Bingjia Wang

Yunyan Zhang

Randolph Paul Sinnott, Jr

Liubo Li

Carlos Alberta Huertas Villegas

Eran Dvir

Siavash Yasani

Lin Song

Siavash Yasini

Mark Dredze

Kevin Livingston

John Wickens Merrill

This application describes systems and methods for generating machine learning models (MLMs). An exemplary method includes obtaining a sample and user input data characterizing a product or service. A subset of the data is selected from the sample based on sampling the sample according to the user input data. An MLM is trained by applying the data subset as training input to the MLM, thereby providing a trained MLM to emulate a customer selection process unique to the product or service. A user interface (UI) configured to receive other user input data and cause the trained MLM to execute on the other user input data, thereby testing the trained MLM, is presented. A summary of results from the execution of the trained MLM is generated and presented in the UI. The summary of results indicates a contribution to the trained MLM of each of a plurality of features.

Automated systems for machine learning model development, analysis, and refinement

Systems and methods for generating and processing modeling workflows are disclosed. In some examples, a reference distribution of scores generated by a model is determined. The reference distribution of scores is recorded in a structured database. One or more unexpected scores are detected during execution of the model. To detect the one or more unexpected scores, a production distribution of scores is compared with the reference distribution of scores recorded in the structured database. The production distribution of scores is generated by the model for a production input data set. An alert is then provided to an external system, when an alert condition is determined to be satisfied based on the comparison. The alert indicates detection of the one or more unexpected scores.

Systems and methods for enriching modeling tools and infrastructure with semantics

Systems and methods for training models to improve fairness.

Systems and methods for model fairness

Methods, non-transitory computer readable media, and model evaluations systems for understanding diverse machine learning models (MLMs) are disclosed. In some examples, a feature contribution value is determined for features included in a reference or evaluation input data set. The evaluation input data set represents a protected class population and each feature contribution value identifies a contribution by a feature to a difference in output generated by an MLM for the evaluation input data set. Model explanation information is generated using the feature contribution values and execution of the MLM is monitored. The model explanation information explains the difference in output generated by the MLM for the evaluation input data set and includes information relating to a model-based decision. A report is generated from a knowledge graph for the MLM and output via a GUI to an operator device that includes the model explanation information.

Systems and methods for decomposition of differentiable and non-differentiable models

Systems and methods for generating tree-based models with improved fairness are disclosed. The disclosed process generates a first tree-based machine learning model, which is preferably trained to predict if a financial loan will be repaid. The process also determines an accuracy of the first tree-based machine learning mode. In addition, the process determines a fairness of the first tree-based machine learning model. The fairness is preferably associated with at least one of gender, race, ethnicity, age, marital status, military status, sexual orientation, and disability status. The process then generates a second different tree-based machine learning model, which is preferably trained based on the accuracy of the first tree-based machine learning model and the fairness of the first tree-based machine learning model. The process then combines the first tree-based machine learning model and the second tree-based machine learning model to produce a gradient-boosted machine learning model.

Systems and methods for generating gradient-boosted models with improved fairness

Systems and methods for model evaluation. A model is evaluated by performing a decomposition process for a model output, relative to a baseline input data set.

Systems and methods for providing machine learning model evaluation by using decomposition

Systems and methods for model evaluation. A protected class model that satisfies an accuracy threshold is built by using: data sets for use by a modeling system being evaluated, and protected class membership information for each data set. A target for the protected class model is a protected class membership variable indicating membership in a protected class. Each predictor of the protected class model is a predictor of an evaluated model used by the modeling system. A target of the evaluated model is different from the target of the protected class model. Each predictor is a set of one or more variables of the data sets. For each predictor of the protected class model, a protected class model impact ranking value and a modeling system impact ranking value are determined.

Explainable machine learning financial credit approval model for protected classes of borrowers

Systems and methods for generating and processing modeling workflows.

Systems and methods for understanding diverse machine learning models.

An information retrieval system for automatically retrieving information related to the context of an active task being manipulated by a user. The system observes the operation of the active task and user interactions, and utilizes predetermined criteria to generate context representation of the active task that are relevant to the context of the active task. The information retrieval system then processes the context representation to generate queries or search terms for conducting an information search. The information retrieval system determines the relevance of a word to the context by utilizing an adaptive weighting system. The information retrieval system assigns varying weights to different attributes of a word and calculates an accumulated weight of the word by accumulating all weights assigned to the word. The attributes may include word size, style, location of the word, etc. The system then ranks the importance of words based on their respective accumulated weight, and chooses words that rank within a predetermined number from the top to form search terms to conduct an information search using various data sources.

Method and system for assessing relevant properties of work contexts for use by information services

Growing community of inventors

Altadena, CA, United States of America

Jerome Louis Budzik