Equifax Inc.

Vickey Chang

Jeffrey Q Ouyang

Lefei Liu

Peter Gao

Jeffrey Feng

Peter Liu

Zhenyu Wang

Jiawei Liu

Trevis J Litherland

Wei Zhang

Rupesh Patel

Jeffrey Ouyang

Yujun Chen

Certain aspects involve evaluating modeling algorithms whose outputs can impact machine-implemented operating environments. For instance, a computing system generates, from a comparison of a set of estimated attribute values of an attribute to a set of validation attribute values of the attribute, a discretized evaluation dataset with data values in multiple categories. The computing system computes, for a modeling algorithm used to generate the estimated attribute values, an evaluation metric. The computing system provides a host computing system with access to the evaluation metric, one or more modeling outputs generated with the modeling algorithm, or both. Providing one or more of these outputs to the host computing system can facilitate modifying one or more machine-implemented operations.

Evaluation of modeling algorithms with continuous outputs

Certain aspects involve providing automated performance monitoring of statistical models. For example, a processing device is used for performing a statistical analysis on information in an archive to extract historical data, scores, and attributes. The processing device calculates performance metrics based at least in part on the historical data, scores, and attributes. The processing device pre-calculates summary performance data based at least in part on the performance metrics. The summary performance data is stored in files with predefined layouts, which are stored in a non-transitory, computer-readable medium. Segmented data is presented from a file to a user through a graphical user interface (GUI). In some aspects, various reports of the segmented data are presented interactively by detecting a selection by the user of a segmentation and displaying the corresponding segmented data.

Interactive model performance monitoring

Certain aspects involve updating data structures to indicate relationships between attribute trends and response variables used for training automated modeling systems. For example, a data structure stores data for training an automated modeling algorithm. The training data includes attribute values for multiple entities over a time period. A computing system generates, for each entity, at least one trend attribute that is a function of a respective time series of attribute values. The computing system modifies the data structure to include the generated trend attributes and updates the training data to include trend attribute values for the trend attributes. The computing system trains the automated modeling algorithm with the trend attribute values from the data structure. In some aspects, trend attributes are generated by applying a frequency transform to a time series of attribute values and selecting, as trend attributes, some of the coefficients generated by the frequency transform.

Updating attribute data structures to indicate trends in attribute data provided to automated modeling systems

An automated model development tool can be used for automatically developing a model (e.g., an analytical model). The automated model development tool can perform various automated operations for automatically developing the model including, for example, performing automated operations on variables in a data set that can be used to develop the model. The automated operations can include automatically analyzing the predictor variables. The automated operations can also include automatically binning (e.g., combining) data associated with the predictor variables to provide monotonicity between the predictor variables and one or more output variables. The automated operations can further include automatically reducing the number of predictor variables in the data set and using the reduced number of predictor variables to develop the analytical model. The model developed using the automated model development tool can be used to identify relationships between predictor variables and one or more output variables in various machine learning applications.

Automated model development process

Growing community of inventors

Suwanee, GA, United States of America

Vickey Chang