Wells Fargo Bank, N.a.

Weicheng Liu

Jie Chen

Agus Sudjianto

Vijayan Narayana Nair

Kevin D Oden

Gabriela Garcia

Hope Ann Staroselsky

Systems and methods for operating an artificial intelligence and machine learning model to generate annotations of data and to generate training datasets is disclosed. One disclosed system includes one or more processors configured to: assign a task to a machine learning model; receive an output from the machine learning model associated with the task; compare the output to task data associated with a user performing the assigned task; and when there is a difference between the output of the machine learning model and the task data, generate an annotation.

Systems and methods for adversarial annotations

The innovation disclosed and claimed herein, in one aspect thereof, comprises systems and methods of validating models guided by machine learning algorithms. The innovation can begin by receiving a risk model for validation having multiple sets of data. A first data set is selected from as an input. Outputs are generated for validation. One output can be generating a second set of analysis results using a comparable algorithm to the risk model. Another output can be generating a second set of variables and transformations using a machine learning algorithm and an untransformed set of the selected variables to assess the set of selected transformations. Another output can be generating a third set of variables using one or more machine learning algorithms and an extended feature set of variables to assess the selected variables. The outputs are compared to the analysis results, coefficients, selected variables, and selected transformations. A report of the comparison is generated.

Model validation of credit risk

Dynamic predictive modelling is disclosed. A subset of joined pairs can be selected from a table of joined pairs in which each pair includes snapshot time and forecast time, the joined pairs form a forecasting triangle that spans predetermined increments of time, and the subset of joined pairs comprises a vector in the forecasting triangle. A predictive model can be utilized to operate over the data associated with the subset of joined pairs and produce a prediction. The subset of joined pairs can be dependent on the predictive model. In one instance, data associated with the subset of pairs can be loaded into processor cache to facilitate efficient processing.

Dynamic prediction modeling

The innovation disclosed and claimed herein, in one aspect thereof, comprises systems and methods of validating models guided by machine learning algorithms. The innovation can begin by receiving a risk model for validation having multiple sets of data. A first data set is selected from as an input. Outputs are generated for validation. One output can be generating a second set of analysis results using a comparable algorithm to the risk model. Another output can be generating a second set of variables and transformations using a machine learning algorithm and an un-transformed set of the selected variables to assess the set of selected transformations. Another output can be generating a third set of variables using one or more machine learning algorithms and an extended feature set of variables to assess the selected variables. The outputs are compared to the analysis results, coefficients, selected variables, and selected transformations. A report of the comparison is generated.

Systems, methods, and other embodiments associated with dynamic predictive modelling. According to one embodiment, a method includes creating a joined pair including a snapshot time and the forecast time. The joined pair is stored in the storage device. A subset of data associated with the joined pair is selected from data stored in the storage device. The subset of data is selected based, at least in part, on predetermined increments of time between the snapshot time and the forecast time. The snapshot time, the forecast time, and the subset of data is provided to a processing device from the storage device.

Growing community of inventors

Winston Salem, NC, United States of America

Weicheng Liu