Google Inc.

Gang Fu

Gideon S Mann

Wei-Hao Lin

Robert Kaplow

Travis H K Green

Travis Green

Robbie Alan Haertel

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for obtaining a plurality of model representations of predictive models, each model representation associated with a respective user and expresses a respective predictive model, and selecting a model implementation for each of the model representations based on one or more system usage properties associated with the user associated with the corresponding model representation.

Predictive model importation

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a plurality of different types of predictive models using training data, wherein each of the predictive models implements a different machine learning technique. One or more weights are obtained wherein each weight is associated with an answer category in the plurality of examples. A weighted accuracy is calculated for each of the predictive models using the one or more weights.

Predictive model evaluation and training based on utility

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for score normalization. One of the methods includes receiving initial training data, the initial training data comprising initial training records, each initial training record identifying input data as input and a category as output. The method includes generating a first trained predictive model using the initial training data and a training function. The method includes generating intermediate training records by inputting input data of the initial training records to a second trained predictive model, the second trained predictive model generated using the training function, each intermediate training record having a score. The method also includes generating a score normalization model using a score normalization training function and the intermediate training records.

Normalization of predictive model scores

A system includes a computer(s) coupled to a data storage device(s) that stores a training data repository and a predictive model repository. The training data repository includes retained data samples from initial training data and from previously received data sets. The predictive model repository includes at least one updateable trained predictive model that was trained with the initial training data and retrained with the previously received data sets. A new data set is received. A richness score is assigned to each of the data samples in the set and to the retained data samples that indicates how information rich a data sample is for determining accuracy of the trained predictive model. A set of test data is selected based on ranking by richness score the retained data samples and the new data set. The trained predictive model is accuracy tested using the test data and an accuracy score determined.

Assessing accuracy of trained predictive models

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for utilizing predictive models from an application scripting language.

Predictive model application programming interface

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for receiving a plurality of first training examples, training a first predictive model using the first training examples, for each example in the first training examples, providing the first features of the example to the trained first predictive model to generate a respective first prediction, generating a second training example for each of the first training examples, wherein the second training example comprises the first features of the first training example and an answer that indicates whether the first answer of the first training example matches the respective first prediction of the first training example, training a second predictive model using the second training examples, and using the trained second predictive model to determine a confidence score for a prediction generated by the trained first predictive model.

Confidence scoring in predictive modeling

In general, a method includes receiving a training data set that includes a plurality of examples, wherein each example includes one or more features and an answer, generating a plurality of modified training data sets by applying one or more filters to the training data set, each of the plurality of modified training data sets being based on a different combination of the one or more filters, training a plurality of predictive models, each of the plurality of predictive models being trained using a different modified training data set of the plurality of modified training data sets, determining a respective accuracy for each of the plurality of predictive models, identifying a most accurate predictive model based on the determined accuracies, and specifying an association between the training data set and the combination of filters used to generate the modified training data set that was used to train the most accurate predictive model.

Predictive modeling accuracy

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for multi-label models. One of the methods includes receiving training records, each training record having an input, a first output, and a second output. The method includes generating a first classifier using as input one of the inputs and using as output a corresponding one of the first outputs. The method includes generating a second classifier using as input one of the inputs and using as output a corresponding one of the second outputs. The method includes inputting the inputs into the first classifier and generating first predictive outputs. The method includes inputting the inputs into the second classifier and generating second predictive outputs. The method also includes generating a third classifier using as input the first output and the second output and using as output the first output and the second output of the corresponding training record.

Multi-label modeling using a plurality of classifiers

A computer-implemented method includes obtaining a database table, the database table including data arranged in a plurality of rows and a plurality of columns, each column of data being associated with a different tag that specifies a category for data in the column, using one or more processors to identify a first predictive model, from a collection of predictive models, that can be applied to the database table to generate a predictive output, in which identifying the first predictive model is based on one or more of the different tags, adding a name associated with the first predictive model to a set of names of predictive models that are compatible with the database table, and providing the set of names of predictive models to a client device.

Predictive analytical model selection

Methods, systems, and apparatus, including computer programs encoded on one or more computer storage devices, for caching predictive models are described. Records are obtained, each record including a time of a previously submitted predictive request and an identifier of a trained predictive model. A trained scheduling model is generated using the records as training data. A set of identifiers of trained predictive models are determined from a plurality of trained predictive models that are stored in a secondary memory of a computing system. The target time is inputted to the trained scheduling model. In response, a second predictive output is received that comprises the set of identifiers. A set of trained predictive models are obtained that correspond to the set of identifiers from the secondary memory. The set of trained predictive models are stored in a primary memory of the computing system.

Growing community of inventors

Kearny, NJ, United States of America

Gang Fu