Google Inc.

Qwest Communications International Inc.

Jon Vaver

Stefan F Schnabl

Wenjie Jiang

John Huang

Ruixue Fan

Stephanie Zhang

Stephanie Sapp

Arjun Satyapal

Systems and methods for model validation includes generating a first and a second time series of segmentation states for a data set representative of a simulated population, e.g., a collection of membership counts corresponding to respective segments of the simulated population. The first and second time series of segmentation states are generated by respectively processing the data set through a first and a second simulation each comprising iterative application of a plurality of event functions. The first and the second simulation differ in at least one capacity, e.g., one including a first event function configured with a first parameter, and the second not. Analysis of differences between the first and second time series may be compared to analysis of one of the time series using a subject model. The comparison is then used to validate the model or demonstrate accuracies, inaccuracies, and/or model bias with respect to a performance metric.

Evaluating models that rely on aggregate historical data

Systems, methods, and computer-readable storage media for attribution modeling using withheld or near impression data are provided. One method involves determining, for a first content item impression, withheld or near impressions for a competing content item within a content auction. The method further involves identifying a first set of paths including a sequence of events that includes an interaction with the first content item impression. The method further involves identifying a second set of paths, each including the sequence of events with the competing content item impression replacing the first content item impression. The method compares conversion metrics for the first and second paths to determine attribution credit for the first content item impression.

Attribution modeling using withheld or near impressions

Methods, systems, and apparatus including computer programs encoded on computer-readable storage media are provided for evaluating an attribution model, based on simulated activity streams. Parameters are specified that describe how users behave in the absence and presence of advertising, and parameters that regulate advertising in a simulation. A first set of simulated activity streams is generated with advertising turned on, observational metrics associated with the first set of streams are determined, and an attribution model is applied to determine a first fraction of incremental conversions associated with one or more advertising channels. Further sets of simulated activity streams are generated, each with a single advertising channel turned off, observational metrics associated with the further sets of streams are determined, and a second fraction of incremental conversions is determined for each advertising channel. The first and second fractions of incremental conversions are compared to evaluate the attribution model.

Evaluating attribution models based on simulated activity streams

Methods, systems, and apparatus including computer programs encoded on computer-readable storage media are provided for evaluating attribution models and comparing estimates produced by the attribution models with causal measurements from controlled experiments. An attribution model is identified for use in determining an estimate of an effectiveness of a campaign. An experiment is identified including experiment data that reflects implementation of a change in an experiment environment including identifying a measure of effectiveness of the change within a predetermined confidence level. The estimate is evaluated as compared to the identified measure of effectiveness. A determination is made that the attribution model is an effective measure of the change for the campaign, based at least in part on the evaluating.

Evaluating attribution models

The present disclosure includes systems and techniques relating to identifying value marketing activities. In some implementations, an apparatus, systems, or methods can include receiving conversion path information including data relating to user interactions with a content item associated with a marketing activity, determining a first attribution credit by applying a first attribution model to the received information, and a second attribution credit by applying a second attribution model to the received information, determining an attribution contrast ratio based on the first and second attribution credit, identifying an opportunity based on the determined attribution contrast ratio, and presenting a recommendation for the marketing activity based on the identified opportunity.

Attribution marketing recommendations

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for evaluating clusters of geographic entities, for example, to be used in a randomized geographic experiment. One method includes using a clustering algorithm to cluster geographic entities into a set of clusters, and identifying whether each geographic entity is an ambiguously classified entity or a definitively classified entity. The method further includes determining a measurement for the set of clusters according to a quantification of an attribute of the definitively classified entities and the ambiguously classified entities. Similar measurements can be calculated for other sets of clusters, and the clusters can be compared according to their measurements.

Evaluating techniques for clustering geographic entities

A system for monitoring the reliability of a network includes a network management system that gathers information relating to modules, nodes and paths in the network. The information includes the configuration, history and status of the modules and paths, and the failure states of the paths and the modules responsible for any failed paths. Based on the information, a reliability monitoring tool produces output that includes the condition of network paths and the actual reliability of one or more paths. Embodiments of the invention include storage arrangements that store the module and path information, in which case the output may include time-based reliability of network paths. The output may be available via the Internet. Methods of monitoring the reliability of a network are also provided.

Systems and methods for tracking the reliability of communications networks

A system for tracking modules in a network includes a service history recording arrangement associated with a module. The arrangement is configured to store data relating to the module. The system further includes a data storage arrangement configured to store data relating to the module. The system also includes a network monitoring arrangement configured to read information from or write information to the service history recording arrangement and to read information from or write information to the data storage arrangement. The system also includes a processor configured to receive and process data from the data storage arrangement and thereby produce analyzed network information. The service history recording arrangement may be integral to the module. The service history recording arrangement may include nonvolatile memory. Modules that include a service history recording arrangement are also provided. Associated methods are also provided.

Growing community of inventors

Lafayette, CO, United States of America

Jon Vaver