Google Inc.

Deepak Ramamurthi Sivaramapuram Chandrasekaran

Tanmay Sanjay Khirwadkar

Chetan Pitambar Bhole

Sanjay Mangla

Sourabh Prakash Bansod

Thomas Price

Sarvjeet Singh

Prachi Gupta

Bo Fu

Sue Yi Chew

Kunal Jain

Jierui Xie

Methods, systems, and media for timing presentation of content are provided. In some embodiments, the method comprises: causing a first video content item to be presented on a user device of a user; wherein the video content item is provided by a video sharing service; identifying a promotion for the video sharing service that is to be presented to the user during a time window; determining a probability that the user will select the promotion if the promotion is presented at a first time point within the time window in connection with the first video content item; determining whether to present the promotion at the first time point based on at least on the probability; in response to determining that the promotion is to be presented at the first time point, causing the promotion to be presented on the user device at the first time point in connection with the first video content item; and, in response to determining that the promotion is not to be presented at the first time point, causing the promotion to be presented on the user device at a second time point in connection with a second video content item, wherein the second time point is within the time window and wherein the second time point is after the first time point.

Methods, systems, and media for timing presentation of content

Balancing content distribution between a machine learning model and a statistical model provides a baseline assurance in combination with the benefits of a well-trained machine learning model for content selection. In some implementations, a server receiving requests for a content item assigns a first proportion of the received requests to a first group and assigns remaining requests to a second group. The server uses a machine learning model to select variations of the requested content item for responding to requests assigned to the first group and uses a statistical model to select content variations for requests assigned to the second group. The server obtains performance information, e.g., acceptance rates for the different variations, and compares performance of the different models used for content selection. Audience share assigned to the machine learning model is increased when it outperforms the statistical model and decreased when it underperforms the statistical model.

Assessing accuracy of a machine learning model

A simulation test is run on a first machine learning model trained using first training data historically collected over a time period. The first training data includes a set of training inputs and a set of target outputs. In response to a determination that a result of the simulation test run on the first machine learning model satisfies one or more criteria, a size of the set of target outputs of the first training data is determined. Second training data for training a second machine learning model is obtained. A size of a set of target outputs of the second training data meets or exceeds the size of the target outputs of the first training data. The second machine learning model is trained using the second training data.

System and methods for machine learning training data selection

Methods, systems, and media for timing presentation of content are provided. In some embodiments, the method comprises: causing a first video content item to be presented on a user device of a user, wherein the video content item is provided by a video sharing service; identifying a promotion for the video sharing service that is to be presented to the user during a time window; determining a probability that the user will select the promotion if the promotion is presented at a first time point within the time window in connection with the first video content item; determining whether to present the promotion at the first time point based on at least on the probability; in response to determining that the promotion is to be presented at the first time point, causing the promotion to be presented on the user device at the first time point in connection with the first video content item; and, in response to determining that the promotion is not to be presented at the first time point, causing the promotion to be presented on the user device at a second time point in connection with a second video content item, wherein the second time point is within the time window and wherein the second time point is after the first time point.

A system and method are disclosed for running a plurality of simulation tests on a first machine learning model to obtain a plurality of results that are each produced during a respective simulation test, the first machine learning model gradually trained using first training data historically collected over a period of time, the first training data comprising a plurality of first training data sets each including a subset of first training inputs and first target outputs associated with one of a plurality of points in time during the period of time, determining a simulation test of the plurality of simulation tests at which corresponding results of the first machine learning model satisfy a threshold condition, wherein the threshold condition is based on historical data at a first point in time of the plurality of points in time, identifying a first training data set of the plurality of first training data sets on which the first machine learning model used during the determined simulation test was trained, wherein the first training data set on which the first machine learning model used during the determined simulation test was trained is associated with one or more second points in time that precede the first point in time, and determining a subset of target outputs from the identified first training data set on which the first machine learning model used during the determined simulation test was trained, the determined subset of first target outputs to define an amount of second training data to be sufficient to train a second machine learning model.

Growing community of inventors

San Jose, CA, United States of America

Deepak Ramamurthi Sivaramapuram Chandrasekaran