Facebook, Inc.

Meta Platforms, Inc.

Tianshi Gao

Liang Xiong

Wenlin Chen

Dinkar Jain

Shyamsundar Rajaram

Ou Jin

Xianjie Chen

Yifei Huang

Mircea Grecu

Yifei Zhang

Stuart Michael Bowers

Anand Sumatilal Bhalgat

Robert Oliver Burns Zeldin

Jurgen Anne Francois Marie Van Gael

Hao Zhang

Ahmad Abdulmageed Mohammed Abdulkader

Vibhor Rastogi

Bingqing Wang

David Michael Viner

Feng Yan

Puneet Sharma

Pengjun Pei

Qian Yan

Sina Jafarpour

Lu Zheng

Satya Satyavarta

Christian Alexander Martine

Darshan Kantak

Xiangyu Wang

Vignesh Ramanathan

To present one or more content items to users of an online system, the online system identifies a content evaluation pipeline including an order of a plurality of stages having one or more computer models for evaluating a likelihood of user interaction with a content item. The content evaluation pipeline selects a decreasing number of content items, from each stage of the order, according to the order of the stages. The online system optimizes the selection of content items selected at the plurality of stages of the content evaluation pipeline by training the computer models to predict content selection values that the subsequent model would generate for a content items in a training data set and content items that the subsequent model would select for input to the next stage of the content evaluation pipeline.

Model compression for selecting content

To select the content to be presented to the user, a first latent vector is determined for a content item based on a first object associated with the content item. A second latent vector is determined for the content item based on a second object associated with the content item. A content item vector is then determined based on the first and second latent vectors. Furthermore, a user vector is determined based on interactions of the user with the first set of content objects and the second set of content objects. A score indicative of the likelihood of the user interacting with the content item is determined based on the content item vector and the user vector.

Deep semantic content selection

An online system trains a content selection model based on a selected subset of presented content items as well as a sampled set of content items. The content selection model is configured to receive a set of features characterizing a user-content item pair and output a likelihood that the user will interact with the content item. The sampled set of content items may include content items that were not selected for display based on their likelihoods in addition to those that were selected, and may represent a wider distribution of user-content item pairs than the selected subset. By incorporating the sampled set of content items as well as the selected subset of content items in the training process, the online system can reduce bias in the content selection process such that content items similar to the unselected subset can also be adequately represented.

Training content selection models with reduced bias

An online system tracks stores information identifying content provided by third party systems and accessed by online system users as well as interactions with advertisements performed by online system users. When the online system identifies an opportunity to present an advertisement to a viewing user, the online system identifies content from third party systems accessed by the viewing user and content from third party systems accessed by additional online system users who interacted with advertisements. A score is computed for various advertisements based at least in part on correlations between content from third party systems accessed by the viewing user and content from third party systems accessed by additional online system users who interacted with advertisements. The online system selects candidate advertisements to evaluate for presentation to the viewing user based on the scores.

Selecting content for presentation to online system users based on correlations between content accessed by users via third party systems and interactions with online system content

A preprocessing module of a neural network has a first input and second input. The module generates multiple, different first latent vector representations of its first input, and multiple, different second latent vector representations of its second input. The module then models pairwise interactions between every unique pairwise combination of the first and second latent vector representations. The module then produces an intermediate output by combining the results of the modeled pairwise interactions.

Mixed machine learning architecture

In one embodiment, a method includes a preprocessing stage of a neural network model, where the preprocessing stage includes first and second preprocessing modules. Each of the two modules has first input that may receive a dense input and a second input that may receive a sparse input. Each module generates latent vector representations of their respective first and second inputs, and combine the latent vectors with the original first input to define an intermediate output. The intermediate output of the first module is fed into the first input of the second module.

Nested machine learning architecture

To present one or more content to users of an online system, the online system identifies a content evaluation pipeline including an order of a plurality of stages having one or more computer models for evaluating a likelihood of user interaction with a content item. The content evaluation pipeline selects a decreasing number of content items, from each stage of the order, according to the order of the stages in the order. The online system identifies a set of candidate modifications to one or more operational parameters of the content evaluation pipeline. For each candidate modification, the online system determines a compute time value and a content selection value. For a given amount of compute time, the online system optimizes the one or more operational parameters based on the determined content time value and the determined content selection value to increase the content selection value of the content evaluation pipeline.

Multi-stage ranking optimization for selecting content

An online system maintains one or more models that determine likelihoods of a user performing various interactions after being presented with a content item. Additionally, the online system receives information identifying interactions by users with content, and generates embeddings for various users based on the interactions by the users with content. When determining whether to present a content item including an objective identifying an interaction to a user, the online system applies a maintained model to determine a likelihood of the user performing the interaction identified by the objective after being presented with the content item. Additionally, the online system determines a similarity of the embedding of the user to embeddings of users who performed the interaction identified by the objective. Based on a combination of the likelihood determined by the model and the similarity, the online system determines whether to present the content item to the user.

Accounting for organically occurring interactions with content when selecting content items for presenstation to users of an online system

A system predicts user intent to take an action and delivers content items to the user that match that intent. A plurality of features or attributes for each tracking pixel in a set of tracking pixels can be acquired based on content items and landing pages associated with each tracking pixel. For example, features for a tracking pixel can be determined based on information associated with a content item that enabled a user to access a landing page from which the tracking pixel was fired or triggered. In this example, features for the tracking pixel can also be determined based on information associated with the landing page. The features for the tracking pixels can be utilized to train a machine learning model. The machine learning model can be trained to predict whether or not a particular user intends to produce a conversion (e.g., make a purchase).

Determining intent based on user interaction data

An online system receives tracking requests from client devices interacting with a web page to analyze user interactions with the web page. The online system extracts parameters from tracking requests such as a uniform resource locator (URL) associated with the web page that generated the tracking request and/or data tokens describing keywords within the URL. The online system may extract parameters by crawling web pages that generate tracking requests. The online system may compare extracted parameters to a taxonomy of categories maintained by the online system to determine a category describing the item displayed on the web page. The online system determines a category describing the item via an item catalog maintained by the online system comprised of previously determined categories for various items. The online system uses the determined categories, attributes, and temporal relevance scores to direct content to users.

Analyzing tracking requests generated by client devices based on attributes describing items

An online system receives content items from a third party content provider. For each content item, the online system inputs an image into a neural network and extracts a feature vector from a hidden layer of the neural network. The online system compresses each feature vector by assigning a label to each feature value representing whether the feature value was above a threshold value. The online system identifies a set of content items that the user has interacted with and determines a user feature vector by aggregating feature vectors of the set of content items. For a new set of content items, the online system compares the compressed feature vectors of the content item with the user feature vector. The online system selects one or more of the new content items based on the comparison and sends the selected content items to the user.

Neural network based content distribution in an online system

When an online system receives a request to present content items to a user, a content selection system included in the online system selects content items for presentation to the user. A feedback control mechanism communicates with each computing device of the content selection system to determine the latency period and the CPU utilization of each computing device. The feedback control mechanism also determines a target latency period and a target CPU utilization in which content items are selected. By comparing the latency period of each computing device to the target latency period and the CPU utilization to the target CPU utilization, an amount of information to be evaluated by each computing device is determined based on the comparisons.

Latency and CPU utilization control for content item selection

Some embodiments include a method of generating a compatibility score for a grouping of objects based on correlations between attributes of the objects. An example grouping is a pair of user and ad. The method may be implemented using a multi-threaded pipeline architecture that utilizes a learning model to compute the compatibility score. The learning model determines correlations between a first object's attributes (e.g., user's liked pages, user demographics, user's apps installed, pixels visited, etc.) and a second object's attributes (e.g., expressed or implied). Example expressed attributes can be targeting keywords; example implied attributes can be object IDs associated with the ad.

Growing community of inventors

Fremont, CA, United States of America

Tianshi Gao