Microsoft Technology Licensing, LLC

Yahoo! Inc.

Bee-Chung Chen

Deepak Agarwal

Pradheep K Elango

Shaunak Chatterjee

Bo Long

Nitin Motgi

Yiming Ma

Liang Zhang

Wei Chu

Raghu Ramakrishnan

H Scott Roy

Raghunath Ramakrishnan

Malcolm G Slaney

Qi He

Venkatesh Ganti

Yang Yang

John Thrall

Zheng Li

Weidong Zhang

Jaewon Yang

Mohamed Gamal Mohamed Mahmoud

Amitabh Seth

Huiji Gao

Kaushik Shriraghav

Grace W Tang

Howard Scott Roy

Vik Singh

Joel Daniel Young

Ravi Kiran Holur Vijay

Romer E Rosales

David J Stein

Yitong Zhou

Seung-Taek Park

Boyi Chen

Vijay Krishna Narayanan

Joshua Daniel Hartman

Wei Zhuang

Paul Taylor Ogilvie

Wenxuan Gao

Amit Chandak

Liang Tang

Guangde Chen

Mikhail Obukhov

Josh Fleming

Yuwei Qiu

Chengming Jiang

Alex Shelkovnykov

XianXing Zhang

Joe Zachariah

Sharat Israni

Chandar Venkataraman

Amit Phadke

Michael Salisbury

Varnit Agnihotri

Divya Gadde

Ruoyang Wang

Amit Yadav

Yangchun Luo

Ke Wu

Kwei-you Tao

Priyanka Gariba

Jun Jia

Sheng Wu

Menglin L Brown

The disclosed embodiments provide a system for processing data. During operation, the system obtains a training dataset containing a first set of records associated with a first set of identifier (ID) values and an evaluation dataset containing a second set of records associated with a second set of ID values. Next, the system selects a random subset of ID values from the second set of ID values. The system then generates a sampled evaluation dataset comprising a first subset of records associated with the random subset of ID values in the second set of records. The system also generates a sampled training dataset comprising a second subset of records associated with the random subset of ID values in the first set of records. Finally, the system outputs the sampled training dataset and the sampled evaluation dataset for use in training and evaluating a machine learning model.

Data sampling for model exploration utilizing a plurality of machine learning models

The disclosed embodiments provide a system for processing data. During operation, the system obtains a feature configuration for a feature. Next, the system obtains, from the feature configuration, an anchor containing metadata for accessing the feature in an environment. The system then uses one or more attributes of the anchor to retrieve one or more feature values of the feature from the environment. Finally, the system provides the one or more feature values for use with one or more machine-learning models.

Framework for managing features across environments

Computer-implemented techniques for learning composite machine learned models are disclosed. Benefits to implementors of the disclosed techniques include allowing non-machine learning experts to use the techniques for learning a composite machine learned model based on a learning dataset, reducing or eliminating the explorative trial and error process of manually tuning architectural parameters and hyperparameters, and reducing the computing resource requirements and model learning time for learning composite machine learned models. The techniques improve the operation of distributed learning computing systems by reducing or eliminating straggler effects and by reducing or minimizing synchronization latency when executing a composite model search algorithm for learning a composite machine learned model.

Distributed learning of composite machine learning models

In an example embodiment, a warm-start training solution is used to dramatically reduce the computational resources needed to train when retraining a generalized additive mixed-effect (GAME) model. The problem of retraining time is particularly applicable to GAME models, since these models take much longer to train as the data grows. In the past, the strategy to reduce computational resources during retraining was to use less training data, but this affects the model quality, especially for GAME models, which rely on fine-grained sub-models at, for example, member or item levels. The present solution addresses the computational resources issues without sacrificing GAME model accuracy.

Warm start generalized additive mixed-effect (game) framework

A system and method for intermediate landing page rerouting are provided. In example embodiments, determine whether a webpage associated with a hyperlink has corresponding social network activities. Extract content from the webpage determined to have corresponding social network activities. In response to a selection of the hyperlink, reroute a web browser to an intermediate landing page. Cause presentation, at a user interface, of the extracted content and the corresponding social network activities.

Rerouting to an intermediate landing page

A system and method of adjusting an affinity score between an entity pair in a social network is disclosed. The method may include determining, with a processor, whether a first member of the entity pair is a heavy user member. The method further includes if the first member is the heavy user member, determining, with the processor, an affinity adjustment factor between the first member and the second member, and adjusting, with the processor, the affinity score between the first member and the second member of the entity pair in accordance with the adjustment factor to determine an adjusted affinity score. The method may include determining, with the processor, whether a number of interactions on content items indicates that the first member is the heavy user member. The second member is associated with a content item that is being considered for display to the first member.

Adjusting heavy users' affinity for heavy user entity-pairs in a social network

This disclosure relates to systems and methods that include a member activity database including data indicative of interactions with content items on a social network by a population of users of the social network. A processor is configured to obtain an optimization criterion based on at least two constraints related to a performance of the social network, obtain, for a subset of the population of users, at least some of the data indicative of interactions with content items from the member activity database, determine, based on the at least some of the data as obtained, an operating condition for the social network that is estimated to meet the optimization criterion, and provide, to at least some of the user devices via the network interface, the social network based, at least in part, on the operating condition.

Subset multi-objective optimization in a social network

A news feed system provided with an on-line social network system determines that a news feed is to be constructed for a viewer. The news feed system accesses the viewer's profile and other information associated with the viewer, accesses an inventory of activities that have been identified as potentially of interest to the viewer, and calculates relevance score for each item inventory of activities using the combined coefficients methodology. The activities are then arranged for presentation to the viewer via a news feed web page, using respective calculated relevance scores.

Using combined coefficients for viral action optimization in an on-line social network

A social networking system receives from a member an item for sharing on the social networking system. The system determines whether the item for sharing is a first sharing for the member or whether the member has not shared an item for a time period that transgresses a threshold. When the item for sharing is a first sharing or a sharing that transgresses the threshold, the system marks the item for a promotion in a feed of another member of the social networking system.

Social incentive optimization in online social networks

The disclosed embodiments provide a system for processing data. During operation, the system obtains a global version of a statistical model and a user-specific version of the statistical model for a user. Next, the system applies the global version to member features of the user and job features of a set of jobs to generate a first ranking of the jobs for the user. The system then applies the user-specific version to the member features and the job features for a highest-ranked subset of jobs in the first ranking to generate a second ranking of the jobs for the user. Finally, the system outputs at least a portion of the second ranking as a set of job recommendations.

Scalable response prediction using personalized recommendation models

Systems and methods for federating social network content items include determining an engagement value for content items based on previous interactions with content items from first and second content item sources stored in an electronic data storage, certain content items having selection metrics and value metrics according to varying selection schemas and a value schema. A utility value is generated for each of the at least some of the content items, the utility value including an engagement component and a monetary component. A user device displays the content items in an order based on the utility values.

Social network content item federation based on item utility value

Software for initialized explore-exploit creates a plurality of probability distributions. Each of these probability distributions is generated by inputting a quantitative description of one or more features associated with an image into a regression model that outputs a probability distribution for a measure of engagingness for the image. Each of the images is conceptually related to the other images. The software uses the plurality of probability distributions to initialize a multi-armed bandit model that outputs a serving scheme for each of the images. Then the software serves a plurality of the images on a web page displaying search results, based at least in part on the serving scheme.

Finding engaging media with initialized explore-exploit

Content items are selected to be displayed on a portal page in such a way as to maximize a performance metric such as click-through rate. Problems relating to content selection are addressed, such as changing content pool, variable performance metric, and delay in receiving feedback on an item once the item has been displayed to a user. An adaptation of priority-based schemes for the multi-armed bandit problem, are used to project future trends of data. The adaptation introduces experiments concerning a future time period into the calculation, which increases the set of data on which to solve the multi-armed bandit problem. Also, a Bayesian explore/exploit method is formulated as an optimization problem that addresses all of the issues of content item selection for a portal page. This optimization problem is modified by Lagrange relaxation and normal approximation, which allow computation of the optimization problem in real time.

Enhanced matching through explore/exploit schemes

Content display policies are evaluated using two kinds of methods. In the first kind of method, using information, collected in a 'controlled' manner about user characteristics and content characteristics, truth models are generated. A simulator replays users' visits to the portal web page and simulates their interactions with content items on the page based on the truth models. Various metrics are used to compare different content item-selecting algorithms. In the second kind of method, no explicit truth models are built. Events from the controlled serving scheme are replayed in part or whole; content item-selection algorithms learn using the observed user activities. Metrics that measure the overall predictive error are used to compare different content-item selection algorithms. The data collected in a controlled fashion plays a key role in both the methods.

Framework to evaluate content display policies

Content items are selected to be displayed on a portal page in such a way as to maximize a performance metric such as click-through rate. Problems relating to content selection are addressed, such as changing content pool, variable performance metric, and delay in receiving feedback on an item once the item has been displayed to a user. An adaptation of priority-based schemes for the multi-armed bandit problem are used to project future trends of data. The adaptation introduces experiments concerning a future time period into the calculation, which increases the set of data on which to solve the multi-armed bandit problem. Also, a Bayesian explore/exploit method is formulated as an optimization problem that addresses all of the issues of content item selection for a portal page. This optimization problem is modified by Lagrange relaxation and normal approximation, which allow computation of the optimization problem in real time.

Growing community of inventors

San Jose, CA, United States of America

Bee-Chung Chen