Microsoft Technology Licensing, LLC

Jinyun Yan

Shaunak Chatterjee

Yuan Gao

Birjodh S Tiwana

David Pardoe

Zhiyuan Xu

Gaurav Chandalia

Hsiao-Ping Tseng

Rupesh Gupta

Shipeng Yu

Xiaoyu Chen

Romer E Rosales

Kinjal Basu

Yan Gao

Sandor Nyako

Onkar Anant Dalal

Changji Shi

Yin Zhang

Revant Kumar

Sarah Yan Xing

Peng Du

Vinay Praneeth Boda

Zhongen Tao

Lijun Peng

Techniques are provided for automatically tuning a parameter in a layered model framework. One or more machine learning techniques are used to train multiple versions of a first model that includes a first version and a second version. A second model is stored that includes a parameter and accepts, as input, output from the first model. Multiple parameter values of the parameter are tested when processing content requests using the first and second versions of the first model. A strict subset of the plurality of parameter values are selected for the parameter of the second model, such that processing a first subset of the content requests using the first version of the first model results in a first value of a particular metric that matches a second value of the particular metric resulting from processing a second subset of the content requests using the second version of the first model.

Automatically tuning parameters in a layered model framework

Techniques for using a machine-learned model to personalize content item density. In one technique, an entity that is associated with a content request is identified. Multiple sets of content items are identified that includes content items of different types. A first position of a first slot is determined in a content item feed that comprises multiple slots. A second position of a previous content item is determined, in the content item feed, that is of a first type. A difference between the first position and the second position is determined. Based on the difference, a gap sensitivity value that is associated with the entity and is different than the difference is determined. Based on the gap sensitivity value, a content item from the multiple sets of content items is selected and inserted into the first slot. The content item feed is transmitted to a computing device to be presented thereon.

Using a machine-learned model to personalize content item density

Techniques for controlling item frequency using machine learning are provides. In one technique, two prediction models are trained: one based on interaction history of multiple content items by multiple entities and the other based on predicted interaction rates and an impression count for each of multiple content items. In response to a request, a particular entity associated with the request is identified and multiple candidate content items are identified. For each identified candidate content item, the first prediction model is used to determine a predicted interaction rate, an impression count of the candidate content item is determined with respect to the particular entity, the second prediction model is used to generate an adjustment based on the impression count, and an adjusted entity interaction rate is generated based on the predicted interaction rate and the adjustment. A particular candidate content item is selected based on the generated adjusted entity interaction rates.

Controlling item frequency using a machine-learned model

Techniques for varying content item density are provided. A first minimum gap value is stored that dictates how close two content items of a first type may appear in a content item feed that contains content items of multiple types that includes the first type and a second type. The first minimum gap value is used to place content items in a first set of content item feeds. For each content item feed of the first set of content item feeds, performance data that indicates how well content items of the first type perform in the content item feed is generated. Based on the performance data and the first minimum gap value, a second minimum gap value that is different than the first minimum gap value is generated. The second minimum gap value is used to place content items in a second plurality of content item feeds.

Feedback based controller for varying content item density

A method can include determining, based on learned parameter values, an intrinsic interest and an affinity for the user to be influenced to visit the website, determining, using the learned parameter values, intrinsic interest, and affinity for the user to be influenced to visit the website, a first probability indicating a likelihood that the user will, in response to viewing a badge notification, turn off notifications or delete an app and a second probability indicating a likelihood that the user will, in response to viewing the badge notification on the app, visit a website, in response to determining the second probability is greater than a threshold larger than the first probability, causing the app to include the badge notification when displayed on the user device.

Preventing notification blindness

Techniques for accounting for position-specific differences in user interaction while conducting content item selection events are provided. In one technique, a position-specific factor is determined. The position-specific factor may be based on a ratio of an observed interaction and a predicted interaction. Different positions in a content item feed or on a web page may be associated with different position-specific factors. Eventually, multiple content items are identified for presentation on a screen of a computing device. The content items include a first content item for which a predicted interaction rate is calculated and a second content item for which no predicted interaction rate is calculated. An order of the content items is determined based on the position-specific factor. For example, the predicted interaction rate of the first content item is modified based on the position-specific factor. The content items are presented on the screen based on the order.

Position-aware corrections in content item selection events

A system and method for content generation and targeting using machine learning are provided. In example embodiments, a probability that a user will visit a webpage based on historical data is calculated. A probability that the user will engage with a particular content category based on past user engagement is calculated. In response to the probability of the user engaging with the particular content category being equal to or greater than a first threshold, the content is generated. Further, in response to the probability of the user not visiting a webpage meeting or exceeding a second threshold, the generated content is sent to the user.

Content generation and targeting using machine learning

Techniques for automatically merging multiple content item queues are provided. In one technique, a first set of content items of a first type is identified. A second set of content items of a second type that is different than first type is identified. The first set of content items and the second set of content items are merged in a content item feed. Such merging involves, for a particular slot in the content item feed: determining a previous slot that contains a first content item from the first set; determining a number of slots between the previous slot and the particular slot; based on the number of slots, generating a score for a second content item from the second set; and based on the score, determining whether to insert, into the particular slot, the second content item or a third content item from the first set of content items.

Automatically merging multiple content item queues

A notification platform for distribution of notification content in an on-line social network system addresses the technical problem of optimizing the volume of quality notifications that are being delivered to a given member. A notification delivery system is designed as a stream processing system that can fetch, store, and process data in a near-line fashion. It can perform feature generation, processing and scoring of notifications, as well as ranking of the notifications based on their respective relevance scores that are calculated using machine learning techniques. The notification delivery system is positioned centrally with respect to different producers of notifications, such that it can consume centrally-stored information about members' holistic notification experiences.

Growing community of inventors

Sunnyvale, CA, United States of America

Jinyun Yan