Microsoft Technology Licensing, LLC

Qing Duan

Xiaowen Zhang

Junrui Xu

Aman Grover

Wen Pu

Benjamin Hoan Le

Xiaoqing Wang

Jianqiang Shen

Qingyun Wan

Liang Zhang

Kun Liu

Keqing Liang

Huichao Xue

Sumedha K Swamy

Zhifeng Deng

Qianqi Shen

In an example embodiment, a first machine learned model is trained to produce output, and a second machine learned model is then trained using training data that has been labeled, at least partially, using the output of the first machine learned model. The first machine learned model is trained to output a measure of how strong a positive signal in the training data really is. Specifically, this measure indicates the level of intention of a user who has engaged in a first user interface action with respect to a piece of content to engage in a subsequent second user interface action with the same piece of content.

Click intention machine learned models

The disclosed embodiments provide a system for processing data. During operation, the system performs processing related to a first set of features for a first entity using a first series of embedding layers, wherein the processing includes applying each embedding layer in the first series of embedding layers to a concatenation of all outputs of one or more layers preceding the embedding layer. Next, the system obtains a first embedding as an output of a first final layer in the first series of embedding layers. The system then outputs the first embedding for use by a machine learning model.

Entity pyramid embeddings

The disclosed embodiments provide a system for processing scoring requests. During operation, the system matches an identifier for an entity in a scoring request to a cache entry in a score cache. Next, the system retrieves, from the cache entry, a previous value of a score generated by a machine learning model from previous values of features for the entity and a first encoded representation of the previous values of a subset of the features with greater than a threshold effect on the score. The system then compares the first encoded representation with a second encoded representation of the most recent values of the subset of the features for the entity. When the comparison indicates that the most recent values match the previous values, the system outputs the previous value of the score for the entity in a response to the scoring request.

Data-driven online score caching for machine learning

Methods, systems, and computer programs are presented for presenting return-on-investment (ROI) information, for budgeted services that resulted in a successful service delivery, on a user interface for setting the budget for a service request. One method includes an operation for identifying daily budgets for budgeted services that resulted in a successful service delivery (BSSSD). Each daily budget indicates an amount for spending in promotion of the BSSSD in an online service. The method further includes receiving a request, in a graphical user interface (GUI) of the online service, for posting a daily budget for a first budgeted service. Further, a performance value, associated with the daily budgets of the BSSSD that are similar to the first budgeted service, is selected. Further, the method includes causing presentation, by the one or more processors, of the performance value in the GUI.

Job-post budget recommendation based on performance

Technologies for generating a graph containing clusters of feature attribute values for training a machine learning model for content item selection and delivery are provided. The disclosed techniques include, for each entity, of a plurality of entities, a system identifies transitions from one geographic location to another geographic location. A graph is generated based on the transitions associated with each entity. The graph comprises nodes representing geographic locations and edges connecting the nodes. Each of the edges connects two nodes, represents a transition from one geographic location to another geographic location, and each edge represents an edge weight value that is based on frequencies of transitions between geographic locations represented by the two connected nodes. The system generates a plurality of clusters from the nodes based upon the edge weight value of each edge. The system includes the plurality of clusters as features in a machine learning model.

Location dimension reduction using graph techniques

In an example the output of a machine learned model is a score is then compared to a threshold, and if the score transgresses the threshold, the corresponding item is available to be recommended to the user via the graphical user interface. In an example embodiment, rather than a fixed (static) threshold, a dynamic threshold is utilized. This dynamic threshold is based on a harmonic mean of probabilities utilized in the GLMix model. Specifically, the GLMix model may calculate and utilize the probability that a user will engage with a particular item via a graphical user interface, and also a probability that a user will dismiss a particular item via a graphical user interface.

Personalized model threshold

Technologies for determining performance gains for content item delivery based on modifications to content item selection parameters are provided. The disclosed techniques include implementing a multi-objective optimization model for content item selection using a value for a parameter. The model generates a first plurality of scores and a first ranking for content items. Subset of content items is selected for delivery based on the first ranking. New values for the parameter are identified and for each new value, the content item selection event is replayed. A second plurality of scores and a second ranking is generated for the content items, where the second ranking is different from the first ranking. A third plurality of scores and a third ranking is generated where the third ranking matches the second ranking. A set of gains is calculated for each new value, where each gain corresponds to a different objective of the model.

Estimating performance gains for content item delivery using performance replay

Techniques for user-defined quality control of resource usage are provided. In one technique, resource split data is received that indicates a split of a total resource amount that is associated with a content delivery campaign. In response, based on the split, a first resource amount and a second resource amount is determined, each a subset of the total resource amount. The first resource amount is associated with a first utilization strategy and a first mapping function and the second resource amount is associated with a second utilization strategy and a second mapping function. In response to receiving a request from a client device, an entity of the client device is determined and associated with the first mapping function. A quality score of the entity is determined and, based on the first mapping function and the quality score, an adjustment factor is determined. A content item selection event is conducted based on the adjustment factor.

User defined quality-control resource usage

Instead of a fixed fee for a particular job application, discussed in some examples are methods, systems, and machine readable mediums which provide for a job posting service that utilizes a pay-per-click model. That is, job posters pay a fee for each time the member selects the job posting for further inquiry when that posting is shown to a member (called an impression). The fee that is paid is determined by the job poster. Selecting a job posting may comprise clicking on or otherwise entering an input signifying an intention to view the job.

Growing community of inventors

Santa Clara, CA, United States of America

Qing Duan