Microsoft Technology Licensing, LLC

Dan Liu

Qi Guo

Daniel Sairom Krishnan Hewlett

Yi Wu

Wei Guang Lu

Baolei Li

Xuhong Zhang

Wenxiang Chen

Lester Gilbert Cottle, Iii

Xiaoyi Zhang

Mingzhou Zhou

Xuebin Yan

Anthony Hsu

Haitong Tian

Keqiu Hu

Siyao Sun

Chenya Zhang

Pei-Lun Liao

Wensheng Sun

Yu Gong

In an example embodiment, a platform is provided that utilizes information available to a computer system to feed a neural network. The neural network is trained to determine both the probability that a searcher would select a given potential search result if it was presented to him or her and the probability that a subject of the potential search result would respond to a communication from the searcher. These probabilities are combined to produce a single score that can be used to determine whether to present the searcher with the potential search result and, if so, how high to rank the potential search result among other search results. During the training process, a rescaling transformation for each input feature is learned and applied to the values for the input features.

Rescaling layer in neural network

In an example embodiment, a platform is provided that utilizes information available to a computer system to feed a neural network. The neural network is trained to determine both the probability that a searcher would select a given potential search result if it was presented to him or her and the probability that a subject of the potential search result would respond to a communication from the searcher. These probabilities are essentially combined to produce a single score that can be used to determine whether to present the searcher with the potential search result and, if so, how high to rank the potential search result among other search results. In a further example embodiment, embeddings used for the input features are modified during training to maximize an objective.

Embedding layer in neural network for ranking candidates

In an example embodiment, position bias and other types of bias may be compensated for by using two-phase training of a machine-learned model. In a first phase, the machine-learned model is trained using non-randomized training data. Since certain types of machine-learned models, such as those involving deep learning (e.g., neural networks) require a lot of training data, this allows the bulk of the training to be devoted to training using non-randomized training data. However, since this non-randomized training data may be biased, a second training phase is then used to revise the machine-learned model based on randomized training data to remove the bias from the machine-learned model. Since this randomized training data may be less plentiful, this allows the deep learning machine-learned model to be trained to operate in an unbiased manner without the need to generate additional randomized training data.

Two-stage training with non-randomized and randomized data

In an example embodiment, position bias is addressed by introducing an inverse propensity weight into a loss function used to train a machine-learned model. This inverse propensity weight essentially increases the weight of candidates in the training data that were presented lower in a list of candidates. This achieves the benefit of counteracting the position bias and increases the effectiveness of the machine-learned model in generating scores for future candidates. In a further example embodiment, a function is generated for the inverse propensity weight based on responses to contact requests from recruiters. In other words, while the machine learned-model may factor in both the likelihood that a recruiter will want to contact a candidate and the likelihood that a candidate will respond to such a contact, the function generated for the inverse propensity weight will be based only on training data where the candidate actually responded to a contact.

Position debiasing using inverse propensity weight in machine-learned model

Growing community of inventors

Santa Clara, CA, United States of America

Dan Liu