Microsoft Technology Licensing, LLC

Vijay K Dialani

Abhishek Gupta

Yan Yan

Sahin Cem Geyik

Shakti Dhirendraji Sinha

Viet Thuc Ha

Ye Xu

Xianren Wu

Satya Kanduri

Parul Jain

Anthony Duane Duerr

Anish Ramdas Nair

Benjamin Hoan Le

Cagri Ozcaglar

Kevin Kao

Caleb Timothy Johnson

Sara Smoot Gerrard

The disclosed embodiments provide a system for processing data. During operation, the system obtains a set of feature additions and an evaluation metric for assessing the performance of a statistical model. Next, the system automatically builds treatment versions of the statistical model using a set of baseline features for the statistical model and feature combinations generated using the feature additions. The system then uses a hypothesis test and a fixed set of feature values to compare a baseline value of the evaluation metric for a baseline version of the statistical model that is built using the set of baseline features with additional values of the evaluation metric for the treatment versions. Finally, the system outputs a result of the hypothesis test for use in assessing an impact of the feature combinations on a performance of the statistical model.

Feature selection impact analysis for statistical models

The disclosed embodiments provide a system for processing data. During operation, the system obtains events reflecting responses by a user to job recommendations outputted to the user. Next, the system updates a set of features for the user from the events. The system then includes the updated set of features in a feature repository for use by a statistical model in generating a ranking of jobs for the user. Finally, the system retrains the statistical model using the events prior to using the statistical model to update the outputted job recommendations using the ranking.

Nearline updates to personalized models and features

Techniques for query intent clustering for automated sourcing are described. In an example embodiment, disclosed is a system comprising a processor, a storage device, and a memory device holding an instruction set executable on the processor to cause the system to perform operations. The system obtains one or more recent hire member profiles used as a basis for a search on member profiles in a social networking service. Additionally, the system extracts one or more attributes from the one or more recent hire member profiles and stores the attributes on the storage device. Moreover, the system identifies skills clusters based on the extracted attributes retrieved from the storage device. Furthermore, the system generates a search query based on the identified skills clusters. Then, a search can be performed on member profiles in the social networking service using the generated search query, returning one or more result member profiles as candidates.

Query intent clustering for automated sourcing

Techniques for providing explanations of candidate search queries are described. The queries can be created using query intent clustering in an automated sourcing tool. In an example embodiment, disclosed is a system that obtains one or more current candidate member profiles used as a basis for a search on member profiles in an online system. Additionally, the system extracts one or more attributes from the one or more current candidate member profiles. Moreover, the system identifies query intent clusters based on the extracted one or more attributes. Furthermore, the system generates a search query based on the identified query intent clusters. Next, an explanation of the search query can be displayed on a display device of the system. In some embodiments, the online system hosts a social networking service that includes the member profiles, and the identified query intent clusters include skills clusters.

Providing query explanations for automated sourcing

In an example embodiment, one or more specified ideal candidates are used to perform a search in a database. One or more attributes are extracted from one or more ideal candidate member profiles. A search query is then generated based on the extracted one or more attributes. Then, a search is performed on member profiles in the social networking service using the generated search query, returning one or more result member profiles.

Query building for search by ideal candidates

Techniques for using recruiter review data to create training, validation and test sets for automated sourcing are described. An example system obtains sample suggested candidate member profiles and sample search result member profiles in an online system. The system identifies unique pairs of member profiles, each pair consisting of one of the suggested candidate profiles and one of the search result profiles. Additionally, the system generates a label for each of the unique pairs of profiles. The label is generated using a score generated from log information of the online system, the log information including historical records of searcher feedback regarding members of the online system, the score being higher if the searcher accepted the sample search result member profile in a search session. Furthermore, the system inputs the labels into a machine learning algorithm to train a combined ranking model that outputs ranking scores for search result member profiles.

Using log data to train for automated sourcing

In an example embodiment, one or more sample ideal candidate member profiles in a social networking service are obtained, as well as one or more sample search result member profiles in the social networking service. Then, for each unique pair of sample ideal candidate member profile and sample search result member profile, a label is generated using a score generated from log information of the social networking service, the log information including records of communications between a searcher and members of the social networking service, the score being higher if the searcher communicated with both the member corresponding sample ideal candidate member profile and the member corresponding to the sample search result member profile in a same search session. The generated labels are fed into a machine learning algorithm to train a combined ranking model used to output ranking scores for search result member profiles.

Generation of training data for ideal candidate search ranking model

In an example embodiment, a query for search results is received, the query including at least one value for one facet, a facet defining a categorical dimension for the search results. It is then determined that the facet in the query is exclusive. In response to the determination that the facet is exclusive: for each potential facet different from the facet in the query: for each potential value in the potential facet: conditional entropy gain of the value in the query and the potential value is determined. The potential value in the potential facet that has the highest conditional entropy gain is determined, as is the potential facet with the minimum maximum conditional entropy gain. Then the potential facet with the minimum maximum is input into a machine learning model, causing the machine learning model to output one or more suggested facets to add to the query.

Growing community of inventors

Fremont, CA, United States of America

Vijay K Dialani