Microsoft Technology Licensing, LLC

The State University of New York

Yahoo! Inc.

Linkedin Corporation

Bo Long

Zhongfei Zhang

Huiji Gao

Liang Zhang

Deepak Agarwal

Frank Emmanuel Astier

Bee-Chung Chen

Jonathan David Traupman

Weiwei Guo

Lin Guo

Yi Chang

Yang Yang

Srinivas Vadrevu

Yiming Ma

Jianling Zhong

Michaeel M Kazi

Jun Shi

Anmol Bhasin

Belle L Tseng

Zhaohui Zheng

Zheng Li

Kun Liu

Anlei Dong

Bing Zhao

Tie Wang

Abhimanyu Lad

Yu Liu

Boyi Chen

Yi Zhen

Venu Javarappa

Nageswara R Pobbathi

Sudarshan Lamkhede

Junghoon Andrew Ahn

Ya Zhang

Yu Gan

Gonzalo Jorge Aniano Porcile

Yanbin Jiang

Josh Fleming

Yuwei Qiu

Chengming Jiang

Xiaowei Liu

Vignesh Murugesan

Chi-Chung Chan

Zhong Yi Wan

Deepak Argawal

Alex Shelkovnykov

Jun Jia

Sheng Wu

Menglin L Brown

Yunsong Meng

Yen-Jung Chang

Yung-Yu Chung

Ying Xiong

The disclosed embodiments provide a system for streamlining machine learning. During operation, the system determines a resource overhead for a baseline version of a machine learning model that uses a set of features to produce entity rankings and a number of features to be removed to lower the resource overhead to a target resource overhead. Next, the system calculates importance scores for the features, wherein each importance score represents an impact of a corresponding feature on the entity rankings. The system then identifies a first subset of the features to be removed as the number of features with lowest importance scores and trains a simplified version of the machine learning model using a second subset of the features that excludes the first subset of the features. Finally, the system executes the simplified version to produce new entity rankings.

Feature removal framework to streamline machine learning

The disclosed embodiments provide a system for processing data. During operation, the system obtains a training dataset containing a first set of records associated with a first set of identifier (ID) values and an evaluation dataset containing a second set of records associated with a second set of ID values. Next, the system selects a random subset of ID values from the second set of ID values. The system then generates a sampled evaluation dataset comprising a first subset of records associated with the random subset of ID values in the second set of records. The system also generates a sampled training dataset comprising a second subset of records associated with the random subset of ID values in the first set of records. Finally, the system outputs the sampled training dataset and the sampled evaluation dataset for use in training and evaluating a machine learning model.

Data sampling for model exploration utilizing a plurality of machine learning models

Computer-implemented techniques for learning composite machine learned models are disclosed. Benefits to implementors of the disclosed techniques include allowing non-machine learning experts to use the techniques for learning a composite machine learned model based on a learning dataset, reducing or eliminating the explorative trial and error process of manually tuning architectural parameters and hyperparameters, and reducing the computing resource requirements and model learning time for learning composite machine learned models. The techniques improve the operation of distributed learning computing systems by reducing or eliminating straggler effects and by reducing or minimizing synchronization latency when executing a composite model search algorithm for learning a composite machine learned model.

Distributed learning of composite machine learning models

Machine learning based method for generating personalized query suggestions is described. Different users may have different search intent even when they are inputting the same search query. The technical problem of personalizing search query suggestions produced by a machine learning model is addressed by extending the sequence to sequence machine learning model framework to be able to take into consideration additional, personalized features of the user, such as, e.g., profile industry, language, geographic location, etc. This methodology includes an offline model training framework as well as an online serving framework.

Generating personalized query suggestions

Disclosed are systems, methods, and non-transitory computer-readable media for generating personalized smart responses. A messaging system receives a message transmitted as part of a communication session between a first user and a second user and determines a set of candidate responses for replying to the message. The messaging system filters the set of candidate responses based on a personalized vocabulary for the first user, yielding a set of personalized candidate responses. The personalized vocabulary for the first user is determined based on member profile data associated with the first user. The messaging system determines, based on a set of personalized candidate selection rules, a subset of the candidate responses, yielding a set of recommended responses to the message, and causes the set of recommended responses to be presented on the second client device.

Generating personalized smart responses

A neural related query generation approach in a search system uses a neural encoder that reads through a source query to build a query intent vector. The approach then processes the query intent vector through a neural decoder to emit a related query. By doing so, the approach gathers information from the entire source query before generating the related query. As a result, the neural encoder-decoder approach captures long-range dependencies in the source query such as, for example, structural ordering of query keywords. The approach can be used to generate related queries for long-tail source queries, including long-tail source queries never before or not recently submitted to the search system.

Neural related search query generation

The disclosed embodiments provide a system for performing a natural language search. During operation, the system applies a first machine learning model to a natural language query to predict one or more search intentions associated with the natural language query. Next, the system applies a second machine learning model to the natural language query to produce one or more search parameters associated with a first intention in the search intention(s), wherein the search parameter(s) include a field and a value of the field. The system then performs a first search of a first vertical associated with the first intention using the search parameter(s). Finally, the system generates a ranking containing a first set of search results from the first search of the first vertical and outputs at least a portion of the ranking in a response to the natural language query.

Vertical processing of natural language searches

In an embodiment, the disclosed technologies include extracting, from at least one search log, session data including at least three semantically related queries and corresponding timestamp data; using the session data, creating a training sequence that includes source query data, context query data, and target query data, the source query data having both a temporal relationship and a lexical relationship to the target query data and the context query data having a temporal relationship to the source query data; creating a learned model by, using a machine learning-based modeling process, learning a mapping of a semantic representation of the context query data and the source query data to a semantic representation of the target query data; in response to a new query, using the learned model to generate at least one recommended query that is semantically related to the new query.

Session-aware related search generation

In an example embodiment, a warm-start training solution is used to dramatically reduce the computational resources needed to train when retraining a generalized additive mixed-effect (GAME) model. The problem of retraining time is particularly applicable to GAME models, since these models take much longer to train as the data grows. In the past, the strategy to reduce computational resources during retraining was to use less training data, but this affects the model quality, especially for GAME models, which rely on fine-grained sub-models at, for example, member or item levels. The present solution addresses the computational resources issues without sacrificing GAME model accuracy.

Warm start generalized additive mixed-effect (game) framework

The disclosed embodiments provide a system for identifying root causes of performance issues. During operation, the system obtains a call graph containing a set of call paths for a set of services. Next, the system determines, based on a load test of the set of services, severity scores for the set of services, wherein the severity scores represent levels of abnormal behavior in the set of services. The system then groups the severity scores by the set of call paths and identifies, based on the grouped severity scores, one or more services as potential root causes of performance issues in the set of services. Finally, the system outputs the identified one or more services as the potential root causes of the performance issues.

Identifying root causes of performance issues

Techniques for predicting a user response to content are described. According to various embodiments, a configuration file is accessed, where the configuration file includes a user-specification of raw data accessible via external data sources and raw data encoding rules. In some embodiments, the raw data includes raw member data associated with a particular member and raw content data associated with a particular content item. Thereafter, source modules encode the raw data from the external data sources into feature vectors, based on the raw data encoding rules. An assembler module assembles one or more of the feature vectors into an assembled feature vector, based on user-specified assembly rules included in the configuration file. A prediction module performs a prediction modeling process based on the assembled feature vector and a prediction model, to predict a likelihood of the particular member performing a particular user action on the particular content item.

Systems and methods for content response prediction

Relational clustering has attracted more and more attention due to its phenomenal impact in various important applications which involve multi-type interrelated data objects, such as Web mining, search marketing, bioinformatics, citation analysis, and epidemiology. A probabilistic model is presented for relational clustering, which also provides a principal framework to unify various important clustering tasks including traditional attributes-based clustering, semi-supervised clustering, co-clustering and graph clustering. The model seeks to identify cluster structures for each type of data objects and interaction patterns between different types of objects. Under this model, parametric hard and soft relational clustering algorithms are provided under a large number of exponential family distributions. The algorithms are applicable to relational data of various structures and at the same time unify a number of state-of-the-art clustering algorithms: co-clustering algorithms, the k-partite graph clustering, and semi-supervised clustering based on hidden Markov random fields.

System and method for probabilistic relational clustering

One particular embodiment clusters a plurality of documents using one or more clustering algorithms to obtain one or more first sets of clusters, wherein: each first set of clusters results from clustering the documents using one of the clustering algorithms; and with respect to each first set of clusters, each of the documents belongs to one of the clusters from the first set of clusters; accesses a search query; identifies a search result in response to the search query, wherein the search result comprises two or more of the documents; and clusters the search result to obtain a second set of clusters, wherein each document of the search result belongs to one of the clusters from the second set of clusters.

Clustering of search results

Method, apparatus, and programs for customizing a web site are provided. In one example, a method for customizing a web site is provided. One or more representations corresponding to one or more customizable components of a web site are provided. The one or more representations are to be displayed on a display screen to a user. An input entered by the user and directed to a specific customizable component of the web site is received. How to customize the specific customizable component of the web site is determined based on the input. An instruction is generated with respect to the customizable component. The instruction is used to implement customization of the specific customizable component of the web site in accordance with the input from the user.

Method and system for customizing a web site

Growing community of inventors

Palo Alto, CA, United States of America

Bo Long