Microsoft Technology Licensing, LLC

Milad Shokouhi

Paul Nathan Bennett

Pushmeet Kohli

Sanaz Ahari

John Guiver

Nitin Agrawal

Yamin Wang

Matteo Venanzi

Nick Craswell

Amy Huyen Phuoc Nguyen

Gabriella Kazai

Philipp Cannons

Dong Hwi Yoo

Rachel Imogen Solimeno

Chia-Jung Lee

Ivan Valeryevich Zhiboedov

Ryen William White

Sarangarajan Parthasarathy

Hugh Evan Williams

Benoit F Dumoulin

Gopi Prashanth Gopal

Michael James Taylor

Gaurav Sareen

Rosie Jones

Filip Andrzej Radlinski

Kevyn Breca Collins-Thompson

Derrick Leslie Connell

Andrew Peter Oakley

Karthik Raghunathan

Jay Goyal

Christopher A Meyers

Umut Ozertem

Timothy C Hoad

Neil Sharman

Muhammad Aatif Awan

Nicholas Craswell

Richard A Caruana

Song Zhou

Bodo Billerbeck

Kyriakos Karenos

Francesco Nidito

Igor Nitto

Siranush Sarkizova

Marc Sloan

Ali Ahmadi

Filiip Radlinski

Systems and methods are provided that automatically process message input and provide action responses according to the processing results. The automatic action response system may leverage at least one machine-learning algorithm that is trained using a dataset. The provided action responses may comprise of default action responses and/or intelligent action responses that are based at least in part on prior conversational data between a user and a sender. Some intelligent action responses may include text-based replies, which eliminate the need for a user to type a reply on a device screen, thereby saving previous time, conserving device battery life, and preserving the integrity of the device hardware. A portion of a message may be highlighted manually by a user or automatically by the action response system to initiate the automatic action response system. In this way, a more efficient and productive user experience across various devices and applications is achieved.

Automatic action responses

Crowdsourcing systems with machine learning are described. Specifically, item-label inference methods and systems are presented, for example, to provide aggregated answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples, an item-label inference system infers variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, an item-label inference system provides aggregated labels while considering the inferred worker characteristics and the inferred characteristics of the worker communities. In examples the item-label inference system provides uncertainty information associated with the inference results for selecting workers and generating future tasks.

Crowdsourcing system with community learning

Various technologies related to generating and applying content retrieval rules are described herein. A content retrieval rule maps a combination of a query and a context to one of a query reformulation or content. The content retrieval rule is learned from search logs of a search engine, and is applied when the query having the context is received at the search engine.

Learning and using contextual content retrieval rules for query disambiguation

Methods and systems are provided for providing alternative query suggestions. For example, a spoken natural language expression may be received and converted to a textual query by a speech recognition component. The spoken natural language expression may include one or more words, terms, and/or phrases. A phonetically confusable segment of the textual query may be identified by a classifier component. The classifier component may determine at least one alternative query based on identifying at least the phonetically confusable segment of the textual query. The classifier may further determine whether to suggest the at least one alternative query based on whether the at least one alternative query is sensical and/or useful. When it is determined to suggest the at least one alternative query, the at least one alternative query may be provided to and displayed on a user interface display.

Interactive reformulation of speech queries

A set of content items, such as web pages, are identified in response to a query generated by a user. The Identified content items are initially ranked using a ranking scheme. User-interaction data that describes preferences that the user may have towards some of the ranked content items is received. In order to personalize the ranking of the content items for the user, the user-interaction data is used to re-rank the ranked content items in a way that favors content items that are preferred by the user, while also preserving the initial broadly applicable ranking with respect to content items that are not preferred or that are equally preferred by the user.

Ranking content items based on preference scores

Crowdsourcing systems with machine learning are described, for example, to aggregate answers to a crowdsourced task in a manner achieving good accuracy even where observed data about past behavior of crowd members is sparse. In various examples a machine learning system jointly learns variables describing characteristics of both individual crowd workers and communities of the workers. In various examples, the machine learning system learns aggregated labels. In examples learnt variables describing characteristics of an individual crowd worker are related, by addition of noise, to learnt variables describing characteristics of a community of which the individual is a member. In examples the crowdsourcing system uses the learnt variables describing characteristics of individual workers and of communities of workers for any one or more of: active learning, targeted training of workers, targeted issuance of tasks, calculating and issuing rewards.

Merging search results is required, for example, where an information retrieval system issues a query to multiple sources and obtains multiple results lists. In an embodiment a search engine at an Enterprise domain sends a query to the Enterprise search engine and also to a public Internet search engine. In embodiments, results lists obtained from different sources are merged using a merging model which is learnt using a machine learning process and updates when click-through data is observed for example. In examples, user information available in the Enterprise domain is used to influence the merging process to improve the relevance of results. In some examples, the user information is used for query modification. In an embodiment a user is able to impersonate a user of a specified group in order to promote particular results.

Merging search results

A system and method are provided for detecting entity information contained within search results. The detected entity information can be used to determine a category of entity as well as a specific entity within the search results. Entity information can be extracted from the documents associated with the search results. This information can be used as part of the information for an entity card, which can be displayed to a user in conjunction with and/or in place of the search results.

Entity detection and extraction for entity cards

Ranking search results using result repetition is described. In an embodiment, a set of results generated by a search engine is ranked or re-ranked based on whether any of the results were included in previous sets of results generated in response to earlier queries by the same user in one or more searching sessions. User behavior data, such as whether a user clicks on a result, skips a result or misses a result, is stored in real-time and the stored data is used in performing the ranking. In various examples, the ranking is performed using a machine-learning algorithm and various parameters, such as whether a result in a current set of results has previously been clicked, skipped or missed in the same session, are generated based on the user behavior data for the current session and input to the machine-learning algorithm.

Ranking search results using result repetition

One or more techniques and/or systems are disclosed for constructing a temporal visualization of query results. That is, query results associated with temporal information, such as timestamps, may be used to create a temporal distribution (e.g., query results comprising images and image capture dates, web pages and web page creation dates, etc.). The temporal distribution may comprise a distribution of the query results over time (e.g., query results may be sorted and/or grouped based upon dates specified by the timestamps). A temporal visualization of the query results may be constructed using the temporal distribution. For example, query results may be assigned to date slots within the temporal visualization. The temporal visualization may illustrate a substantial number of query results (e.g., thousands) that may otherwise not be visually presentable at once. Zoom commands and/or hover commands may be processed to provide finer and/or coarser granularity of the query results.

Temporal visualization of query results

Methods, systems, and media are provided for identifying and clustering queries that are rising in popularity. Resultant clustered queries can be compared to other stored queries using textual and temporal correlations. Fresh indices containing information and results from recently crawled content sources are searched to obtain the most recent query activity. Historical indices are also searched to obtain temporally correlated information and results that match the clustered query stream. A weighted average acceleration of a spike can be calculated to distinguish between a legitimate spike and a non-legitimate spike. Legitimate clusters are combined with other stored clusters and presented as grouped content results to a user output device.

Detecting spiking queries

Growing community of inventors

Cambridge, United Kingdom

Milad Shokouhi