Microsoft Technology Licensing, LLC

Gaurav Sareen

Girish Kumar

Pratibha Permandla

Michael Isard

Anna Timasheva

Yinzhe Yu

Abhas Kumar

Deepak Shankar

Qinghua Zou

Jay Goyal

Neil Sharman

Raghavendra Vinayak Prabhu

Junhua Wang

Yofay Kari Lee

Michael Burrows

Mark Steven Manasse

Martin Abadi

Hugh Evan Williams

William Ramsey

Gopi Prashanth Gopal

Darren Arthur Shakib

Sanaz Ahari

Nitin Agrawal

Namita Gupta

Milad Shokouhi

Derrick Leslie Connell

Utkarsh Jain

Andrew Peter Oakley

Kushagra Vaid

Janine Crumb

Christopher A Meyers

Bhuvan Middha

Vibhaakar Sharma

Vinay Deshpande

Rohit Wad

Daniel Yuan

Silviu Petru Cucerzan

Nicholas Craswell

Wee Wah Alfian Tan

Jason Douglas

Charles Lester Alexander Clark

Pei Pei

Systems and methods for regularly updating portions of a merged index are provided. Initially, upon receiving an indication that modifications have occurred to content of web-based documents, dynamic update of index (DUI) objects that identify the documents and expose the modified content are composed by ascertaining relative positions of the modified content within the documents, and packaging identifiers of the documents, the relative positions, and metadata underlying the modified content into a message. The DUI objects are applied to an overloading index that maintains structured records of recent modifications. In particular, portions of the overloading index are targeted utilizing the document identifiers and the relative positions specified by the DUI object, thereby updating the targeted portions within the overloading index corresponding to the modified content without rewriting the entire overloading index. Periodically, an association process is invoked for grouping the merged index with the overloading index for search purposes.

Updating an inverted index in a real time fashion

A search engine can provide referencing information as context for a particular search result when an excerpt from the search result, comprising at least some similar elements to the user's query, is not generated. Referencing information can include one or more anchor texts having similarity to at least some elements of the user's query, the anchor texts being used by referencing pages to link to the page returned as a search result. User selection of the anchor text can enable the user to visit a referencing page using that anchor text to link to the page returned as a search result, and having a high static rank.

Using anchor text to provide context

A search engine system is described herein that provides an alternative data path for collecting results provided by index servers. The alternative data path collects the results in a direct and asynchronous manner; this is in contrast to a synchronous path used to deliver search results to end users via one or more aggregator modules. An analysis system can use the alternative data path to collect a large amount of richly descriptive information regarding the performance of the search engine system, circumventing bottlenecks and other constraints that would otherwise be imposed by the synchronous data path. The analysis system can analyze the information collected from the index servers to improve the performance of the search engine system.

Extraction of rich search information from index servers via an alternative asynchronous data path

Methods, systems, and media are provided for an optimized search engine index. The optimized index is formed by merging small lower level indexes of fresh documents together into a hierarchical cluster of multiple higher level indexes. The optimized index of fresh documents is formed via a single threaded process, while a fresh index serving platform concurrently serves fresh queries. The hierarchy of higher level indexes is formed by merging lower and/or higher level indexes with similar expiration times together. Therefore, as some indexes expire, the remaining un-expired indexes can be re-used and merged with new incoming indexes. The single threaded process provides fast serving of fresh documents, while also providing time to integrate the fresh indexes into a long term primary search engine index, prior to expiring.

Hierarchical merging for optimized index

Systems and methods are provided for regularly updating a web index with new or updated content, such as meta words or meta streams, for a particular web page address, such as a URL. Web page addresses and associated updated information, such as meta words, meta streams, values, and locations in the web index for those meta words are received. In order to update a web index, which is used by search engines to search web documents, a document identification is retrieved and associated with the updated information. As information in the web index is stored by document identification and not by web page addresses, the document identification may replace the web page address. Each meta word received is matched with corresponding document identifications and associated updated information, which creates an inverted format of the information. The web index may now be updated and stored by the system.

Dynamic update of a web index

Search macros suggestions are provided to refine a user's search. When a search query is received from an end user, one or more search macros are determined to be relevant to the search query. The search macros are then provided to the end user as suggestions for refining the user's search. In some instances, the end user may chose to select one of the suggested search macros. A search is then performed using the search query and the selected search macro to provide search results to the end user that may be more relevant to the user's search.

Search macro suggestions relevant to search queries

Methods, systems, and media are provided for identifying and clustering queries that are rising in popularity. Resultant clustered queries can be compared to other stored queries using textual and temporal correlations. Fresh indices containing information and results from recently crawled content sources are searched to obtain the most recent query activity. Historical indices are also searched to obtain temporally correlated information and results that match the clustered query stream. A weighted average acceleration of a spike can be calculated to distinguish between a legitimate spike and a non-legitimate spike. Legitimate clusters are combined with other stored clusters and presented as grouped content results to a user output device.

Detecting spiking queries

Computational units of any task may run in different silos. In an embodiment, a search query may be evaluated efficiently on a non-uniform memory architecture (NUMA) machine, by assigning separate chunks of the index to separate memories. In a NUMA machine, each socket has an attached memory. The latency time is low or high, depending on whether a processor accesses data in its attached memory or a different memory. Copies of an index manager program, which compares a query to an index, run separately on different processors in a NUMA machine. Each instance of the index manager compares the query to the index chunk in the memory attached to the processor on which that instance is running. Thus, each instance of the index manager may compare a query to a particular portion of the index using low-latency accesses, thereby increasing the efficiency of the search.

Leveraging low-latency memory access

Systems, methods, computer-readable media, and graphical user interfaces for presenting search results having collapsed domains are provided. A search result obtaining module obtains search results based upon a received query. Upon obtaining the search results, search results having the same domain are associated with one another. Thereafter, search result clusters of associated search results are formed. In some embodiments, the search result clusters may be formatted to include desired search result cluster attributes. The search result clusters are presented such that two or more associated search results form a single cluster of search results rather than being presented individually. In some embodiments, an option to view more search results with the same domain may be provided.

Domain collapsing of search results

The embodiments contemplate a system and method for obtaining related results for a portion of a query and for generating an updated set of queries for a cache of a server. Other queries beginning with the portion of the query may be identified and obtained from a data structure that includes a server cache and a set of common queries. Once the other queries are identified, results for the other queries are obtained from the server cache or from a back-end database. A set of common queries, which may include deleted and additional queries, may be utilized to generate the updated set of queries for the server. Both missing queries and deleted queries that may belong to the server based on an assignment function are inserted into a queue, which is later delivered to the cache of the server. The transfer may occur during a low-activity or idle state.

Serving cached query results based on a query portion

A probabilistic detector is utilized to query a database. Utilization of a probabilistic detector provides assurance with 100 per cent probability that a search expression in the query is not in the database index. The probabilistic detector is implemented in the form of a Bloom filter. The probabilistic detector is created by hashing expressions in the database index and mapping the resulting hash values into the probabilistic detector. Upon receiving a query, expressions of the query are hashed. The probabilistic detector is queried using these hash values. If the results of querying the probabilistic detector indicate that searched for information may be in the database, the database is not queried. If the results of querying the probabilistic detector indicate that the information may be in the database, the database is queried for the information using the original query. This technique is advantageous in mitigating detrimental effects of denial of service attacks.

Searching for information utilizing a probabilistic detector

Index entries representing ranges of values are queried with a query value to determine if the query value is in one of the ranges. Numeric ranges are represented in an index and queries are constructed to match index entries. Search terms are constructed from the query value, such that a search term matches an index entry if and only if the queried for valued is in the range represented by the index entry. To facilitate querying the indexes, queries are constructed to contain search terms having a data structure format similar to that of the index entries.

Indexing and searching numeric ranges

Growing community of inventors

Bellevue, WA, United States of America

Gaurav Sareen