System and methodology for a multi-site search engine

Abstract

Techniques for query processing in a multi-site search engine are described. During an indexing phase, each site of a multi-site search engine indexes a set of assigned web resources and each site calculates, for each term in the set of assigned web resources, a site-specific upper bound ranking score on the contribution of the term to the search engine ranking function for a query containing the term. During a propagation phase, all sites exchange their site-specific upper bound ranking scores with each other. In response to a site receiving a query, the site determines the set of locally matching resources and compares the ranking score of a locally matching resource with the site-specific upper bound ranking scores for the terms of the query that were received during the propagation phase and determines whether to communicate the query to other sites. By exchanging appropriately defined site-specific upper bound ranking scores, the site initially receiving the query can determine whether the locally matching resources would be identical to the resources obtained from a single-site search system without having to communicate the query to each of the other sites.