International Business Machines Corporation

Andreas Neumann

Eugene Jon Shekita

Marcus Felipe Fontoura

Runping Qi

John A McPherson, Jr

Reiner Kraft

Ronny Lempel

Sridhar Rajagopalan

Jason Yeong Zien

Andrei Zary Broder

Ning Li

Roland Seiffert

Jochen F Doerre

Nadav Eiron

Michael Busch

Tom William Jacopi

Monika Matschke

Andrea E Baader

Shila Ofek-Koifman

Felipe Marcus Fontoura

Ting Yu Leung

Sreeram Viswanath Balakrishnan

Michael Herscovici

Tony Leung

Tony Kai-chi Leung

Frederick Stephen Grennan

Pradhan Pattanayak

Christopher John Hinrichs

Liem Gioi Tran

Techniques are disclosed for searching a plurality of documents using a dynamically defined ignore string. The ignore string may be specified by a user. An overlay index may be generated over the plurality of documents. The overlay index may include a posting list for each term in the ignore string. Each posting list may specify the documents of the plurality of documents in which the respective term occurs outside of the ignore string. The overlay index may also include a posting list that specifies all occurrences of the ignore string in the plurality of documents. Once generated, a user may search the plurality of documents while occurrences of the ignore string in the plurality of documents are ignored in text-based searches.

Searching documents using a dynamically defined ignore string

A method and apparatus are provided for enhancing search results generated by a search engine. A graphical user interface may display search results, which may include at least one multi-part record comprising a plurality of independent documents. Upon receiving a selection of the multi-part record from the search results a detailed view of the record may be displayed. The detailed view of the record may include a listing of the multiple documents that form the record along with an indication of whether keywords from the search criteria are present in the documents. By providing an indication of exactly which documents of a multi-part record include keywords associated with a search, embodiments of the invention allow users to easily identify desired materials from a list of search results comprising the multipart record.

Multi-part record searches

Disclosed is a method, system, and program for handling redirects in documents. At least one equivalence class that includes documents that are connected through a redirect. Cycles for each equivalence class are detected, wherein documents in a cycle are marked so that they are not indexed. Incomplete chains for each equivalence class are detected, wherein documents in an incomplete chain are marked so that they are not indexed. A representative for each equivalence class is selected.

Method, system, and program for handling redirects in a search engine

Disclosed is a system and program for processing anchor text for information retrieval. A set of anchors that point to a target document is formed. Anchors with same anchor text are grouped together. Information is computed for each group. Context information is generated for the target document based on the computed information.

System and program for handling anchor text

A method for indexing a plurality of documents, that includes a plurality of duplicate documents, first identifies one or more duplicate groups of documents from among the plurality of documents. Then, one index of content for the duplicate group is created instead of indexing the content from every document within the duplicate group. However, in contrast to the content index, an index of metadata for each of the documents in the duplicate group is created. Thus the content of each duplicate group is indexed only once, while a search engine using such indexing techniques retains the capability to answer queries as if the duplicated content was indexed for each document of the group.

Generic architecture for indexing document groups in an inverted text index

A method and system for querying multifaceted information. An inverted index is constructed to include unique indexed tokens associated with posting lists of one or more documents. An indexed token is either a facet token included in a document as an annotation or a path prefix of the facet token. The annotation indicates a path within a tree structure representing a facet that includes the document. The tree structure includes nodes representing categories of documents. A query is received that includes constraints on documents. The constraints are associated with indexed tokens and corresponding posting lists. An execution of the query includes identifying the corresponding posting lists by utilizing the constraints and the inverted index and intersecting the posting lists to obtain a query result.

Efficient multifaceted search in information retrieval systems

Techniques are disclosed for allowing efficient updating of metadata and high performance searching through the use of a text index and a separate updateable metadata index. Generally, an updateable metadata index is used to store document metadata. A text index is used to store document text. Documents in the text index are stored in the same order as the corresponding metadata entries. Upon receiving a search query, a search engine decomposes the query into a metadata condition and a text condition. Search engine performs a parallel scan upon the metadata index and the text index. To increase performance, metadata entries are skipped over if the corresponding text entries do not match the text condition. During the scan, when a document in the metadata index matches a document in the text index, the document is stored in the search results. After the scan, search results are displayed.

Adding low-latency updateable metadata to a text index

Method, system, and computer program product for query processing are provided. In one implementation, the method comprises receiving a query that includes a full text portion. An invariant part of the full text portion is then compiled and stored. Additional queries received are executed by reusing the compiled invariant part.

Optimizing scalar search in a text indexing and search server

A method, system and computer program product implementing the method are provided to process a text search query in a collection of documents. A full posting index is generated for the documents in the collection. The full posting index comprises one or more first index terms and a full posting list for each first index term, enumerating the occurrences of the first index term in the documents. In addition to the full posting index, at least one additional posting index is generated for the documents. The additional posting index is related to a defined document part and comprises one or more second index terms and a restricted posting list for each second index term, enumerating all occurrences of the second index term in the document part of the documents of the collection. The text search query is performed using the additional posting index.

Method and system for processing a text search query in a collection of documents

Provided are techniques for building a search index. While building the search index and using the search index to respond to one or more search requests, an anchor information store is maintained, wherein each entry of the anchor information store identifies a referring document, a target document, and anchor text associated with a link from the referring document to the target document; a document is received for processing; one or more entries in the anchor information store for which the document to be processed is identified as the target document are located; anchor text is retrieved from each of the identified entries; and the retrieved anchor text is stored in an entry of the search index for the document.

Global anchor text processing

Provided is a technique for building an index. A new indexis built and an anchor text tableand a duplicates tableare output using a store, a delta store, and previously generated global analysis computations, wherein the previously generated global analysis computationsinclude an anchor text table, a rank table, and a duplicates table. New global analysis computationsare generated using the anchor text tablei+1, the duplicates table, and the previously generated global analysis computations.

Pipelined architecture for global analysis and index building

Disclosed is a technique for indexing data. For each token in a set of documents, a sort key is generated that includes a document identifier that indicates whether a section of a document associated with the sort key is an anchor text section or a context section, wherein the anchor text section and the context text section have a same document identifier; it is determined whether a data field associated with the token is a fixed width; when the data field is a fixed width, the token is designated as one for which fixed width sort is to be performed; and, when the data field is a variable length, the token is designated as one for which a variable width sort is to be performed. The fixed width sort and the variable width sort are performed. For each document, the sort keys are used to bring together the anchor text section and the context section of that document.

Architecture for an indexer

Disclosed is a method for processing anchor text for information retrieval. A set of anchors that point to a target document is formed. Anchors with same anchor text are grouped together. Information is computed for each group. Context information is generated for the target document based on the computed information.

Method for handling anchor text

A method for querying multifaceted information. An inverted index is constructed to include unique indexed tokens associated with posting lists of one or more documents. An indexed token is either a facet token included in a document as an annotation or a path prefix of the facet token. The annotation indicates a path within a tree structure representing a facet that includes the document. The tree structure includes nodes representing categories of documents. Constructing the inverted index includes generating a full path token and an associated full path token posting list. A query is received that includes constraints on documents. The constraints are associated with indexed tokens and corresponding posting lists. An execution of the query includes identifying the corresponding posting lists by utilizing the constraints and the inverted index and intersecting the posting lists to obtain a query result.

Disclosed is a technique for indexing data. A token is received. It is determined whether a data field associated with the token is a fixed width. When the data field is a fixed width, the token is designated as one for which fixed width sort is to be performed. When the data field is a variable length, the token is designated as one for which a variable width sort is to be performed.

Growing community of inventors

Gilroy, CA, United States of America

Andreas Neumann