International Business Machines Corporation

Andrei Zary Broder

Michael Herscovici

Ronny Lempel

Eugene Jon Shekita

Nadav Eiron

John A McPherson, Jr

David Carmel

Aya Soffer

Marcus Felipe Fontoura

Andreas Neumann

Jason Yeong Zien

Runping Qi

Ning Li

Shila Ofek-Koifman

Felipe Marcus Fontoura

Andrew Stephen Tomkins

David Angelo Ferrucci

Wlodek Wlodzimierz Zadrozny

Yoelle Smadja Maarek

Yosi Mass

Arthur Charles Ciccolo

Ziv Bar-Yossef

Shanmagasundaram Ravikumar

Marcus Fontoura

John McPherson, Jr

Eugene Shekita

A method is carried out by storing information describing configurations of discussion threads formed of respective series of EMTs that are exchanged among at least two individuals. The discussion threads have a root EMT, zero or more reply EMTs, and a last offspring EMT. The method is further carried out by compacting the EMT discussion threads, and indexing the compacted EMT discussion threads.

Indexing and searching of electronic message transmission thread sets

A method includes describing the thread configurations of a volume of well-ordered electronic message transmissions (EMT) and utilizing the thread configuration data to conduct selective searches of the EMT volume. An apparatus includes a thread processor and a query manager. The thread processor analyzes the EMT threads and records the thread configuration data. The query manager utilizes the thread configuration data to conduct selective searches of the EMT volume.

Disclosed is a system architecture, components and a searching technique for an Unstructured Information Management System (UIMS). The UIMS may be provided as middleware for the effective management and interchange of unstructured information over a wide array of information sources. The architecture generally includes a search engine, data storage, analysis engines containing pipelined document annotators and various adapters. The searching technique makes use of a two-level searching technique. A search query includes a search operator containing of a plurality of search sub-expressions each having an associated weight value. The search engine returns a document or documents having a weight value sum that exceeds a threshold weight value sum. The search operator is implemented as a Boolean predicate that functions as a Weighted AND (WAND).

System, method and computer program product for performing unstructured information management and automatic text analysis, including a search operator functioning as a Weighted AND (WAND)

A method for indexing a plurality of documents, that includes a plurality of duplicate documents, first identifies one or more duplicate groups of documents from among the plurality of documents. Then, one index of content for the duplicate group is created instead of indexing the content from every document within the duplicate group. However, in contrast to the content index, an index of metadata for each of the documents in the duplicate group is created. Thus the content of each duplicate group is indexed only once, while a search engine using such indexing techniques retains the capability to answer queries as if the duplicated content was indexed for each document of the group.

Generic architecture for indexing document groups in an inverted text index

A method and system for querying multifaceted information. An inverted index is constructed to include unique indexed tokens associated with posting lists of one or more documents. An indexed token is either a facet token included in a document as an annotation or a path prefix of the facet token. The annotation indicates a path within a tree structure representing a facet that includes the document. The tree structure includes nodes representing categories of documents. A query is received that includes constraints on documents. The constraints are associated with indexed tokens and corresponding posting lists. An execution of the query includes identifying the corresponding posting lists by utilizing the constraints and the inverted index and intersecting the posting lists to obtain a query result.

Efficient multifaceted search in information retrieval systems

A signal-bearing medium is disclosed that includes operations including establishing a link threshold, wherein a web page will be assessed as lacking currency if a percentage of hyperlinks contained in the web page that link to an active page is less than the link threshold, accessing a web page containing hyperlinks, and testing the hyperlinks. Testing includes: selecting a hyperlink; and monitoring a number of redirects encountered by following the selected hyperlink until a final web page is reached or a failure occurs, and assessing the selected hyperlink as linking to a dead web page if a redirect limit is exceeded, the redirect limit greater than one, wherein exceeding the redirect limit causes occurrence of a failure. The operations also include calculating a percentage of hyperlinks that return active web pages, and comparing the percentage of hyperlinks that return active web pages with the link threshold.

Methods and apparatus for assessing web page decay

System, method and computer program product for performing unstructured information management and automatic text analysis, including a search operator functioning as a weighted and (WAND)

A method for querying multifaceted information. An inverted index is constructed to include unique indexed tokens associated with posting lists of one or more documents. An indexed token is either a facet token included in a document as an annotation or a path prefix of the facet token. The annotation indicates a path within a tree structure representing a facet that includes the document. The tree structure includes nodes representing categories of documents. Constructing the inverted index includes generating a full path token and an associated full path token posting list. A query is received that includes constraints on documents. The constraints are associated with indexed tokens and corresponding posting lists. An execution of the query includes identifying the corresponding posting lists by utilizing the constraints and the inverted index and intersecting the posting lists to obtain a query result.

Disclosed is a system architecture, components and a searching technique for an Unstructured Information Management System (UIMS). The UIMS may be provided as middleware for the effective management and interchange of unstructured information over a wide array of information sources. The architecture generally includes a search engine, data storage, analysis engines containing pipelined document annotators and various adapters. The searching technique makes use of a two-level searching technique. Also disclosed is system, method and computer program product to process document data. The method includes inputting a document and operating at least one text analysis engine that comprises a plurality of coupled annotators for tokenizing document data for identifying and annotating a particular type of semantic content. Operating the at least one text analysis engine generates a plurality of views of a document, where each of the plurality of views are derived from a different tokenization of the document. The method further includes storing the plurality of views in a common data structure associated with the document.

System, method and computer program product for performing unstructured information management and automatic text analysis, and providing multiple document views derived from different document tokenizations

Growing community of inventors

Bronx, NY, United States of America

Andrei Zary Broder