International Business Machines Corporation

Su Yan

Ying Chen

Sheng Hua Bao

William Scott Spangler

Ioana Roxana Stanoi

Mo Yu

A set of documents is parsed. Members of the set of documents include a set of text elements and a set of visual elements. A text content stream based on the set of text elements and a visual content stream based on the set of visual elements are produced. For respective documents, a set of respective visual element summarizations is built from the visual content stream. Each visual summarization includes a textual description of a respective visual element in the respective document. A holistic index is created by indexing the text content from the text content stream and the text descriptions of the visual elements for each document in a single search index. The indexing uses a set of semantic relationships between the text content from the text content stream and the textual descriptions of the visual elements. A user interface allows a user to selectively search text content and visual content.

Holistic document search

A data handling system enables the unsupervised creation of an information extraction dictionary by expanding upon a word or phrase included within an expansion query. Prior to receiving the expansion query, the data handling system performs an unsupervised learning of an information corpus which includes text to assign a corpus vector to each word and phrase of the text. After the expansion query, the data handling system compares the expansion query to the corpus vectors. The data handling system ranks the corpus vectors by similarity to the expansion query and provides a ranked list of words or phrases associated with the ranked corpus vectors. The ranked list may be subsequently utilized as the information extraction dictionary.

Unsupervised information extraction dictionary creation

A set of documents is parsed. Members of the set of documents include a set of text elements and a set of visual elements. A text content stream based on the set of text elements and a visual content stream based on the set of visual elements are produced. For respective documents, a set of respective visual element summarizations is built from the visual content stream. Each visual summarization includes a text description of a respective visual element in the respective document. A holistic index is created by indexing the text content from the text content stream and the text descriptions of the visual elements in a single search index. The indexing uses a set of semantic relationships between the text content from the text content stream and the textual descriptions of the visual elements. A user interface allows a user to selectively search text content and visual content.

According to one embodiment, a method is provided for approximate named-entity extraction from a dictionary that includes entries, where each of the entries includes one or more words. Words are read from the entries of the dictionary, and network resources are searched to determine a frequency of occurrence of the words on the network resources. In view of the frequency of occurrence of the words located on the network resources, domain relevancy of the words in the entries of the dictionary is determined. A domain repository is created using top-ranked words as determined by the domain relevancy of the words. In view of the domain repository, signatures for both the entries of the dictionary and strings of an input document are computed. The strings of the input document are filtered by comparing the signatures of the strings against the signatures of the entries to identify approximate-match entity names.

Approximate named-entity extraction

According to one embodiment, approximate named-entity extraction from a dictionary that includes entries is provided, where each of the entries includes one or more words. Words are read from the entries of the dictionary, and network resources are searched to determine a frequency of occurrence of the words on the network resources. In view of the frequency of occurrence of the words located on the network resources, domain relevancy of the words in the entries of the dictionary is determined. A domain repository is created using top-ranked words as determined by the domain relevancy of the words. In view of the domain repository, signatures for both the entries of the dictionary and strings of an input document are computed. The strings of the input document are filtered by comparing the signatures of the strings against the signatures of the entries to identify approximate-match entity names.

Growing community of inventors

San Jose, CA, United States of America

Su Yan