International Business Machines Corporation

Yahoo! Inc.

Jeonghee Yi

Neelakantan Sundaresan

Balakrishna Raghavendra Iyer

Atul Chadha

Hammou Messatfa

Marc Eliot Davis

Amit Kumar

Tetsuya Nasukawa

Amit Basu

Kevin Haas

Razvan Constantin Bunescu

Paul Tarjan

The present invention provides a method and system for generating search results including receiving a search request including at least one search term and accessing a corpus of data to determine relevant content for inclusion in a search result set on the basis of the search request. The method and system includes determining a plurality of applications associated with the search request and generating a search result output display for the presentation of at least a portion of the search result set and at least a portion of the applications.

Systems and methods for providing advanced search result page content

A method and system for extracting opinions about a subject of interest from a text document in which each sentence is analyzed individually to identify the opinions. The most relevant feature terms related to the subject are extracted from the document based on their relevancy scores. Candidate feature terms are definite noun phrases at the beginning of the sentences. For each sentence that refers to the subject or a feature term, the invention determines whether the sentence includes an opinion polarity about the subject or the feature term. The opinion polarity is detected by identifying opinion terms in the sentence using an opinion dictionary or an opinion rule base, parsing the sentence with an English parser to identify grammatical components in the sentence and their relationships, and finding a matching entry in the dictionary or the rule base.

Method and system for extracting opinions from text documents

A system for sharing data within a network, the system including a first peer device coupled with the network that comprises local cache storage configured to store data comprising at least one entry designated as network accessible cache data and a cache control module operative to control access to the data stored in the local cache storage. The system further includes a second peer device coupled with the first peer device via the network where the second peer device is configured to request network accessible cache data stored in the local cache storage of the first peer device. Furthermore, the cache control module of the first peer device is configured to transmit at least a portion of the requested network accessible cache data to the second peer device in response to the request for network accessible data stored from the second peer device.

Community caching networks

A computer program product is provided as an automatic mining system to identify a set of relevant terms from a large text database of unstructured information, such as the World Wide Web (WWW), with a high degree of confidence, by association mining and refinement of co-occurrences using hypertext link metadata. The automatic mining system includes a software package comprised of a metadata extractor, a document vector module, an association module, and a filtering module. The automatic mining system further includes a database for storing the mined sets of relevant terms. The automatic mining system scans the downloaded hypertext links, rather than the entire body of the documents for related information. As a result, the crawler is not required to provide a relatively lengthy download of the document content, and thus, the automatic mining system minimizes the download and processing time.

System and method for the automatic recognition of relevant terms by mining link annotations

A computer program product is provided as an automatic mining system to discover terms that are relevant to a given target topic from a large databases of unstructured information such as the World Wide Web. The operation of the automatic mining system is performed in three stages: The first stage is carried out by a new terms discoverer for discovering the terms in a document, the second stage is carried out by a candidate terms discoverer for discovering potentially relevant terms, and the third stage is carried out by a relevant terms discoverer for refining or testing the discovered relevance to filter false relevance. The new terms discoverer includes a system for the automatic mining of patterns and relations, a system for the automatic mining of new relationships, and a system for selecting new terms from relations. In one embodiment, the system for the automatic mining of patterns and relations identifies a set of related terms on the WWW with a high degree of confidence, using a duality concept, and includes a terms database and two identifiers: a relation identifier and a pattern identifier. The system for the automatic mining of new relationships includes a database a knowledge module and a statistics module. The knowledge module includes a stemming unit, a synonym check unit, and a domain knowledge check unit. The candidate terms discoverer includes a metadata extractor, a document vector module, an association module, a filtering module, and a database. The relevant terms discoverer includes a stop word filter and a system for the automatic construction of generalization&mdash;specialization hierarchy of terms comprised of a terms database, an augmentation module, a generalization detection module, and a hierarchy database.

System and method of automatic discovery of terms in a document that are relevant to a given target topic

A classifier for semi-structured documents and associated method dynamically and accurately classify documents with an implicit or explicit schema by taking advantage of the term-frequency and term distribution information inherent in the document. The system uses a structured vector model that allows like terms to be grouped together and dissimilar terms to be segregated based on their frequency and distribution within the sub-vectors of the structure vector, thus achieving context sensitivity. The final decision for assigning the class of a document is based on a mathematical comparison of the similarity of the terms in the structured vector to those of the various class models. The classifier of the present invention is capable of both learning and testing. In the learning phase the classifier develops models for classes with information it develops from the composite information gleaned from numerous training documents. Specifically, it develops a structured vector model for each training document. Then, within a given class of documents it adds and then normalizes the occurrences of terms.

Method and system for classifying semi-structured documents

An automatic mining system that identifies a set of relevant terms from a large text database of unstructured information, such as the World Wide Web with a high degree of confidence. The automatic mining system includes a software program that enables the discovery of new relationships by association mining and refinement of co-occurrences, using automatic and iterative recognition of new binary relations through phrases that embody related pairs, by applying lexicographic and statistical techniques to classify the relations, and further by applying a minimal amount of domain knowledge of the relevance of the terms and relations. The automatic mining system includes a knowledge module and a statistics module. The knowledge module is comprised of a stemming unit, a synonym check unit, and a domain knowledge check unit. The stemming unit determines if the relation being analyzed shares a common root with a previously mined relation. The synonym check unit identifies the synonyms of the relation, and the domain knowledge check unit considers extrinsic factors for indications that would further clarify the relationship being mined. The statistics module optimizes the confidence level in the relationship.

System and method for the automatic mining of new relationships

A computer program product is provided as an automatic mining system to build a generalization hierarchy of terms from a database of terms and associated meanings, using for example the Least General Generalization (LGG) model. The automatic mining system is comprised of a terms database, an augmentation module, a generalization detection module, and a hierarchy database. The terms database stores the terms and their meanings, and the hierarchy database stores the generalization hierarchy which is defined by a set of edges and nodes. The augmentation module updates the terms using the LGG model. The generalization detection module maps the generalizations derived by the augmentation module, updates the edges, and derives a generalization hierarchy. In operation, the automatic mining system begins with no predefined taxonomy of the concept terms, and the LGG model derives a generalization hierarchy, modeled as a Directed Acyclic Graph from the terms.

System and method for the automatic construction of generalization-specialization hierarchy of terms from a database of terms and associated meanings

A computer program product is provided as an automatic mining system to identify a set of related terms on the World Wide Web that define a relationship, using the duality concept. Specifically, the mining system iteratively refines pairs of terms that are related in a specific way, and the patterns of their occurrences in web pages. The automatic mining system runs in an iterative fashion for continuously and incrementally refining the relates and their corresponding patterns. In one embodiment, the automatic mining system identifies relations in terms of the patterns of their occurrences in the web pages. The automatic mining system includes a relation identifier that derives new relations, and a pattern identifier that derives new patterns. The newly derived relations and patterns are stored in a database, which begins initially with small seed sets of relations and patterns that are continuously and iteratively broadened by the automatic mining system.

System and method for automatically and iteratively mining related terms in a document through relations and patterns of occurrences

A computer program product is provided as an automatic mining system to identify a set of related information on the World Wide Web using the duality concept. The mining system addresses iteratively refines mutually dependent approximations to their identifications. Specifically, the mining system iteratively refines (i) pairs of phrases related in a specific way; (ii) the patterns of their occurrences in web pages; and (iii) the formation rules. In one embodiment, the automatic mining system identifies (acronym, expansion) pairs in terms of the patterns of their occurrences in the web pages and their formation rules. The automatic mining system includes a formation rule identifier that derives the formation rules, an acronym-expansion pair identifier that derives the (acronym, expansion) pairs, and a pattern identifier that derives the patterns. The database stores the (acronym, expansion) pairs, patterns, and formation rules. Initially, the database begins with small seed sets of (acronym, expansion) pairs, patterns, and formation rules that are continuously and iteratively broadened by the automatic mining system.

System and method for the automatic mining of acronym-expansion pairs patterns and formation rules

A method, apparatus, and article of manufacture for a computer-implemented random reliability engine for computer-implemented association rule reduction using association rules for data mining application. The data mining is performed by the computer to retrieve data from a data store stored on a data storage device coupled to the computer. The data store has records that have multiple attributes. Attribute value associations are determined between attributes and their values. Attribute associations are determined from the determined attribute value associations. Attributes are selected based on the determined attribute associations for performing data mining.

Dimension reduction using association rules for data mining application

A method, apparatus, and article of manufacture for a computer-implemented random reliability engine for computer-implemented dimension reduction using association rules for data mining application. The data mining is performed by the computer to retrieve data from a data store stored on a data storage device coupled to the computer. The data store has records that have multiple attributes. The multiple attributes of a table are clustered to produce a plurality of sets of attributes. Each set of attributes is clustered to obtain data mining attributes.

Growing community of inventors

San Jose, CA, United States of America

Jeonghee Yi