Claritech Corporation

Justsystems Evans Research, Inc.

Clairvoyance Corporation

Justsystems Evans Reasearch, Inc.

David A Evans

James G Shanahan

Michael J McInerny

Norbert Roma

Jesse Montgomery

Victor M Sheftel

Jeffrey Bennett

Gregory T Grefenstette

Michael L Horowitz

Thomas P Neuendorffer

Christopher C Lichti

Xiang Tong

A computer-readable medium comprises data structure for providing information about levels of similarity between pairs of N documents. The data structure comprises a plurality of entries of similarity values representing levels of similarity for a plurality of pairs of the documents. Each of the similarity values represents a level of similarity of one document of a given pair relative to the other document of the given pair. The similarity value of each entry is greater than a threshold similarity value that is greater than zero. The plurality of similarity-value entries are fewer than N&#x2212;N in number if the similarity values are asymmetric with regard to document pairing, and the plurality of similarity-value entries are fewer than in number if the similarity values are symmetric with regard to document pairing. A method and apparatus for generating the data structure are described.

Method and apparatus for constructing a compact similarity structure and for using the same in analyzing document relevance

A method and system for computer-assisted analysis of information from a data source employs operators including search terms and filters. A visual representation of a network of operators is generated, the operators being user selectable and linked to form the network, which is configured to analyze information from a selectable data source. The network of operators can be compiled as a unit. If the data source includes a database, the operators are applied to the database, and if the data source includes a data stream, a data unit of the data stream is evaluated with the operators. An output indicator corresponding to each of the operators is generated on the visual representation of the network, wherein the output indicator visually represents a quantified output of a corresponding operator. A further output corresponding to one or more of the output indicators is also generated.

Method and apparatus for information mining and filtering

A technique for representing an information need and employing one or more filters to select documents that satisfy the represented information need, including a technique of creating filters that involves (a) dividing a set of documents into one or more subsets such that each subset can be used as the source of features for creating a filtering profile or used to set or validate the score threshold for the profile and (b) determining whether multiple profiles are required and how to combine them to create an effective filter. Multiple profiles can be incorporated into an individual filter and the individual filters combined to create an ensemble filter. Ensemble filters can then be further combined to create meta filters.

Method and apparatus for document filtering using ensemble filters

A method and system identifying the language of a textual passage is disclosed. The method and system includes parsing the textual passage into n-grams and assigning an initial weight to each n-gram, and adjusting the weight initially assigned to a word or n-gram parsed from the textual passage. The initially assigned weight is adjusted in a manner proportionate to the inverse of the number of languages within which such words or n-grams appear. Reducing the weight assigned to such words or n-grams diminishes&#x2014;without completely eliminating&#x2014;their importance in comparison to other words or n-grams parsed from the same textual passage when determining the language of a passage. The method and system of the present invention appropriately weighs the short words or n-grams common to multiple languages without affecting the short words or n-grams that are uncommon to several languages.

Method of identifying the language of a textual passage using short word and/or n-gram comparisons

An information need can be modeled by a binary classifier such as support vector machine (SVM). SVMs can exhibit very conservative precision oriented behavior when modeling information needs. This conservative behavior can be overcome by adjusting the position of the hyperplane, the geometric representation of a SVM. The present invention describes a couple of automatic techniques for adjusting the position of an SVM model based upon a beta-gamma thresholding procedure, cross fold validation and retrofitting. This adjustment technique can also be applied to other types of learning strategies.

Method and apparatus for adjusting the model threshold of a support vector machine for text classification and filtering

The present mechanism provides a method and apparatus for retrieving documents that are stored in a language other than the language that is used to formulate a search query. This invention decomposes the query into terms and then translates each of the terms into terms of the language of the database. Once the database language terms have been listed, a series of subqueries is formed by creating all the possible combinations of the listed terms. Each subquery is then scored on each of the documents in the target language database. Only those subqueries that return meaningful scores are relevant to the query. Thus, the semantic meaning of the query is determined against the database itself and those documents in the database language that are most relevant to that semantic meaning are returned.

Method for cross-linguistic document retrieval

A method and apparatus for retrieving similar or identical textual passages among different documents is disclosed. Normal discourse structures along with textual content attributes are used to encode a known passage with &ldquo;marker sequences&rdquo; that give a characterizing &ldquo;signature&rdquo; to the passage. The encoded known passage is then evaluated against similarly encoded passages appearing in a database of documents. If it is determined that there is a possible match between the encoded known passage and an encoded passage in a database document, a sequential string search is performed to determine whether the two passages are likely to be similar or identical. If the sequential string search records a probable match between the known passage and the database passage, the database passage is displayed for further review.

Method and apparatus for retrieving text using document signatures

An document image that is the source of Optical Character Recognition (OCR) output is displayed so that a user can select a region of the displayed document image. When the region is selected, text of the OCR output corresponding to the selected region is submitted as an input to a search engine.

Apparatus and methodology for submitting search oueries

The present invention is a method and apparatus for retrieving information from a database. Initially, the documents within the database are divided into mutually exclusive subdocuments that generally correspond to paragraphs of text. The present invention further creates a second set of subdocuments that overlap adjacent paragraphs of text. In particular, the location of the overlapping subdocuments depends on the size of the initial paragraphs. This second set of overlapping subdocuments are scored just as the mutually exclusive subdocuments are scored. The scores from both the mutually exclusive and overlapping subdocuments are used in ranking the relevance of documents to a query. The use of both sets of subdocument scores improves the effectiveness of the scoring algorithm.

Overlapping subdocuments in a vector space search process

In a novel approach for retrieving information a set of sub-documents first is established based upon a set of documents. A query is processed which operates on the set of sub-documents, causing a score to be generated for each sub-document. The score for each sub-document is indicative of the relevance of the corresponding sub-document to the query. The scores are reviewed and the best sub-document is retrieved. According to one aspect of the invention, the best sub-document has a score that indicates the highest relevance between the sub-document and the query. According to another aspect of the invention, in response to a user selection, the next best sub-document is identified and retrieved. The sub-documents are also presented to the user in an order based upon the scores. According to another aspect of the invention, the document containing the sub-document having the best score is displayed and automatically scrolled to the location of the sub-document having the best score. Based upon user input, the document is automatically scrolled to the location containing the next best sub-document. If the next best sub-document is contained in another document, that document is automatically loaded.

Information retrieval based on using and locating subdocuments

Document texts are produced by recognizing characters in document images by an Optical Character Recognition (OCR) process. When such a document text matches one or more search terms of a query, the corresponding document image is displayed. Regions of the document image, corresponding to words of the document text that match the search terms, are displayed in a visually distinctive manner. The display of the document image may be augmented by displaying a region corresponding to a reference text within the document text in another visually distinctive manner.

Methodology for displaying search results using character recognition

The present invention provides a method and apparatus for retrieving documents that are stored in a language other than the language that is used to formulate a search query. This invention decomposes the query into terms and then translates each of the terms into terms of the language of the database. Once the database language terms have been listed, a series of subqueries is formed by creating all the possible combinations of the listed terms. Each subquery is then scored on each of the documents in the target language database. Only those subqueries that return meaningful scores are relevant to the query. Thus, the semantic meaning of the query is determined against the database itself and those documents in the database language that are most relevant to that semantic meaning are returned.

Growing community of inventors

Pittsburgh, PA, United States of America

David A Evans