Xerox Corporation

Lauri J Karttunen

Ronald M Kaplan

Jan O Pedersen

Martin Kay

Annie E Zaenen

Per-Kristian G Halvorsen

Douglass R Cutting

Kenneth R Beesley

A system constructs finite-state networks. The system initially compiles an intermediate finite-state network from a source file of regular expressions. The intermediate finite-state network includes a delimited subpath that defines a substring having the form of a regular expression. The system subsequently produces an output finite-state network in which the delimited subpath is replaced with an FSN compiled from the substring encoded by the delimited subpath.

Method and apparatus for constructing finite-state networks modeling non-concatenative processes

Valid positions for hyphens in input strings are determined by reading in and processing the symbols of the input string through a finite state transducer which has a state-transition data structure determined by a compilation of a set of hyphenation rules. The output of the encoding system can include a hyphenated string, or can accept a hyphenated string and output an indication of whether the input hyphenation is proper according to the set of hyphenation rules.

Finite-state encoding system for hyphenation rules

A technique of using an electronic dictionary in conjunction with electronically-encoded running text that gives the user the most relevant information rather than belaboring the user with all possible information about a selected word. The technique maps the selected word from its inflected form to its citation form, analyzes the selected word in the context of neighboring and surrounding words to resolve ambiguities, and displays the information that is determined to be the most likely to be relevant. The dictionary preferably has information about multi-word combinations that include the selected word, and the context determination typically entails checking whether the selected word is part of a predefined multi-word combination.

Context-sensitive method of finding information about a word in an

The present invention solves a number of problems in using stems (canonical indicators of word meanings) in full-text retrieval of natural language documents, and thus permits recall to be improved without sacrificing precision. It uses various arrangements of finite-state transducers to accurately encode a number of desirable ways of mapping back and forth between words and stems, taking into account both systematic aspects of a language's morphological rule system and also the word-by-word irregularities that also occur. The techniques described apply generally across the languages of the world and are not just limited to simple suffixing languages like English. Although the resulting transducers can have many states and transitions or arcs, they can be compacted by finite-state compression algorithms so that they can be used effectively in resource-limited applications. The invention contemplates the information retrieval system comprising the novel finite state transducer as a database and a processor for responding to user queries, for searching the database, and for outputting proper responses, if they exist, as well as the novel database used in such a system and methods for constructing the novel database.

Finite-state transduction of related word forms for text indexing and

A method and apparatus for adding a word to a lexical transducer in a computer system. The invention allows a user of the computer system to specify a word to be added to the lexical transducer database. The lexical transducer represents words as ordered sequences of symbols, i.e., characters and morphological tags. 'Upper' and 'lower' symbols are associated with arcs. The arcs join states and form a path. Each path determines an upper and lower sequence of ordered symbols. The upper sequence of symbols represents a base form of a word and the lower sequence of symbols represents a surface form of the same word. The user adds a word to the lexical transducer by specifying a 'model' word already existing in the lexical transducer, along with a new word that has surface forms analogous to the model word. The new word is added to the lexical transducer by sharing, as much as possible, the existing arcs of the path of the model word.

Growing community of inventors

Redwood City, CA, United States of America

Lauri J Karttunen