Method for typographical detection and replacement in japanese text

Abstract

Detecting typographical errors in a Japanese sentence by using a bottom-up approach analysis. The bottom-up analysis employs probabilities, dictionaries and heuristics to words that are found in morpho-lexical information derived from the Japanese sentence. This bottom-up approach combines valid phrases analyses into well-formed combined phrases, i.e., phrase lists, to determine the existence of “holes”. Holes are characters contained in the input sentence but not in the well-formed phrase lists. Probabilities are used to determine which phrase list is most representative of the input sentence. The hole contained in the phrase list having the lowest cost (highest probability) is analyzed to determine if it is a typographical error. This analysis includes checking the hole to determine if it is an extended dictionary and whether it is a proper noun. The hole may be “relaxed” by adding contiguous characters and rechecking the “relaxed” hole in the extended dictionary to determine if it is a proper noun. If the hole represents a typographical error, a replacement string is generated using reverse transformations to counteract the text entry error which created the typographical error. A dictionary is used in the replacement string generation process to determine the valid phrases.