Method and apparatus for diagnosing lexical errors

Abstract

A method and lexical analyzer for diagnosing lexical errors in a stream of symbols. An error-diagnosing lexical analyzer is constructed by a lexical analyzer generator based on a list of regular expression patterns. The list of regular expression patterns includes patterns which define specific invalid lexeme types as well as specific valid lexeme types. The lexical analyzer generated based on the list of regular expression patterns has a first plurality of states and a second plurality of states. Each state of the first plurality of states represents a specific valid lexeme type and is associated with a label identifying the specific valid lexeme type. Each state of the second plurality of states represents a specific invalid lexeme type and is associated with a label identifying the specific invalid lexeme type. The lexical analyzer reads symbols from a stream of symbols and changes state based on the symbols according to transition rules. When the lexical analyzer reads a terminating character, the lexical analyzer transmits a signal indicating the lexeme type associated with the state of the lexical analyzer at the time the terminating symbol was encountered. The lexeme type indicated in the signal is used to characterize the series of symbols processed prior to encountering the terminating symbol.