High speed large scale dictionary matching

Abstract

A mechanism is provided for dictionary matching. The mechanism loads a plurality of dictionary memory arrays with a set of dictionary words and updates a plurality of status arrays. Each status array of the plurality of status arrays corresponds to a respective one of the plurality of dictionary memory arrays. Each entry of a given status array stores a status bit, which indicates whether a corresponding entry of the corresponding dictionary memory array stores a valid dictionary word. The mechanism receives an input data word and generates a hash value based on the input data word. The mechanism reads a dictionary word from each of the dictionary memory arrays and a status bit from each of the status arrays using the hash value as a read address. The mechanism determines whether a dictionary memory array within the plurality of dictionary memory arrays stores a valid dictionary word that matches the input data word.