Hardware implementation of frequency table generation for asymmetric-numeral-system-based data compression

Abstract

A lossless data compressor prevents normalization overruns on-the-fly as symbol occurrence counts are rounded to generate symbol frequencies, allowing an encoding table generator to generate encoding table entries without waiting for the symbol frequency table to finish filling. Rounding errors are accumulated as symbols are normalized and compensated for by reducing a symbol frequency when the symbol frequency is at least 2 and the accumulated errors have exceeded a threshold. The symbol frequency is also reduced when the number of remaining states in the encoding table is insufficient for a number of remaining unprocessed symbols and states for a current encoding table entry. Since error compensation occurs as symbols are being normalized, encoding table generation is not forced to wait for all symbols in the block to be processed, reducing latency. Three pipeline stages can operate on three input blocks: symbol counting, normalization/error compensation/encoding table generation, and data encoding.