Access method data compression with system-built generic dictionaries

Abstract

A computer system constructs a compression dictionary for compressing a character string by interrogating an initial substring portion to determine input string characteristics that are used to select one or more dictionary segments from a library of predetermined dictionary segments individually adapted for compressing strings with particular characteristics. The initial substring portion is dynamically determined during the interrogation. A first set of dictionary segments that meet predetermined automatic selection criteria are selected and a second set of candidate dictionary segments that meet second-level selection criteria are identified for a sampling phase. During the sampling phase, the candidate dictionary segments are alternately used to compress the initial substring portion and determine compression performance statistics. The performance of the dictionary segments in the sampling phase determines which candidate dictionary segments will be added to the first selected dictionary segments, within dictionary total size limits. The first selected dictionary segments and the identified segments constitute a system-built compression dictionary that is used to compress the remainder of the input string. In this way, predetermined compression dictionaries are selected for maximum efficiency in accordance with the data actually being compressed and compression can be carried out quickly and efficiently as input data is received.