International Business Machines Corporation

Permabit Technology Corporation

Burnside Acquisition, LLC

Jeffrey M Keller

Garth Allen Dickie

William B Ackerman

Norman H Margolus

Jered J Floyd

George E Homsy, Ii

Daniel Dietterich

Calisto Paul Zuzarte

Adam J Storm

Ian R Finlay

According to embodiments of the present invention, machines, systems, methods and computer program products for processing data are provided. Compressed data is received and a requested operation for uncompressed data is performed on the compressed data by determining an intermediate location in a compression hierarchy of compression nodes and applying the requested operation to the data at that intermediate location.

Query predicate evaluation and computation for hierarchically compressed data

Elements of a database object are removed. The database object is stored as a plurality of different object portions, where each object portion is associated with one or more versions of transaction identifiers stored separately from the database object. An oldest transaction identifier is determined for a transaction for which data portions of the database object remains visible. Each object portion is examined and object portions with a threshold amount of data to remove are determined based on a comparison of the transaction identifiers for those object portions and the oldest transaction identifier. Data from the database object are removed in response to a sufficient quantity of data is to be removed from object portions containing the threshold amount of data.

Removal of garbage data from a database

An illustrative embodiment for optimizing scans using a Bloom filter synopsis, defines metadata to encode distinct values in a range of values associated with a particular portion of a managed object in a database management system into a probabilistic data structure of a Bloom filter that stores an indicator, encoded in a fixed size bit map with one or more bits, indicating whether an element of the particular portion of the managed object is a member of a set of values summarized in the Bloom filter using a value of 1 or definitely not in the set using a value of 0. The Bloom filter is compressed to create a compressed Bloom filter. The Bloom filter is added to the metadata associated with the managed object and used when testing for values associated with predicates.

Scan optimization using bloom filter synopsis

According to embodiments of the present invention, apparatus, systems, methods and computer program products for sorting and compressing an unordered set of data records from a structured database are provided. Fields of the unordered set of data records are prioritized based on an impact of those fields to a compression scheme for column-oriented compression. The unordered set of data records are sorted based on the prioritized field(s) with a greatest impact on the performance metric. Data of the sorted data records are compressed according to a compression scheme. In some embodiments, prioritizing the fields may be based on an anticipated level of usage of data within those fields and/or a cost function associated with a performance metric as well as optimization of compression. A performance metric may include a faster computational time, reduced I/O computation, faster scan time, etc.

Reordering of database records for improved compression

According to embodiments of the present invention, machines, systems, methods and computer program products for hierarchical compression of data are presented comprising creating a compression hierarchy of compression nodes, wherein each compression node is associated with a compression operation to produce compressed data. An output of any of the compression nodes may be compressed by another compression node or the same compression node. A path of one or more compression nodes is determined through said compression hierarchy based upon compression statistics to compress data, and the data is compressed by the compression nodes of the path. Various computational techniques are presented herein for manipulating the compression hierarchy to defer or reduce computation during query evaluation.

Growing community of inventors

Cambridge, MA, United States of America

Jeffrey M Keller