International Business Machines Corporation

Tianchao Li

Namik Hrle

Oliver Draese

Peter Bendel

Vijayshankar Raman

Knut Stolze

Oliver Koeth

A computer implemented method is provided for processing data representing a data entity having sub entities. The method includes analyzing queries to the data entity for deriving information about sets of the sub entities frequently queried together, and grouping the sub entities to a number of banks, each bank having a maximum width, based on the information about sets of sub entities frequently queried together, in order to reduce an average number of banks to be accessed for data retrieval.

Organization of data within a database

A method for generating occurrence data of data values for enabling encoding of a data set includes determining occurrences of data values in a first data batch and determining occurrence count information for a first number of most frequent data values in the first data batch, the occurrence count information identifying the most frequent data values and their occurrence counts. The method also includes generating for rest of the data values in the first data batch at least a first histogram having a second number of buckets and merging the occurrence count information of the first data batch with occurrence count information of a second data batch. The method further includes merging the first histogram of the first data batch to a merged histogram corresponding to the second data batch and processing a next data batch as a first data batch until the data set to be encoded is processed.

Data value occurrence information for data compression

A cell-specific dictionary is applied adaptively to adequate cells, where the cell-specific dictionary subsequently optimizes the handling of frequency-partitioned multi-dimensional data. This includes improved data partitioning with super cells or adjusting resulting cells by sub-dividing very large cells and merging multiple small cells, both of which avoid the highly skewed data distribution in cells and improve the query processing. In addition, more efficient encoding is taught within a cell in case the distinct values that actually appear in that cell are much smaller than the size of the column dictionary.

Growing community of inventors

Boeblingen, Germany

Tianchao Li