International Business Machines Corporation

Timothy Ray Malkemus

Bishwaranjan Bhattacharjee

Sriram Kolinjivadi Padmanabhan

Matthew Albert Huras

Christian Alexander Lang

Michael Jeffrey Winer

Leslie Anne Cranston

Mohammad Sadoghi Hamedani

George Andrei Mihaila

Lipyeow Lim

Calisto Paul Zuzarte

Adam J Storm

Sam Sampson Lightstone

Ian R Finlay

Yuan-Chi Chang

Naresh K Chainani

David C Kalmuk

Daniele Costante Zilio

Gary Randall Horn

Kiran Chinta

Catherine Suzanne McArthur

Daniel J Coyle, Jr

Tony Wen Hsun Lai

Sherman Lau

Leslie Anne Buback

Kwai Wong

Bruce Gilbert Lindsay

Eugene Jon Shekita

David Everett Simmen

Xun Xue

Oktie Hassanzadeh

Ronen Grosman

Aamer U Sachedina

Ioana Roxana Stanoi

Ramesh Chandra Agarwal

Keriley Kay Romanufa

Anant Deep Jhingran

James Russell Boykin

Philip Y Chang

Lubor J Kollar

Amy Chang

Fred Koo

Sean W McKeough

Kaarel Truuvert

Yuan Chi Chang

Walter Gene Wilson

Jo-anne Mona Louise Kirton

Petrus K C Chan

Kelly W Schlamb

Nickolay V Tchervenski

Philip J Welti

Rebecca A Rodriguez

Lisa A Geer

Cathy McArthur

Reza Sherkat

Zoltan G Toth

Techniques are provided for pre-filtering of join execution over multi-column range summaries and other synopses. An exemplary method comprises maintaining a synopsis for a plurality of data tables, wherein a given synopsis summarizes a set of records in a corresponding data table; and, in response to a request for a join operation for a set of the data tables: joining the synopses associated with the set of data tables to generate a joined synopsis; for joined records in the joined synopsis, obtaining corresponding records from the set of data tables as candidate records; and joining the candidate records. Two or more of the set of data tables can be distributed across a plurality of nodes and the synopses can be replicated and/or broadcasted across the plurality of nodes. Incremental updates to broadcasted and/or replicated synopses are optionally provided to at least one node.

Pre-filtering of join execution over multi-column range summaries and other synopses

A computer-implemented method for a partitioned bloom filter merge is provided. A non-limiting example of the computer-implemented method includes partitioning, by a processing device, a bloom filter into N equal size filter partitions. The method further includes distributing, by the processing device, each of the filter partitions to an associated node. The method further includes merging, by the processing device, the filter partitions in each of the associated nodes. The method further includes redistributing, by the processing device, the merged filter partitions to each of the N nodes. The method further includes joining, by the processing device, the merged filter partitions in each of the N nodes to assemble a complete merged bloom filter.

Partitioned bloom filter merge for massively parallel processing clustered data management

Conventionally, in addition to indexing, a synopsis of a base table of a database is used to skip and compress data. However, scanning of the entire synopsis for all queries is required, which takes a long time when the synopsis gets significantly big in a large data warehouse. A method for efficient data skipping and compression through vertical partitioning of data is provided to eliminate the cost of synopsis storage overhead while enabling the synopsis search functionality.

Data skipping and compression through partitioning of data

Organizing data within a database is provided. In response to determining that a group of coarsified data records within a database table is not an aligned group of data records, a virtually replicated subgroup of coarsified data records that corresponds to the group of coarsified data records is generated from different groups of coarsified data records within the database table. The virtually replicated subgroup of coarsified data records is aligned with the corresponding group of coarsified data records.

Database query processing using horizontal data record alignment of multi-column range summaries

An ordering of operations in log records includes: performing update operations on a database object by a node; writing log records for the update operations into a local buffer by the node, the log records each including a local virtual timestamp; determining that a log flush to write the log records in the local buffer to a persistent storage is to be performed; in response, sending a request from the node to a log flush sequence server for a log flush sequence number; receiving the log flush sequence number by the node; inserting the log flush sequence number into the log records in the local buffer; and performing the log flush to write the log records in the local buffer to the persistent storage, where the log records written to the persistent storage comprises the local virtual timestamps and the log flush sequence number.

Methods to minimize communication in a cluster database system

Techniques are disclosed for selecting a delete-safe compression method for a plurality of delta encoded data values (e.g., delta encoded integers or deltas). For example, a computer-implemented method for selecting an optimal delete-safe compression algorithm from among two or more compression algorithms for use on a plurality of delta encoded data values includes the following steps. The maximum number of data values eliminated by each of the two or more compression algorithms is computed. For the plurality of delta encoded data values to be compressed, the minimum size of the plurality of delta encoded data values before compression thereof is computed. A delete-safe threshold value is computed based on the minimum size of the plurality of delta encoded data values. Then, the compression algorithm is selected from the two or more compression algorithms that achieves the delete-safe threshold value.

Method and apparatus for selecting an optimal delete-safe compression method on list of delta encoded integers

A method, information processing system, and computer program storage product to compress sorted values. At least a first prefix and a second prefix in a plurality of prefixes are compared. Each prefix comprises at least a portion of a plurality of sorted values. A respective prefix comprises a set of consecutive characters including at least a first character of a respective sorted value. The respective sorted value further comprising a respective suffix comprising consecutive characters of the respective sorted value that are after the respective prefix. At least a respective first character of the first prefix and a respective first character of the second prefix are determined to be substantially identical. The first prefix is merged with the second prefix into a single prefix comprising the first character. A set of suffixes associated with the first prefix is updated to reflect an association with the second prefix.

Compression of sorted value indexes using common prefixes

An index scan processor and method to perform concurrent scans of stored indexed data. An indexed data scanner performs a first scan of stored data by sequentially scanning through an index data structure and maintains, over a duration of the first scan, a first scan current scan location within the index data structure that indicates a currently accessed record within the stored data structure for the first scan. A scan manager accepts a request for a second scan of the stored data, determines, during the performing the first scan, a starting index scan location within the index data structure for the second scan that is sufficiently near the first scan current index scan location to cause records indicated by the starting location index to be stored within a buffer due to the performing the first scan, and starts the second scan at the starting location and proceeding to the second end key.

System and method for executing multiple concurrent index-driven table access operations

A computer implemented method, apparatus, and computer usable program code for detecting monotonicity. A determination is made as to whether a function that is assumed to be monotonic will remain monotonic after performing an operation in response to receiving the operation. The operation is allowed to be performed in response to a determination that the function will remain monotonic after performing the operation.

Detecting and tracking monotonicity for accelerating range and inequality queries

Disclosed is a data processing system implemented method, a data processing system and an article of manufacture for improving execution efficiency of a database workload to be executed against a database. The database includes database tables, and the database workload identifies at least one of the database tables. The data processing system includes an identification module for identifying candidate database tables being identifiable in the database workload, the identified candidate database tables being eligible for organization under a clustering schema, a selection module for selecting the identified candidate tables according to whether execution of the database workload is improved if the selected identified candidate table is organized according to the clustering scheme, and an organization module for organizing the clustering schema of the selected organized identified candidate tables prior to the database workload being execution against the database.

Method, system and article of manufacture for improving execution efficiency of a database workload

Techniques are disclosed for encoding a variable length structure such that it facilitates forward and reverse scans of a list of such structures as needed. While the techniques are applicable to a wide variety of applications, they are particularly well-suited for use with structures such as those found in compressed database indexes. For example, a computer-implemented method for processing one or more variable length data structures includes the following steps. Each variable length data structure is obtained. Each variable length structure comprises one or more data block. A variable length encoding process is applied to the one or more blocks of each variable length data structure which comprises setting a continuation data value in each block to a first value or a second value, wherein the setting of the continuation data values enables bi-directional scanning of each variable length structure.

Method and apparatus for encoding list of variable length structures to support bi-directional scans

Disclosed are a method, information processing system, and computer readable medium for managing table scan processes. The method includes monitoring a plurality of storage medium table scan processes. Each storage medium table scan process in the plurality of storage medium table scan processes is placed into a plurality of scan groups based on storage medium pages to be scanned by each of the storage medium table scan processes. Each storage medium table scan process in a scan group can share data within a storage medium page.

Increasing buffer locality during multiple table access operations

A computer implemented method, apparatus, and computer usable program code for deleting rows in a multidimensionally clustering database. A multidimensional clustering processor receives a request to delete a set of multidimensionally clustering data. The multidimensional clustering processor determines whether all multidimensionally clustering data is located in a block. Responsive to all multidimensionally clustered data qualifying for deletion, the multidimensional clustering processor deletes all data in the block to delete the set of multidimensionally clustered data.

Method for fast deletion of physically clustered data

Disclosed are a method, information processing system, and computer readable medium for scanning a storage medium table. The method includes retrieving location information associated with at least one other storage medium table scan. A storage medium table scan is started at a location within a storage medium table based on at least a location of the one other storage medium table scan. A weight is assigned to at least one storage medium block based on at least a current scanning location within the storage medium table relative to the location of the one other table scan. The method determines if a distance between the current scanning location and the location of the one other table scan is greater than a first given threshold. A current scanning operation is delayed, in response to the distance being greater than the given threshold, until the distance is below a second given threshold.

Speed selective table scan operation

Growing community of inventors

Round Rock, TX, United States of America

Timothy Ray Malkemus