International Business Machines Corporation

Ibm Canada Limited - Ibm Canada Limitee

Ian R Finlay

Calisto Paul Zuzarte

Adam J Storm

Christian Marcelo Garcia-Arellano

Garth Allen Dickie

Nenad Rijavec

Naresh K Chainani

Timothy James Trenary

Joan LaVerne Mitchell

Robin Grosman

Kiran Chinta

Gennady Yakub

Vijayshankar Raman

Guy Maring Lohman

Ronald Jason Barber

Terence Patrick Purcell

Thomas Abel Beavin

Timothy Ray Malkemus

David C Kalmuk

Liping Zhang

John Frederick Hornibrook

Robert Neugebauer

Daniel Costante Zilio

Nattavut Sutyanyong

Joan La Verne Mitchell

Abhinav Goyal

Douglas James Doole

Dmitry Letin

Piotr Mierzejewski

Glenn P Steffler

Mohammad Wasif Khan

Elliot N Linzer

Bruce Gilbert Lindsay

Robert William Lyle

Ephraim Feig

Jeffrey M Keller

David C Sharpe

Karen L Anderson

Daniele Costante Zilio

Gary Valentin

Miroslaw Adam Flasza

Uttam Setty

Tony Wen Hsun Lai

Davey S Thornton

Aran Donohue

Douglas J Doole

Bruce G Lindsay

Guy M Lohman

Daniel C Zilio

Calisto P Zuzarte

Embodiments are disclosed for a method. The method includes receiving a plurality of local deltas for a query execution against a corresponding plurality of data sources hosted by a corresponding plurality of distributed nodes of a dynamic distributed network. The method also includes generating a combined delta by combining the local deltas. Additionally, the method includes generating a determined delta result by performing additional processing on the combined delta. Further, the method includes providing the determined delta for one of the distributed nodes.

Generating a global delta in distributed databases

Embodiments are disclosed for a method. The method includes parsing a distributed database query into component queries corresponding to distributed nodes. The method also includes matching component queries to central cache to determine whether the component queries can be satisfied. This matching generates matched and unmatched component queries. The method further includes transforming the matched component query to a cache query and local delta query for a first node. Additionally, the method includes executing the cache query to generate a cache result. Also, the method includes providing the local delta query for the first node to generate a local delta. Further, the method includes providing the unmatched component query for a second node to generate a remote result. The method also includes generating a global delta based on the local delta. The method additionally includes generating a result by combining the cache result, local delta, and remote result.

Querying distributed databases

The method includes identifying at least one of a minimum value, a maximum value, and a Bloom filter value for a row of data in a metadata table, wherein the metadata table contains metadata corresponding to a row of data in a main table. The method includes adjusting at least one of an identified first minimum value to a second minimum value, an identified first maximum value to a second maximum value, and an identified first Bloom filter value to a second Bloom filter value.

Optimization of metadata via lossy compression

A database-management system evaluates a query that retrieves and transforms encoded symbols stored in a database. If the stored symbols assume a relatively small set of distinct values, the system initially performs the transformation on every value in the set. During execution of subsequent queries, rather than performing the transformation upon every stored symbol fetched from the database, the system merely returns the previously derived encoded transformation results that correspond to the decoded value of each fetched symbol. If the symbols stored in the database span a relatively large set of distinct values, the system does not initially perform the transformation upon every value in the set. Instead, the first time the system fetches a symbol that has a particular value, it saves that symbol's encoded transformation result and reuses that result the next time it fetches an encoded symbol with the same value.

Derived data dictionary for optimizing transformations of encoded data

A computer-implemented method for a partitioned bloom filter merge is provided. A non-limiting example of the computer-implemented method includes partitioning, by a processing device, a bloom filter into N equal size filter partitions. The method further includes distributing, by the processing device, each of the filter partitions to an associated node. The method further includes merging, by the processing device, the filter partitions in each of the associated nodes. The method further includes redistributing, by the processing device, the merged filter partitions to each of the N nodes. The method further includes joining, by the processing device, the merged filter partitions in each of the N nodes to assemble a complete merged bloom filter.

Partitioned bloom filter merge for massively parallel processing clustered data management

A method, system and computer program product includes receiving and parsing an SQL query, identifying at least one common sub expression, or sub-query, or combination thereof, used multiple times within the SQL query, constructing for the at least one common sub expression, or sub-query, or combination thereof, a query execution plan that maintains as part of an initial result set a bit vector for a fact table, storing a result bit vector and an indicator that tracks a last valid tuple processed to produce the result bit vector when a TEMP operation is indicated in the query execution plan, reassessing a TEMP result in other portions of the query execution plan, priming a list of tuples using the TEMP result, and retrieving respective columns for further processing in the query execution plan.

Reducing temp size over a base table

A method, computer system, and a computer program product for resource scaling is provided. The present invention may include receiving a request for resources. The present invention may include receiving a request for a plurality of resources from a virtual device. The present invention may then include estimating a resource allocation based on a predetermined level of service based on the received request. The present invention may also include estimating a benefit curve of a workload for a plurality of tiers of resources based on the estimated resource allocation. The present invention may further include estimating a performance cost of the workload for the plurality of tiers of resources based on the estimated benefit curve.

Database resource scaling

Page filtering in a database using a compression dictionary. A page of a database table is compressed, creating a compression dictionary. The compression dictionary includes entries with a byte sequence from the page and a compression symbol associated with the byte sequence. A part of the compressed page, the compression dictionary, and a page symbol list with compression symbols from the dictionary present in the part of the page, are received. A query having a predicate with a predicate value is received. A predicate symbol list, including symbols in the dictionary whose byte sequences at least partially match the predicate value, is generated. Based on the predicate symbol list and the page symbol list, it is determined that at least one symbol from the predicate symbol list is also present in the part of the page. The query is performed by evaluating the predicate on the part of the page.

Page filtering via compression dictionary filtering

Merging adjacent rows of a synopsis table so as to increase the effectiveness of the synopsis table regarding data skipping. Adjacent rows for merging are identified based on statistics regarding: (i) queries of the database; (ii) effectiveness of the synopsis table for data skipping; and (iii) usage of predicates in queries of the database. Once merged, the synopsis table is smaller, and more effective with respect to data skipping, while fewer computing resources (administrative, maintenance, memory, clock cycles, storage space, etc.) are needed to process the database queries.

Automatically revising synopsis table structure

Embodiments of the present invention provide systems and methods for data management. Synopsis tables have been found to be more effective for maintaining a high level of system performance while answering analytical queries. Synopsis tables, which contain MAX, MIN, and Bloom filter columns, may be modified by dropping ineffective data content within these columns and regenerating dropped data when beneficial. By automatically modifying data, database queries may be optimized.

Pruning of columns in synopsis tables

A method for optimizing query compilation by tuning memory across a database cluster is provided. The method may include receiving, by a global memory tuner, memory configuration information from a plurality of nodes within the database cluster. The method may also include determining a node within the plurality of nodes having a least available memory value relative to a remainder of nodes within the plurality of nodes based on the received memory configuration information. The method may then include calculating a globally tuned memory value based on a memory value associated with the determined node. The method may further include determining a memory distribution based on the calculated globally tuned memory value. The method may also include sending the determined memory distribution to the plurality of nodes.

Tuning memory across database clusters for distributed query stability

Growing community of inventors

Uxbridge, Canada

Ian R Finlay