International Business Machines Corporation

Kevin James Kathmann

David Glenn Carlson

Steven Simonson

Alexei Pytel

Thomas Wasik

Randy Lynn Egan

Steven Virgil Hoeschen

Robert J Bestgen

Bruce Talbott

Thuan Quang Bui

John Matthew Santosuosso

Gary Ross Ricard

Shantan Kethireddy

Kurt Walter Pinnow

Gary Stuart Puchkoff

Ulrich Thiemann

Andrew Peter Passe

Wei Hu

David Lee Hermsmeier

Lance Christopher Amundsen

Abdo Esmail Abdo

Thomas Owen McKinley

Lawrence Kenneth Nash

Tariq Mahmood Choudhry

A method, a computer program product, and a computer system for generating a set of scripts that perform processing equivalent to an original script and achieve maximum parallel executions. A computer receives an original script including annotations defined by a user, builds a graph comprising nodes and representing execution steps in the original script, associates one or more attributes to the nodes, and determines dependencies between the nodes, based on accessing data sets or files by the execution steps. The computer modifies the dependencies between the nodes, using instructions contained in the annotations. The computer generates a set of scripts that perform processing equivalent to the original script and achieve maximum parallel executions, based on the nodes of the graph and the dependencies. The computer generates a script that defines dependencies between the scripts.

Batch script decomposer

A hybrid messaging model including a method that sends a first request message from a control process executing on a computer to a plurality of subordinate processes. The first request message directs the subordinate processes to enter a first state. An expected state is set equal to the first state in response to sending the first request message. A status message, including the expected state, is periodically broadcast from the control process to the plurality of subordinate processes. At least one confirmation message is received from each of the subordinate processes confirming that the subordinate process is in the expected state. Each of the confirmation messages is responsive to either the first request message or to the status message. A second request message is sent from the control process to the plurality of subordinate processes in response to receiving at least one confirmation message from each of the subordinate processes.

Hybrid request/response and polling messaging model

A node container is constructed with multiple binary tree views. A processing device creates a primary view binary tree within the node container based on a primary key. A set of records that match fields of the primary key is chained to the primary view. The processing device also creates one or more alternate views within the node container to allow access to the same set of records via one or more alternate keys, which are subsets of the fields of the primary key. The chain of records is extracted from the node container when matched to a subsequent query search.

Dynamic output selection using highly optimized data structures

Nodes in a query execution data structure have monitor and dump methods defined. The monitor method enables the collection of information from the node. The dump method outputs the monitored information from each node. A monitor and debug mechanism of the preferred embodiments includes a graphical user interface that allows a user to graphically examine a query execution tree, to enable monitoring of the nodes on a node-by-node basis, and to view information dumped from the query execution data structure as the query is executed or after the query is executed. The result is a powerful tool that allows efficiently monitoring and debugging a query implemented in an object oriented query execution data structure.

Monitoring and debugging query execution objects

A method is provided for constructing a cache for storing results of previously evaluated queries in a binary tree based on a cache key. The cache is searched, by a processing device, for a node representing a set of previously evaluated queries that match a given record using an instance of the cache key. In response to finding a representative node that matches the instance of the cache key, the method performs processing associated with the set of previously evaluated queries. In response to finding no representative nodes, the method creates a new node using the instance of the cache key. The method of the invention then updates the cache by classifying the compatibility of newly registered queries with the cache key and evaluating whether compatible newly registered queries match the record.

Efficient selection of queries matching a record using a cache

Transparent file system migration including a method that identifies a source physical storage location in a file system that supports simultaneous access by multiple independent processes. A target physical storage location in the file system is identified and its contents are migrated to the target physical storage location. The migrating includes disallowing new allocations to the source physical storage location and copying contents from the source physical storage location to the target physical storage location. A write request to write data to the source physical storage location is received prior to or during the migrating. The write request is serviced contemporaneously with the migrating. Read and write requests directed to the source physical storage location are redirected to the target physical storage location. The redirecting is responsive to the migrating completing. The requests are from the multiple independent processes and include both logical and physical input/outputs.

Transparent file system migration to a new physical location

A stream data node receives real-time streamed data from one or more input devices, dynamically filters the streamed data to reduce the streamed data, and delivers the reduced data when requested. By providing real-time filtering of the data, the amount of data that must be stored in a database may be substantially reduced. The stream data node can perform aggregation functions, group functions, and select functions, thereby also significantly reducing the amount of data that must be stored in a database. The stream data node may also be part of a query execution data structure, where it delivers its data when requested by another node in the query execution data structure.

Real-time mining and reduction of streamed data

A method, apparatus and program product dynamically activate and/or deactivate buffers during execution of a database query. The dynamic deactivation and activation is based on the monitoring of information associated with the consumption of data by a plurality of threads during execution of an access plan for the database query. By doing so, the selection of a less optimal buffer position during optimization may be replaced with a more optimal buffer position during runtime, often resulting in improved query performance and parallelism.

Database query processing with dynamically activated buffers

Methods, systems and articles of manufacture of processing a multi-state attribute field query. One embodiment provides a method of processing a multi-state attribute field query, comprising instantiating a plurality of live objects; performing, by each live object, an attribute operation, wherein at least one attribute operation is a comparison operation associated with attributes of the query. The plurality of live objects access a global status storage area only when an operand of the comparison operation is null or error. In any case, execution control is passed from each live object to an adjacent live object.

Method for efficient processing of multi-state attributes

Previously-optimized database queries are stored in memory. When a new query needs to be optimized, the previously-optimized queries are examined to determine whether the new query has been previously optimized. If the new query has not been previously optimized, the previously-optimized queries are examined to determine whether any previously-optimized queries differ only in data type of one or more operands when compared to the new query. If a previously-optimized query that differs only in data type is located, the previously-optimized query is refreshed to reflect the different data type without the need of optimizing the new query from scratch. Portions of previously-optimized queries may thus be re-used even when a previously-optimized query is not identical to a new query to be optimized. As a result, the performance of query optimization in a database system is increased.

Growing community of inventors

Rochester, MN, United States of America

Kevin James Kathmann