International Business Machines Corporation

Sandeep Tata

Wei Tan

Guy Maring Lohman

Vijayshankar Raman

Ronald Jason Barber

Mir Hamid Pirahesh

Richard Sefton Sidle

Eugene Jon Shekita

Naresh K Chainani

Lin Qiao

Jun Rao

Augmenting data files in a repository of an append-only file system includes maintaining a companion metadata file for each corresponding data file in a map-reduce system using the append-only file system. Each companion metadata file tracks a logical end-of-file (EOF) for each data file. Global versioning of each companion metadata is maintained. A map-reduce append job is performed for a set of data files using a current global version number for the companion metadata file. The map-reduce job including multiple append tasks. For each successful append job, a logical EOF for each appended file is incremented to a new physical EOF. For each failed append task of the append job, a logical EOF is maintained for each failed append task by not incrementing the logical EOF for each failed append task.

Atomic incremental load for map-reduce systems on append-only file systems

There are provided a system and a computer program product for operating multi-node data stores. The system stores a data table in a first computing node and stores an index table in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. The system performs operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node. The operations optimizes between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

Differentiated secondary index maintenance in log structured NoSQL data stores

There are provided a method for operating multi-node data stores. The method performs storing a data table in a first computing node and stores an index table in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. The method performs operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node. The operations optimizes between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

[object Object]

Augmenting data files in a repository of an append-only file system comprises maintaining metadata corresponding to each data file for tracking a logical end-of-file (EOF) for each data file for appending. A global versioning mechanism for the metadata allows selecting the current version of the metadata to read for performing an append job for a set of data files. Each append job comprises multiple append tasks. For each successful append job, the global versioning mechanism increments a valid metadata version to use for each data file appended. Said valid metadata version indicates the logical EOF corresponding to a new physical EOF for each of the data files appended.

There is provided a method for operating multi-node data stores. A data table is stored in a first computing node and an index table is stored in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. Operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node are performed. The operations optimize between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

There are provided a system and a computer program product for operating multi-node data stores. The system stores a data table in a first computing node and stores an index table in a second computing node. The index table provides keys used for accessing data in the first computing node and other multi-node data stores. The system performs operations that update or read the data table accessed from the first computing node and the index table accessed from the second computing node. The operations optimize between latency in updating or reading the data table and the index table and data consistency maintained between data entries in the data table and data entries pointed by indices in the index table.

Embodiments of the invention relate to processing queries that utilize fact and/or dimension tables. In one aspect, a pre-join filtering phase precedes a star join. The necessary conditions for the pre-join filtering are considered for a given SQL query, including an estimated size of the hash table exceeding a threshold and presence of a local predicate either on the fact table or one or more dimension tables that is not a large dimension table. Once the necessary conditions are satisfied, the execution of the query exploits the pre-join filtering to build a pre-join output filter from columns of a reduced fact table that joins with each large dimension table. Thereafter, all the dimension tables and the fact table are joined in a star join while exploiting each pre-join filter.

Efficient join with one or more large dimension tables

A method for updating a scalable row-store, including: receiving an update to a key within a range of keys in a database table, wherein the database table is distributed across nodes in a cluster of computing devices; and replicating the update over a group of the nodes using a consensus-based replication algorithm, wherein the replication algorithm includes completing the update in response to receiving acknowledgement messages from a majority of the nodes in the group indicating that the majority has received notification of the update.

Scalable row-store with consensus-based replication

A common interface to manage heterogeneous databases and develop enterprise class applications is provided. In particular, it is shown that a client-based system and method can: (a) provide a uniform interface for the DBA or the application developer to use across all the database deployments, (b) provide flexibility in the number and kinds of scenarios it can be used, and finally (c) reduce the total cost of ownership for the enterprise.

Growing community of inventors

San Jose, CA, United States of America

Sandeep Tata