International Business Machines Corporation

Future Wei Technologies, Inc.

Ebay, Incorporated

Huawei Technologies Co., Limited

Guogen Zhang

Yao-Ching Stephen Chen

Yun Yu Wang

Fen-Ling Lin

Mengchu Cai

Qingqing Zhou

Ruiping Li

James Zu-Chia Teng

Qinghua Zou

Jason Alexander Cu

Brian Thinh-Vinh Tran

Ning Wang

Li-Mey P Lee

Yao Ching Stephen Chen

Irene Ching-Hua Liu

Hubertus Franke

Knut Stolze

Balakrishna Raghavendra Iyer

Hong Min

Namik Hrle

Terence Patrick Purcell

Oliver Koeth

Oliver Draese

Yang Sun

Jerry Mukai

Le Cai

Lee-Chin Hsu Liu

Claus Kempfert

Allan Benjamin Lebovitz

Sami Ben-Romdhane

Yumi Kimura Tsuji

Yu Dong

Yue Huang

Ding-Wei Chieh

Robert S Muse

Ronald Morton Smith, Sr

You-Chin Gene Fuh

Cheng Zhu

Michael F Cowlishaw

Kalpana Shyam

Christopher John Crone

Yonghua Ding

Brian Shaun Vickery

Jeffrey Ralph Ruggles

Jason Yang Sun

Fung Lee

Stephen Yao Ching Chen

Patrick Michael Malone

Demai Ni

Desai Paramesh Sampatrai

Huaizhi Li

Bai Yang

Hsuiying Yen Cheng

Jungsang Ahn

Huong T Tran

Kun Ren

Jung-Hsin Hu

Glenn M Yuki

System and method embodiments are provided for using different storage formats for a primary database and its replicas in a database managed replication (DMR) system. As such, the advantages of both formats can be combined with suitable design complexity and implementation. In an embodiment, data is arranged in a sequence of rows and stored in a first storage format at the primary database. The data arranged in the sequence of rows is also stored in a second storage format at the replica database. The sequence of rows is determined according to the first storage format or the second storage format. The first storage format is a row store (RS) and the second storage format is a column store (CS), or vice versa. In an embodiment, the sequence of rows is determined to improve compression efficiency at the CS.

Apparatus and method for managing storage of a primary database and a replica database

A global transaction system receives a transaction request for a plurality of database services of microservices. The global transaction system receives a plurality of local commit decisions of local commit requests from local transaction managers of the database services of the microservices. The local commit request corresponds to the transaction request for each database. The global transaction system generates a physical commit request to each of the local transaction managers based on the local commit decisions and a global commit decision. Each local transaction manager submits the physical commit request to each database server of the database services corresponding to the transaction request.

Global distributed transactions across microservices

An apparatus and method are provided for utilizing different data storage types to store primary and replicated database directories. Included is a first data storage of a first data storage type including a direct-access storage type. The first data storage is configured to store a primary database directory. Also included is a second data storage of a second data storage type including a share type. The second data storage is configured to store a replicated database directory that replicates at least a portion of the primary database directory.

Apparatus and method for utilizing different data storage types to store primary and replicated database directories

A replicated key-value store is implemented using a 'last-write-wins' consensus protocol. To improve throughput and latency in cross-data-center configurations, a system deploys a cross-cluster, learner-only member to a cluster of nodes (e.g., a data center). The cross-cluster, learner-only member submits key-values received at local leader members to remote clusters. Conflicts between the key-values and initial values at the remote clusters are resolved using a 'last-write-wins' consensus protocol.

Key-value replication with consensus protocol

A computer-implemented method and system are provided, including executing an application programming interface (API) in a network switch to define at least one of one or more database functions, performing, using one or more processors, the one or more database functions on at least a portion of data contained in a data message received at the switch, to generate result data, and routing the result data to one or more destination nodes. A database function-defined network switch includes a network switch and one or more processors to perform a pre-defined database function on query data contained in data messages received at the switch, to produce result data, wherein the pre-defined database function is performed on the query data in a first mode of operation to a state of full completion, generating complete result data and no skipped query data, or to a state of partial completion, generating partially completed result data and skipped query data.

Database functions-defined network switch and database system

System and method embodiments are provided for multi-version support in indexes in a database. The embodiments enable substantially optimized multi-version support in index and avoid backfill of commit log sequence number (LSN) for a transaction identifier (TxID). In an embodiment, a method in a data processing system for managing a database includes determining with the data processing system whether a record is deleted according to a delete indicator in an index leaf page record corresponding to the record; and determining with the data processing system, when the record is not deleted, whether the record is visible according to a new record indicator in the index leaf page record and according to a comparison of a system commit TxID at the transaction start with a record commit TxID obtained from the index leaf page record.

Systems and methods to optimize multi-version support in indexes

Data messages having different priorities may be stored in different communication buffers of a network node. The data messages may then be forwarded from the communication buffers to working buffers as space becomes available in the working buffers. After being forwarded to the working buffers, the data messages may be available to be processed by upper-layer operations of the network node. Priorities may be assigned to the data messages based on a priority level of a query associated with the data messages, a priority level of an upper-layer operation assigned to process the data messages, or combinations thereof.

Query plan and operation-aware communication buffer management

A method includes dividing a dataset into partitions by hashing a specified key, selecting a set of distributed file system nodes as a primary node group for storage of the partitions, and causing a primary copy of the partitions to be stored on the primary node group by a distributed storage system file server such that the location of each partition is known by hashing of the specified key.

Data placement control for distributed computing environment

A method includes receiving, by a database system, a query statement and forming a runtime plan tree in accordance with the query statement. The method also includes traversing the runtime plan tree including determining whether a function node of the runtime plan tree is qualified for just-in-time (JIT) compilation. Additionally, the method includes, upon determining that the function node is a qualified for JIT compilation producing a string key in accordance with a function of the function node and determining whether a compiled object corresponding to the string key is stored in a compiled object cache.

System and method for database query

The present technology relates to managing data caching in processing nodes of a massively parallel processing (MPP) database system. A directory is maintained that includes a list and a storage location of the data pages in the MPP database system. Memory usage is monitored in processing nodes by exchanging memory usage information with each other. Each of the processing nodes manages a list and a corresponding amount of available memory in each of the processing nodes based on the memory usage information. Data pages are read from a memory of the processing nodes in response to receiving a request to fetch the data pages, and a remote memory manager is queried for available memory in each of the processing nodes in response to receiving the request. The data pages are distributed to the memory of the processing nodes having sufficient space available for storage during data processing.

System and method for data caching in processing nodes of a massively parallel processing (MPP) database system

In one embodiment, a method of performing point-in-time recovery (PITR) in a massively parallel processing (MPP) database includes receiving, by a data node from a coordinator, a PITR recovery request and reading a log record of the MPP database. The method also includes determining a type of the log record and updating a transaction table when the type of the log record is an abort transaction or a commit transaction.

System and method for massively parallel processing database

A method, apparatus, and article of manufacture for optimizing a query in a computer system. Grouping operations are optimized during execution of the query in the computer system by: (1) translating the grouping operations into a plurality of levels, wherein each of the levels is comprised of one or more grouping sets with the same number of grouping expressions; (2) deriving the grouping sets on a level-by-level basis, wherein the grouping sets in a base level are obtained from the database and the grouping sets in a next one of the levels are derived by selecting as an input a smallest one of the grouping sets in a previous one of the levels with which it has a derivation relationship; and (3) combining the derived grouping sets into an output for the query.

Dynamic selection of optimal grouping sequence at runtime for grouping sets, rollup and cube operations in SQL query processing

System and method embodiments are provided for adaptive vector size selection for vectorized query execution. The adaptive vector size selection is implemented in two stages. In a query planning stage, a suitable vector size is estimated for a query by a query planner. The planning stage includes analyzing a query plan tree, segmenting the tree into different segments, and assigning to the query execution plan an initial vector size to each segment. In a subsequent query execution stage, an execution engine monitors hardware performance indicators, and adjusts the vector size according to the monitored hardware performance indicators. Adjusting the vector size includes trying different vector sizes and observing related processor counters to increase or decrease the vector size, wherein the vector size is increased to improve hardware performance according to the processor counters, and wherein the vector size is decreased when the processor counters indicate a decrease in hardware performance.

System and method for adaptive vector size selection for vectorized query execution

Growing community of inventors

San Jose, CA, United States of America

Guogen Zhang