Sap Se

Sybase, Inc.

Ianywhere Solutions, Inc.

Peter Bumbulis

Anil Kumar Goel

Jeffrey Pound

Nathan Auch

Scott MacLean

Thomas Willhalm

Anisoara Nica

John Smirnios

Ismail Oukid

Daniel Booss

Reza Sherkat

Wolfgang Lehner

Daniel James Farrar

Daniil Golod

Eric Garber

Ivan Schreter

Mihnea Andrei

Alexander Boehm

Werner Thesing

Ivan T Bowman

Frank Renkes

Mohammed Abouzour

Oliver Rebholz

Thomas Bodner

Abdelkader Sellami

Pedram Ghodsnia

Gunter Radestock

Hua Fan

Matthias Ringwald

Pooya Saadat-Panah

Nathan Michael Auch

Disclosed herein are system, method, and computer program product embodiments for performing operations on compressed index vectors in columnar in-memory database management systems. By utilizing SIMD processor instructions, database management systems may perform operations that compress and decompress bit vectors and evaluate predicates.

Database management system performing column operations using a set of SIMD processor instructions selected based on performance

A system includes a non-volatile random access memory storing a column store main fragment of a column of a database table, and a processing unit to read the column store main fragment from the non-volatile random access memory. A volatile random access memory storing a column store delta fragment of the column of the database table may also be included, in which the processing unit is to write to the column store delta fragment. According to some systems, the stored column store main fragment is byte-addressable, and is copied from the volatile random access memory to the non-volatile random access memory without using a filesystem cache.

Column store main fragments in non-volatile RAM and the column store main fragments are merged with delta fragments, wherein the column store main fragments are not allocated to volatile random access memory and initialized from disk

Cracking page-loadable columns for in-memory data management is described herein. An embodiment operates by accessing a column according to a received query, determining that the received query requires a non-critical data structure associated with the column, and rebuilding the non-critical data structure from data associated with the column.

Cracking page-loadable columns for in-memory data management

A system includes reception of a first write request from a client including a first key and a first one or more stream, payload pairs associated with the first key, copying of the first key and the first one or more stream, payload pairs to a first buffer of a volatile memory, storage of data of the first buffer in one or more blocks of a raw block non-volatile memory device, providing of the first buffer to a stream store server, reception of the first buffer at the stream store server, adding of the first key and the first one or more stream, payload pairs to a second buffer of the volatile memory, in key-order, storage of the data of the second buffer in a filesystem storage device, according to stream, and transmission of an indication of the durability of the key to the tail store server.

Storage of log-structured data

Some embodiments provide a non-transitory machine-readable medium that stores a program. The program receives a query for a set of records in a database system having values in a field of a table that fall within a range of values. The program also determines a number of bits used to represent the values in the field of the table. The program further determines a set of operations to perform on the values in the field of the table based on the determined number of bits. The program also performs the determined set of operations on the values in the field of the table in order to identify the set of records in the database.

Range searches for database systems

Disclosed herein are system, method, and computer program product embodiments for quorum-based replication of data records. In one embodiment, a read request for reading a record is received from a user node on a replica node of a cluster of replica nodes. The record is then determined not committed on the replica node. In response to the determination, an update message indicative of whether the number of replica nodes on which the record is durable exceeds a threshold is received on the replica node. In response to the number of replica nodes exceeds the threshold, a value of the record on the replica node is transmitted to the user node.

Quorum-based replication of data records

A system includes writing of log data in chunks over a first range of heap log sequence positions of a heap log stored in the non-volatile memory system, and writing of a heap anchor marker in a first anchor stream of a transaction log stored in the non-volatile memory system, at a first transaction log sequence position of the transaction log, wherein the first anchor stream is associated with a first stream of the transaction log, and the heap anchor marker identifies the first range of heap log sequence positions of the heap log.

Handling large writes to distributed logs

A method includes a primary storage unit receiving a first write request including a first key and a first value; persisting the first value in a first non-volatile memory in association with the first key; broadcasting the first write request and a first set of globally-durable keys to secondary storage units; receiving, from the secondary storage units, an acknowledgement of the first write request and a first set of locally-durable keys, each of the first sets of locally-durable keys including the first key; the primary storage unit receiving a second write request including a second key and a second value; persisting the second value in the first non-volatile memory in association with the second key; and broadcasting the second write request and a second set of globally-durable keys to the secondary storage units, the second set of locally-durable keys including the first key. A system is also disclosed.

Replication of log-structured data

A data recovery system and method are disclosed. Primary data is stored a database in byte-addressable NVRAM, where the database includes one or more persistent tables of data in a byte-addressable, RAM format, and a persistent memory allocator that maps persistent memory pointers of the persistent memory to virtual memory pointers of a virtual memory associated with the database. Secondary data is stored in volatile DRAM. A failure recovery includes recovering the persistent memory allocator, mapping the persistent memory to the virtual memory to recover primary data using their persistent memory pointers, translating the persistent memory pointers to virtual memory pointers, undoing changes to the primary data made by unfinished transactions of the query execution at the time of failure of one of the one or more queries, and reconstructing the secondary data from the primary data.

Hybrid SCM-DRAM transactional storage engine for fast data recovery

A method and system, the system including a plurality of machines each having a processor and a main memory component; a shared distributed storage facility storing a set of data and accessible by the plurality of machines over a communication network; a controller to select, in response to a state of a query execution plan comprising a plurality of executable jobs for the set of data, which one of a set of scheduling algorithms to execute; an execution engine to execute the selected scheduling algorithm to determine, for each job in the plurality of jobs, which server to schedule to execute the respective job; and providing an indication of the scheduling of the servers determined to be schedules for the execution of the jobs.

Distributed resource-aware task scheduling with replicated data placement in parallel database clusters

A plurality of reserve and commit log operations are initiated in a database system. Thereafter, at least a portion of the database operations are logged in a log such that transient data structures are kept in-memory of the database system and persistent data structures are kept in byte-addressable memory. Next, each of one or more clients concurrently accessing the log are registered to enable such clients to access the log.

Database logging using storage class memory

Disclosed herein are system, method, and computer program product embodiments for distributed transaction processing in massively parallel processing (MPP) databases. An embodiment operates by receiving a transaction including a read timestamp corresponding to a version of data of a database associated with the query. It is determined whether a locally stored portion of the data of the database has a local timestamp that is equivalent to or after the read timestamp. A lock may be requested, and if received, the data is updated.

Distributed transaction processing in MPP databases

Disclosed herein are system, method, and computer program product embodiments for accessing a distributed shared log that supports a total order over log writes and multiple logical log streams. An embodiment operates by selecting a range of log entries for a logical log stream from the distributed shared log and mapping the selected range of log entries to associated storage units in a plurality of storage units. The selected range of log entries are stored at the associated storage units according to metadata that indexes the selected range of log entries by logical log stream. The selected range of log entries are then requested from the associated storage units based on the mapping. The selected range of log entries for the logical log stream are then returned from the associated storage units based on a scan of the metadata. The received log entries are then reassembled.

Distributed shared log for modern storage servers

A system includes determination of a plurality of secondary data structures of a database to be rebuilt, determination, for each of the plurality of secondary data structures, of a current ranking based on a pre-crash workload, a crash-time workload, the post-crash workload, and a rebuild time of the secondary data structure, determination to rebuild one of the plurality of secondary data structures based on the determined rankings, and rebuilding of the one of the plurality of secondary data structures in a dynamic random access memory based on primary data of a database stored in non-volatile random access memory.

Adaptive recovery for SCM-enabled databases

A method and system for transforming a serial schedule of transactions into a parallel schedule of transaction is disclosed. In one example, a computer system stores a list of data transactions in a transaction log. The computer system then reads a respective data transaction from the transaction log. The computer system determines whether the respective data transaction is dependent on any other currently pending data transaction. In accordance with a determination that the respective data transaction is not dependent on any other currently pending data transaction, the computer system applies the data changes to a reconstructed data set. In accordance with a determination that the respective data transaction is dependent on a currently pending second data transaction, the computer system delays commitment of the respective data transaction until the second data transaction has been applied to the reconstructed data set.

Growing community of inventors

Cambridge, Canada

Peter Bumbulis