Oracle International Corporation

Teradata Us, Inc.

Krishnan Meiyyappan

Kapil Laxmikant Surlaker

Joseph I Chamdani

Shailendra Mishra

Kothanda Umamageswaran

Jeremy L Branscome

Neerja Bhatt

Namit Jain

Ravi Krishnamurthy

Adrian Tsz Him Ng

Michael Paul Corwin

James Shau

Semen Ustimenko

Liuxi Yang

Hung Viet Tran

Bhagat V Nainani

Ramkumar Venkatesan

Wei Zhang

Wei Wang

Tirthankar Lahiri

James William Stamos

Alan Robert Downing

Dmitry Mikhailovich Potapov

Alexander Tsukerman

Vijay Sridharan

Adrian Ng

Alan Lee Beck

Amit Ganesh

Juan R Loaiza

Shasank Kisan Chavan

Yunrui Li

Sreenivas Gollapudi

Jesse Kamp

Lakshminarayanan Luxi Chidambaran

Jia Shi

Alex Tsukerman

Sanjay Kaluskar

Roger MacNicol

Nancy Reiko Ikeda

Zuoyu Tao

Chao Liang

Viral Shah

Xia Hua

Umesh Panchaksharaiah

Nilesh Choudhury

Allen Brumm

Roger Dermot Macnicol

James Stenoish

Maria Colgan

Robert K Abbott

Daniel McClary

Ashwinder Singh Ahluwalia

Yiran Qin

Yinian Qi

Kai Zhang

Salini Selvaraj Kowsalya

Somayeh Sardashti

Robert Hou

Yiliang Jin

Ke Hu

Xin Zeng

Laura Liaoruo Wang

Devang Mundhra

Ravindran Krishnamurthy

Zheren Zhang

Graham Ivey

Roger D Macnicol

Region summaries of database data are stored in persistent memory of a storage cell. Because the region summaries are stored in persistent memory, when a storage cell is powered off and data in volatile memory is not retained, region summaries are nevertheless preserved in persistent memory. When the storage cell comes online, the region summaries already exist and may be used without the delay attendant to regenerating the region summaries stored in volatile memory.

Storing derived summaries on persistent memory of a storage device

A storage system communicatively coupled to a database management system (DBMS performs storage-side scanning of data sources that are not stored in native database storage format of the DBMS. Data sources for external tables are accessible in a storage system referred to as a distributed data access system (DDAS), e.g. a Hadoop Distributed File System. To execute a query that references an external table, a DBMS first generates an execution plan. The DDAS supplies the DBMS with information that specifies each portion of the data source, and specifies which data node to use to access the portion. The DBMS sends a request for each portion to the respective data node, requesting that the data node generate rows from data in the portion. The request may specify scanning criteria, specifying one or more columns to project and/or filter on, and code modules for the data node to execute to generate records.

Storage-side scanning on non-natively formatted data

A shared storage architecture persistently stores database files in non-volatile random access memories (NVRAMs) of computing nodes of a multi-node DBMS. The computing nodes of the multi-node DBMS not only collectively store database data on NVRAMs of the computing nodes, but also host database server instances that process queries in parallel, host database sessions and database processes, and together manage access to a database stored on the NVRAMs of the computing nodes. To perform a data block read operation from persistent storage, a data block may be transferred directly over a network between NVRAM of a computing node that persistently stores the data block to a database buffer in non-volatile RAM of another computing node that requests the data block. The transfer is accomplished using remote direct memory access ('RDMA).

Database with NVDIMM as persistent storage

Embodiments of the present invention provide a hardware accelerator that assists a host database system in processing its queries. The hardware accelerator comprises special purpose processing elements that are capable of receiving database query/operation tasks in the form of machine code database instructions, execute them in hardware without software, and return the query/operation result back to the host system.

Methods and systems for hardware acceleration of database operations and queries for a versioned database based on multiple hardware accelerators

Techniques are provided for storing in in-memory unit (IMU) in a lower-storage tier and copying the IMU to DRAM when needed for query processing. Techniques are also provided for copying IMUs to lower tiers of storage when evicted from the cache of higher tiers of storage. Techniques are provided for implementing functionality of IMUs within a storage system, to enable database servers to push tasks, such as filtering, to the storage system where the storage system may access IMUs within its own memory to perform the tasks. Metadata associated with a set of data may be used to indicate whether an IMU for the data should be created by the database server machine or within the storage system.

Columnar caching in tiered storage

According to embodiments, a derived cache that is derived from a first instance of particular data is used to speed up queries and other operations over a second instance of the particular data. Traditionally, a DBMS generates and uses derived cache data only for the database data from which the derived data was derived. However, according to embodiments, derived cache data associated with a first instance of database data is relocated to the location of a second, newly created, instance of the database data. Since the derived cache data is derived from an identical copy of the database data, the cache data derived for the first instance can successfully be used to speed up applications running over the second instance of the database data.

Relocating derived cache during data rebalance to maintain application performance

Techniques for optimizing a query with an extrema function are provided. In main memory, a data summary is maintained for a plurality of extents stored by at least one storage server. The data summary includes an extent minimum value and an extent maximum value for one or more columns. A storage server request is received, from a database server, based on a query with an extrema function applied to a particular column of a particular table. The data summaries for a set of relevant extents are processed by maintaining at least one global extrema value corresponding to the extrema function and, for each relevant extent of the set of relevant extents, determining whether to scan records of the relevant extent based on at least one of the global extrema value and an extent summary value of the data summary of the relevant extent.

Optimizing a query with extrema function using in-memory data summaries on the storage server

Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query. Based on the particular in-memory data structure, the DBMS determines whether or not the data block set can possibly contain the filtering value in the column of the target data set. Based on this determination, the DBMS skips or retrieves the data block set from the persistent storage as part of the query evaluation.

Reducing data I/O using in-memory data structures

Techniques are described herein for supporting multiple versions of a database server within a database machine comprising a separate database layer and storage layer. In an embodiment, the database layer includes compute nodes each hosting one or more instances of a database server. The storage layer includes storage nodes each hosting one or more instances of a storage server, also referred to herein as a 'cell server.' In general, the database servers may receive data requests, such as SQL queries, from client applications and service the requests in coordination with the cell servers of the storage layer.

Systems and methods for supporting multiple database server versions on a database machine

Embodiments of the present invention provide hardware-friendly indexing of databases. In particular, forward and reverse indexing are utilized to allow for easy traversal of primary key to foreign key relationships. A novel structure known as a hit list also allows for easy scanning of various indexes in hardware. Group indexing is provided for flexible support of complex group key definition, such as for date range indexing and text indexing. A Replicated Reordered Column (RRC) may also be added to the group index to convert random I/O pattern into sequential I/O of only needed column elements.

Accessing data in a column store database based on hardware compatible indexing and replicated reordered columns

Embodiments of the present invention provide one or more hardware-friendly data structures that enable efficient hardware acceleration of database operations. In particular, the present invention employs a column-store format for the database. In the database, column-groups are stored with implicit row ids (RIDs) and a RID-to-primary key column having both column-store and row-store benefits via column hopping and a heap structure for adding new data. Fixed-width column compression allow for easy hardware database processing directly on the compressed data. A global database virtual address space is utilized that allows for arithmetic derivation of any physical address of the data regardless of its location. A word compression dictionary with token compare and sort index is also provided to allow for efficient hardware-based searching of text. A tuple reconstruction process is provided as well that allows hardware to reconstruct a row by stitching together data from multiple column groups.

Accessing data in column store database based on hardware compatible data structures

Embodiments of the present invention provide fine grain concurrency control for transactions in the presence of database updates. During operations, each transaction is assigned a snapshot version number or SVN. A SVN refers to a historical snapshot of the database that can be created periodically or on demand. Transactions are thus tied to a particular SVN, such as, when the transaction was created. Queries belonging to the transactions can access data that is consistent as of a point in time, for example, corresponding to the latest SVN when the transaction was created. At various times, data from the database stored in a memory can be updated using the snapshot data corresponding to a SVN. When a transaction is committed, a snapshot of the database with a new SVN is created based on the data modified by the transaction and the snapshot is synchronized to the memory. When a transaction query requires data from a version of the database corresponding to a SVN, the data in the memory may be synchronized with the snapshot data corresponding to that SVN.

Methods and systems for real-time continuous updates

An approach for reducing transport of messages between nodes of a multi-node system is presented wherein a message queue is associated with a queue service, and based on which node the message queue resides, one of the nodes is registered as hosting the associated queue service. In response to a client attempting to connect and requesting a particular queue service, the client is caused to connect to the node on which the queue service resides.

Event locality using queue services

Growing community of inventors

Fremont, CA, United States of America

Krishnan Meiyyappan