Oracle International Corporation

Roger MacNicol

Amit Ganesh

Vineet Marwah

Jesse Kamp

Vikram Kapoor

Kam Shergill

Sachin Kulkarni

Allison Holloway

Shasank Kisan Chavan

Manosiz Bhattacharyya

Anindya Chandra Patthak

Michael J Gleeson

Joydip Kundu

Teck Hua Lee

Aurosish Mishra

Douglas Andrew Voss

Goutam Kulkarni

Qinqin Wang

Anthony Adams

Juan R Loaiza

Tirthankar Lahiri

Kothanda Umamageswaran

Krishnan Meiyyappan

Adrian Tsz Him Ng

Adrian Ng

Laura Liaoruo Wang

Sachin Kulkami

A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a 'compression unit' using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Storing compression units in relational tables

Techniques are provided for automatic verification of query results obtained from data formatted different from row-major format. Techniques are also provided for automatic triage of query results obtained from data formatted different from row-major format. In an embodiment, the query results may be obtained from querying data formatted in column-major format. In an embodiment, when the query results are determined to be incorrect, a cause of the error in the query results can be automatically identified.

Automatic verification and triage of query results

A method for accelerating queries using dynamically generated columnar data in a flash cache is provided. In an embodiment, a method comprises a storage device receiving a first request for data that is stored in the storage device in a base major format in one or more primary storage devices. The storage device comprises a cache. The base major format is any one of: a row-major format, a column-major format and a hybrid-columnar format. Based on first one or more criteria, it is determined whether to rewrite the data into rewritten data in a rewritten major format. In response to determining to rewrite the data into rewritten data in a rewritten major format, the storage device rewrites at least a portion of the data into particular rewritten data in the rewritten major format. The rewritten data is stored in the cache.

Method to accelerate queries using dynamically generated alternate data formats in flash cache

A method and apparatus for efficiently processing data in various formats in a single instruction multiple data ('SIMD') architecture is presented. Specifically, a method to unpack a fixed-width bit values in a bit stream to a fixed width byte stream in a SIMD architecture is presented. A method to unpack variable-length byte packed values in a byte stream in a SIMD architecture is presented. A method to decompress a run length encoded compressed bit-vector in a SIMD architecture is presented. A method to return the offset of each bit set to one in a bit-vector in a SIMD architecture is presented. A method to fetch bits from a bit-vector at specified offsets relative to a base in a SIMD architecture is presented. A method to compare values stored in two SIMD registers is presented.

Efficient hardware instructions for single instruction multiple data processors

Techniques are described herein for generating and using in-memory data structures to represent columns in data block sets. In an embodiment, a database management system (DBMS) receives a query for a target data set managed by the DBMS. The query may specify a predicate for a column of the target data set. The predicate may include a filtering value to be compared with row values of the column of the target data set. Prior to accessing data block sets storing the target data set from persistent storage, the DBMS identifies an in-memory summary that corresponds to a data block set, in an embodiment. The in-memory summary may include in-memory data structures, each representing a column stored in the data block set. The DBMS determines that a particular in-memory data structure exists in the in-memory summary that represents a portion of values of the column indicated in the predicate of the query. Based on the particular in-memory data structure, the DBMS determines whether or not the data block set can possibly contain the filtering value in the column of the target data set. Based on this determination, the DBMS skips or retrieves the data block set from the persistent storage as part of the query evaluation.

Reducing data I/O using in-memory data structures

A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

Query and exadata support for hybrid columnar compressed data

Loading values from a value vector into subregisters of a single instruction multiple data register

Efficient hardware instructions for processing bit vectors for single instruction multiple data processors

A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a 'parent' compression unit to which it belongs, and may have one or more 'child' compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format, the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

Structure of hierarchical compressed data structure for tabular data

A method and system that annotates a redo log to provide information concerning the execution of a procedure at a primary database. The annotations include entry and exit markers that indicate the beginning and the end of the execution along with any arguments passed to the procedure, and whether the execution of the procedure was successful. At the standby database, these markers are used to create a logical transaction associated with the procedure. The operations performed by the procedure are grouped into individual transactions, and these individual transactions are grouped as belonging to the logical transaction. If the execution of the procedure was successful at the primary database, then the individual transactions are discarded, and the logical transaction is applied by executing the procedure at the standby database. If the execution of the procedure failed at the primary database, then the individual transactions and the logical transaction are discarded.

Techniques for the log-based replication of high-level procedures

Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

DDL and DML support for hybrid columnar compressed tables

Growing community of inventors

Hummelstown, PA, United States of America

Roger MacNicol