Oracle International Corporation

Kam Shergill

Vineet Marwah

Jesse Kamp

Amit Ganesh

Vikram Kapoor

Roger MacNicol

Sachin Kulkarni

Manosiz Bhattacharyya

Juan R Loaiza

Tirthankar Lahiri

Garret Frederick Swart

Michael J Gleeson

Krishna Kunchithapadam

Neil J S MacNaughton

Siddharth Teotia

Srinivas Vemuri

Sujatha Muthulingam

Ankita Kumar

Roger Dermot Macnicol

Bharath Aleti

Jesse Kemp

Dheerai Pandey

Sachin Kulkami

Himanshu Chatterjee

A method and apparatus for compression units in an index block is provided. A particular leaf block of a database index for a table is generated. The particular leaf block corresponds to a particular range of key values. A compression scheme is determined for a plurality of index entries with key values within the particular range. A compression unit that includes a plurality of compressed index entries is generated by applying the compression scheme to the plurality of index entries. Compression metadata describing the compression scheme is generated. The compression unit and the compression metadata is stored in the particular leaf block. In response to a request, at least a portion of the compression unit is decompressed to access one or more index entries.

Compression units in an index block

A database server stores compressed units in data blocks of a database. A table (or data from a plurality of rows thereof) is first compressed into a 'compression unit' using any of a wide variety of compression techniques. The compression unit is then stored in one or more data block rows across one or more data blocks. As a result, a single data block row may comprise compressed data for a plurality of table rows, as encoded within the compression unit. Storage of compression units in data blocks maintains compatibility with existing data block-based databases, thus allowing the use of compression units in preexisting databases without modification to the underlying format of the database. The compression units may, for example, co-exist with uncompressed tables. Various techniques allow a database server to optimize access to data in the compression unit, so that the compression is virtually transparent to the user.

Storing compression units in relational tables

A hashing scheme includes a cache-friendly, latchless, non-blocking dynamically resizable hash index with constant-time lookup operations that is also amenable to fast lookups via remote memory access. Specifically, the hashing scheme provides each of the following features: latchless reads, fine grained lightweight locks for writers, non-blocking dynamic resizability, cache-friendly access, constant-time lookup operations, amenable to remote memory access via RDMA protocol through one sided read operations, as well as non-RDMA access.

Latchless, non-blocking dynamically resizable segmented hash index

Techniques are provided for enabling a requesting entity to retrieve data that is managed by a database server instance from the volatile memory of a server machine that is executing the database server instance. The techniques allow the requesting entity to retrieve the data from the volatile memory of the host server machine without involving the database server instance in the retrieval operation. Because the retrieval does not involve the database server instance, the retrieval may succeed even when the database server instance has stalled or become unresponsive. In addition, direct retrieval of data using the techniques described herein will often be faster and more efficient than retrieval of the same information through conventional interaction with the database server instance.

Method for efficient primary key based queries using atomic RDMA reads on cache friendly in-memory hash index

Techniques for compressing leaf blocks of an index are provided. The decision whether (and/or how much) to compress leaf blocks of an index is performed on a per block basis. Thus, for example, some leaf blocks of the index may be compressed while other leaf blocks of the index may be uncompressed. As another example, different leaf blocks of an index may be compressed a different amount. In one technique, a set of index entries is identified for storage in a leaf block. Multiple estimates of potential space savings are calculated for compressing the set of index entries, one estimate for each different variation of a compression technique. The compression technique variation that is associated with the most space savings is selected and applied to the set of index entries. Thus, some techniques prevent negative compression altogether. Additionally, users require no knowledge of the underlying data to perform optimal compression.

Adaptive index leaf block compression

A method and apparatus is provided for optimizing queries received by a database system that relies on an intelligent data storage server to manage storage for the database system. Storing compression units in hybrid columnar format, the storage manager evaluates simple predicates and only returns data blocks containing rows that satisfy those predicates. The returned data blocks are not necessarily stored persistently on disk. That is, the storage manager is not limited to returning disc block images. The hybrid columnar format enables optimizations that provide better performance when processing typical database workloads including both fetching rows by identifier and performing table scans.

Query and exadata support for hybrid columnar compressed data

Techniques are described herein for automatically selecting the compression techniques to be used on tabular data. A compression analyzer gives users high-level control over the selection process without requiring the user to know details about the specific compression techniques that are available to the compression analyzer. Users are able to specify, for a given set of data, a 'balance point' along the spectrum between 'maximum performance' and &#x201c;maximum compression&#x201d;. The point thus selected is used by the compression analyzer in a variety of ways. For example, in one embodiment, the compression analyzer uses the user-specified balance point to determine which of the available compression techniques qualify as &#x201c;candidate techniques&#x201d; for the given set of data. The compression analyzer selects the compression technique to use on a set of data by actually testing the candidate compression techniques against samples from the set of data. After testing the candidate compression techniques against the samples, the resulting compression ratios are compared. The compression technique to use on the set of data is then selected based, in part, on the compression ratios achieved during the compression tests performed on the sample data.

Compression analyzer

Disclosed are methods, systems, and computer program products for processing a file which include using a computer system that is programmed for performing a process of receiving the file in response to a request for storing the file, determining whether a database already contains the file, and storing the file in the database if the database does not already contain the file. The process may alternatively include receiving the file in response to a request for storing the file, determining whether a database already contains the file, and storing the file without storing the received file if the database already contains the file. The process may also alternatively include receiving the file in response to a request for storing the file in a database, separating the file into a plurality of portions, and storing the plurality of portions so each of the plurality of portions can be individually accessed.

System and method for data de-duplication

A highly flexible and extensible structure is provided for physically storing tabular data. The structure, referred to as a compression unit, may be used to store tabular data that logically resides in any type of table-like structure. According to one embodiment, compression units are recursive. Thus, a compression unit may have a 'parent' compression unit to which it belongs, and may have one or more 'child' compression units that belong to it. In one embodiment, compression units include metadata that indicates how the tabular data is stored within them. The metadata for a compression unit may indicate, for example, whether the data is stored in row-major or column major-format, the order of the columns within the compression unit (which may differ from the logical order of the columns dictated by the definition of their logical container), a compression technique for the compression unit, the child compression units (if any), etc.

Structure of hierarchical compressed data structure for tabular data

Techniques for storing and manipulating tabular data are provided. According to one embodiment, a user may control whether tabular data is stored in row-level or column-major format. Furthermore, the user may control the level of data compression to achieve an optimal balance between query performance and compression ratios. Tabular data from within the same table may be stored in both column-major and row-major format and compressed at different levels. In addition, tabular data can migrate between column-major format and row-major format in response to various events. For example, in response to a request to update or lock a row stored in column-major format, the row may be migrated and subsequently stored into row-major format. In one embodiment, table partitions are used to enhance data compression techniques. For example, compression tests are performed on a representative table partition, and a compression map is generated and applied to other table partitions.

DDL and DML support for hybrid columnar compressed tables

Described herein are compression and processing optimizations by using data transformation techniques. In example embodiments, a byte-wise differential transformation is applied to columnar data represented as a list of length-value pairs to determine a list of delta pairs that is subsequently compressed and stored on persistent storage. A length separation transformation is applied to separate a list of length-value pairs into a length array and a corresponding data value array, where these two arrays are subsequently compressed and stored separately on persistent storage. A native number transformation is applied to a set of number values to remove the lengths stored in the number values, where the transformed set is stored on persistent storage instead of the original set of number values. A native datetime-type transformation is applied to a set of datetime values to generate an encoding that is used to encode the set of datetime values into an encoded set that is stored on persistent storage instead of the original set.

Growing community of inventors

Maidenhead, United Kingdom

Kam Shergill