Oracle International Corporation

Alexander Tsukerman

Kothanda Umamageswaran

Juan R Loaiza

Jia Shi

Sumanta Kumar Chatterjee

Boris Erlikhman

Nilesh Choudhury

Selcuk Aya

Kiran Badrinarain Goyal

Bharat C V Baddepudi

Margaret Susairaj

Dmitry Mikhailovich Potapov

Sridhar Subramaniam

Vijay Sridharan

Akshay Shah

Sue K Lee

Adrian Ng

Vivekananda C Kolla

Wei-Ming Hu

Krishnan Meiyyappan

Richard L Frank

Neil J S MacNaughton

Roger J Hansen

Adam Y Lee

Eugene Reddy Ho

Vipin Gokhale

Umesh Panchaksharaiah

Poojan Kumar

Sue-Kyoung Lee

Bang Nguyen

Timothy L Shetler

Susy Fan

Scott Martin

Wei Zhang

Vikram Joshi

Sujatha Muthulingam

Zuoyu Tao

Adrian Tsz Him Ng

Allen Brumm

Vishwanath Karra

Semen Ustimenko

James Stenoish

Kesavan P Srinivasan

Robert K Abbott

Tudor Bosman

Daniel McClary

Zuoyo Tao

Mingmin Chen

Johnny Quan Zhou

Arvind Nithrakashvap

Aya Selcuk

Bharat CV Baddepudi

Roger Hansen

Akshay D Shah

A storage system communicatively coupled to a database management system (DBMS performs storage-side scanning of data sources that are not stored in native database storage format of the DBMS. Data sources for external tables are accessible in a storage system referred to as a distributed data access system (DDAS), e.g. a Hadoop Distributed File System. To execute a query that references an external table, a DBMS first generates an execution plan. The DDAS supplies the DBMS with information that specifies each portion of the data source, and specifies which data node to use to access the portion. The DBMS sends a request for each portion to the respective data node, requesting that the data node generate rows from data in the portion. The request may specify scanning criteria, specifying one or more columns to project and/or filter on, and code modules for the data node to execute to generate records.

Storage-side scanning on non-natively formatted data

A method and apparatus for intelligent network resource manager for distributed computing systems is provided. A first priority is assigned to a first virtual channel set that includes at least two virtual channels of a plurality of virtual channels associated with a physical communication channel. A second priority is assigned to a second virtual channel set that includes at least one virtual channel of the plurality of virtual channels. The first virtual channel set has more virtual channels than the second virtual channel set. Outbound messages of the first priority are directed to virtual channels of the first virtual channel set. Outbound messages of the second priority are directed to virtual channels of the second virtual channel set. The virtual channels are processed in a round-robin order, where processing includes sending the outbound messages over the physical communication channel.

Intelligent network resource manager

Techniques for optimizing a query with an extrema function are provided. In main memory, a data summary is maintained for a plurality of extents stored by at least one storage server. The data summary includes an extent minimum value and an extent maximum value for one or more columns. A storage server request is received, from a database server, based on a query with an extrema function applied to a particular column of a particular table. The data summaries for a set of relevant extents are processed by maintaining at least one global extrema value corresponding to the extrema function and, for each relevant extent of the set of relevant extents, determining whether to scan records of the relevant extent based on at least one of the global extrema value and an extent summary value of the data summary of the relevant extent.

Optimizing a query with extrema function using in-memory data summaries on the storage server

Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, d) whether the I/O request is associated with an offloaded filter provided by the database server to the storage system, or e) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Selectively reading data from cache and primary storage based on whether cache is overloaded

Techniques herein are for creating a database snapshot by creating a sparse database. A method involves receiving a creation request to create a sparse database. The creation request has an identity of a parent database. The creation request is processed to create a sparse database. The sparse database has the identity of the parent database. The sparse database does not contain data copied from the parent database. A write request to write data into the sparse database is received. The write request is processed by writing the data into the sparse database. The parent database does not receive the data.

Database snapshots

Techniques herein are for accessing non-materialized blocks of a sparse file. A method involves a storage system receiving a storage command to access a sparse file. A combined content of a set of materialized blocks and a header that identifies one or more non-materialized blocks is assembled. The combined content does not comprise a content of the one or more non-materialized blocks. Responsive to the assembling, the combined content is transferred between the storage system and a computer system.

Sparse file access

In a write by-peer-reference, a storage device client writes a data block to a target storage device in the storage system by sending a write request to the target storage device, the write request specifying information used to obtain the data block from a source storage device in the storage system. The target storage device sends a read request to the source storage device for the data block. The source storage device sends the data block to the target storage device, which then writes the data block to the target storage device. The data block is thus written to the target storage device without the storage device client transmitting the data block itself to the target storage device.

Data block movement offload to storage systems

Techniques are provided for using an intermediate cache between the shared cache of an application and the non-volatile storage of a storage system. The application may be any type of application that uses a storage system to persistently store data. The intermediate cache may be local to the machine upon which the application is executing, or may be implemented within the storage system. In one embodiment where the application is a database server, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which object an item belongs to, the item type of the item, a characteristic of the item, or the type of operation in which the item is involved.

Caching data between a database server and a storage system

Techniques are provided for using an intermediate cache to provide some of the items involved in a scan operation, while other items involved in the scan operation are provided from primary storage. Techniques are also provided for determining whether to service an I/O request for an item with a copy of the item that resides in the intermediate cache based on factors such as a) an identity of the user for whom the I/O request was submitted, b) an identity of a service that submitted the I/O request, c) an indication of a consumer group to which the I/O request maps, or d) whether the intermediate cache is overloaded. Techniques are also provided for determining whether to store items in an intermediate cache in response to the items being retrieved, based on logical characteristics associated with the requests that retrieve the items.

Selectively reading data from cache and primary storage

In a consumer-producer distributed model of computing, consumers request data from producers and perform computing work on the data received from the producers, thereby transforming the data. At least some of the computing work performed by a consumer on data provided by the producer is commutable between the consumer and producer, that is, either the producer as well as the consumer can perform the computing work on the data. A producer redistributes commutable based on the work load of the producer and consumer as well as other factors.

Redistributing computation work between data producers and data consumers

Methods, computer-readable media, and computer systems are provided for initiating storage of data on multiple storage devices and confirming storage of the data after the data has been stored on one but not necessarily all of the devices. A storage server receives, from a client, a request to store data. In response to the request, the storage server initiates, in parallel, storage of the data on multiple storage systems. The storage server detects that the data has been stored on any one of the storage systems, such as an auxiliary system, and, in response, indicates, to the client, that the data has been stored. The storage server may flush or discard data on the auxiliary storage system upon detecting that the data has been successfully stored on a target storage system, where the data persists.

Utilizing multiple storage devices to reduce write latency for database logging

Techniques are provided for managing, within a storage system, the sequence in which I/O requests are processed by the storage system based, at least in part, on one or more logical characteristics of the I/O requests. The logical characteristics may include, for example, the identity of the user for whom the I/O request was submitted, the service that submitted the I/O request, the database targeted by the I/O request, an indication of a consumer group to which the I/O request maps, the reason why the I/O request was issued, a priority category of the I/O request, etc. Techniques are also provided for automatically establishing a scheduling policy within a storage system, and for dynamically changing the scheduling policy in response to changes in workload.

Storage-side storage request management

Techniques are provided for using an intermediate cache between the shared cache of a database server and the non-volatile storage of a storage system. The intermediate cache may be local to the machine upon which the database server is executing, or may be implemented within the storage system. In one embodiment, the database system includes both a DB server-side intermediate cache, and a storage-side intermediate cache. The caching policies used to populate the intermediate cache are intelligent, taking into account factors that may include which database object an item belongs to, the item type of the item, a characteristic of the item; or the database operation in which the item is involved.

A method and apparatus for maintaining an item-to-node mapping among nodes in a distributed cluster is provided. Each node maintains locally-stored system-state information indicating that node's understanding of which master nodes are alive and dead. Instead of employing a global item-to-node mapping, each node acts upon a locally determined mapping based on its locally-stored system-state information. For any two nodes with the same locally-stored system-state information, the locally determined mapping is the same. A node updates its locally-stored system-state information upon detecting a node failure or receiving a message from another node indicating different locally-stored system-state information. The new locally-stored system-state information is transmitted on a need-to-know basis, and consequently nodes with different item-to-node mappings may operate concurrently. Mechanisms to avoid nodes assuming conflicting ownership of items are employed, thus allowing node failures to propagate via asynchronous messaging instead of requiring a cluster-wide synchronization event.

Maintaining item-to-node mapping information in a distributed system

Growing community of inventors

Foster City, CA, United States of America

Alexander Tsukerman