Datacore Software Corporation

Nicholas C Connolly

Roni J Putra

Ziya Aral

Robert Bassett

Edward Ignatoff

David A Linde

Mark H Friedman

Ye (Tom) Chen

Mohamed Ghanem

A containerized stream microservice is described. The containerized stream microservice is configured to provide the functionality of volume presentation along with all related interactions including the receipt and processing of IO requests and related services. The containerized stream microservice preferably implements stream metadata in the management of storage operations, and interacts with a store to provide underlying data storage. The store, which may also be referred to as a data store, is where underlying data is stored in a persistent manner. In one example, the store is an object store.

Containerized storage stream microservice

A containerized storage microservice is described. The containerized storage microservice, and its corresponding architecture, provide an environment wherein an application container accesses containerized storage services through a direct connection. This allows the operating system file system to be effectively bypassed in the provision of storage services. In one example, a library provides a basic file system and is arranged underneath the application container. The library essentially intercepts storage requests and emulates a thin file system. Instead of invoking the kernel of the operating system to process each storage request, the library allows the storage request to bypass the kernel and pass the storage request to the storage microservice. The containerized storage microservices are available in different types, and are configured so that they can be stacked to provide customized sets of storage services to different types of application containers.

Containerized storage microservice with direct connection to requesting application container

An LRU buffer configuration for performing parallel IO operations is disclosed. In one example, the LRU buffer configuration is a doubly linked list of segments. Each segment is also a doubly linked list of buffers. The LRU buffer configuration includes a head portion and a tail portion, each including several slots (pointers to segments) respectively accessible in parallel by a number of CPUs in a multicore platform. Thus, for example, a free buffer may be obtained for a calling application on a given CPU by selecting a head slot corresponding to the given CPU, identifying the segment pointed to by the selected head slot, locking that segment, and removing the buffer from the list of buffers in that segment. Buffers may similarly be returned according to slots and corresponding segments and buffers at the tail portion.

Methods and apparatus for LRU buffer management in performing parallel IO operations

Command list processing in performing parallel IO operations is disclosed. In one example, handling IO requests directed to an operating system having an IO scheduling component entails allocating a command to a thread in association with an IO request. The command is allocated from one of a plurality of command lists accessible in parallel, and the command is also linked to one of a plurality of active command lists that are accessible in parallel. The command lists can be arranged as per-CPU command lists, with each per-CPU command list corresponding to one of a plurality of CPUs on a multi-core processing platform on which the IO requests are processed. Similarly, each of the active command lists can respectively correspond to one of the plurality of CPUs on the multi-core processing platform. Per-volume queues can also be implemented for respective volumes presented to applications.

Methods and apparatus for command list processing in performing parallel IO operations

A node based architecture that supports arbitrary access to any node in a system for data representation and access, while still providing virtual volume coherency that is global to all of the nodes in the system, and while providing underlying data management services that are also variously accessible from any node in the system.

Multi-node data store management

Methods and apparatus for data request scheduling in performing parallel IO operations are disclosed. In one example, IO requests directed to an operating system having an IO scheduling component are processed. There, an IO request directed from an application to the operating system is intercepted. A determination is made whether the IO request is subject to immediate processing using available parallel processing resources. When it is determined that the IO request is subject to immediate processing using the available parallel processing resources, the IO scheduling component of the operating system is bypassed. The IO request is directly and immediately processed and passed back to the application using the available parallel processing resources.

Methods and apparatus for data request scheduling in performing parallel IO operations

A stream architecture for data representation is disclosed. A stream is retained with respect to write transactions to a given volume. The stream includes the write transactions and provides a sequence of the write transactions according to the order in which the write transactions were received for the given volume. By way of example, a stream image can be associated to the stream. The stream image provides a representation of the data in the given volume for a point in time by referencing a non-contiguous subset of the sequence of write transactions in the stream.

Stream architecture for data representation

Disclosed are network managed volumes that enable simplified and more efficient use of information technology assets by allocating physical disk drives as needed. A pool of storage can be defined and populated with physical disks that are accessible in a storage area network. Virtual volumes are created from the storage pool, with each virtual volume being separately allocable to a host device (e.g., client, application server, etc.). Thus, host devices that have been allocated these virtual volumes perceive ownership of the entire volume, without requiring a present allocation of physical storage space having the same characteristics, but with seamless introduction of physical resources as needed.

Network managed volumes

A computer network system includes a client-server architecture, where the client transmits SCSI commands over a network using TCP/IP to a server to request input and output operations for a physical storage device that is communicatively coupled to a storage device. The client perceives the storage device to be a local disk drive, even though the storage device is located on the target side of a network. The system includes a client-side disk-based cache.

Growing community of inventors

Purley, United Kingdom

Nicholas C Connolly