Netapp, Inc.

Lsi Logic Corporation

Lsi Corporation

Sprint Communications Comapny L.p.

Joseph Moore

William Patrick Delaney

Charles D Binford

Joseph Russell Blount

Randolph Wesley Sterns

William Hetrick

Keith Moyer

Donald Roman Humlicek

Stanley Krehbiel, Jr

Carey Wayne Lewis

Ziling Huang

Reid A Kaufmann

Scott Hubbard

Satish Sangapu

Jeff Stilger

Jeffrey A Stilger

Gerald James Fredin

Murat Bog

Martin Jess

Yanling Qi

Sai Rama Krishna Susarla

Matthew Barrow

Hubbert Smith

Scott W Kirvan

Shannon P Silvus

Timothy Gene Kelley

Chaoxin Charles Qiu

Gregory A Yarnell

Forrest Trimbell

Stanley E Krehbiel

Benjamin J Ewy

Reid Kaufmann

The application relates to writing arbitrarily sized data extents using in-line descriptors, such as headers or footers, into physical blocks to describe the data extents in those physical blocks. Some data extents may be smaller than the size of a physical block, and some data extents may be larger than the size of a physical block. A given physical block therefore includes an in-line descriptor for a data extent in that block, indicating that the data extent starts within that block and also an offset from the beginning of the block to where the data extent starts. The in-line descriptor is included within the block so that when the block is read both the data and the descriptor are read, thereby eliminating performance penalty for reading the descriptor information.

Systems, methods, and computer readable media providing arbitrary sizing of data extents

A method includes: storing a first data extent on a physical medium, wherein the physical medium is divided into a plurality of storage blocks, wherein each of the storage blocks has a size that is different than a size of the first data extent, further wherein the first data extent is stored to a first block of the plurality of storage blocks; generating a descriptor for the first data extent, wherein the descriptor indicates that the first data extent starts within the first block of the plurality of blocks and indicates an offset from the beginning of the first block at which the first data extent starts; and storing the descriptor within the first block.

A system shares I/O load between controllers in a high availability system. For writes, a controller determines based on one or more factors which controller will flush batches of data from write-back cache to better distribute the I/O burden. The determination occurs after the local storage controller caches the data, mirrors it, and confirms write complete to the host. Once it is determined which storage controller will flush the cache, the flush occurs and the corresponding metadata at a second layer of indirection is updated by that determined storage controller (whether or not it is identified as the owner of the corresponding volume to the host, while the volume owner updates metadata at a first layer of indirection). For a host read, the controller that owns the volume accesses the metadata from whichever controller(s) flushed the data previously and reads the data, regardless of which controller had performed the flush.

Read and write load sharing in a storage array via partitioned ownership of data blocks

A method, a computing device, and a non-transitory machine-readable medium for allocating data compression activities in a storage system are provided. A method includes tracking, by a storage controller, computing resources corresponding to a storage server. The storage controller processes one or more host read requests to access data requested by one or more hosts, the processing of the one or more host read requests including decompressing the data requested by the one or more hosts from the storage server and providing the decompressed data to the one or more hosts. The storage controller determines an amount of available computing resources after processing the one or more host read requests. Based on the amount of available computing resources, the storage controller performs inline compression of a first portion of host write requests and background compression of a second portion of the host write requests.

Systems and methods for allocating data compression activities in a storage system

A system for tracking metadata changes and recovering from system interruptions. With host I/O, corresponding metadata incremental changes are aggregated and stored in a write-ahead log before being performed to their in-memory buffers. As those buffers are flushed, checkpoints are created and stored in the log. As the log wraps to the start, older entries are overwritten after they are freed from any remaining dependencies by newer checkpoints. If metadata entities have not created new checkpoints, they are instructed to in order to free up space for new aggregated batches and checkpoints. After an interruption, the wrap point is located in the log. From the wrap point, the log is scanned backwards to provide checkpoints to metadata entities. The log is then scanned forwards to perform changes specified by aggregated batches. The metadata entities' volatile memory states are recovered to what they were before the interruption.

Write-ahead log maintenance and recovery

A method for mapping a first address space to a second address space is provided. In some embodiments, the method includes creating a first array of lookup entries and one or more second arrays of metadata entries for maintaining an ordering among the lookup entries using a tree structure. Each of the metadata entries includes one or more data index values identifying a corresponding one of the lookup entries by its position in the first array and one or more metadata index values identifying a corresponding one of the metadata entries by its position in one of the one or more second arrays. The method further includes receiving a request including a lookup value, traversing the tree structure to locate a lookup entry corresponding to the lookup value, and when the lookup value is located among the lookup entries, using the located lookup entry to process the request.

Memory efficient lookup structure

A system, method, and machine-readable storage medium for recovering data in a distributed storage system are provided. In some embodiments, the method includes identifying a failing storage device of a first storage node having an inaccessible data segment. When it is determined that the inaccessible data segment cannot be recovered using a first data protection scheme, a first chunk of data associated with the inaccessible data segment is identified and a group associated with the first chunk of data is identified. A second chunk of data associated with the group is selectively retrieved from a second storage node such that data associated with an accessible data segment of the first storage node is not retrieved. The inaccessible data segment is recovered by recovering the first chunk of data using a second data protection scheme and the second chunk of data.

Data recovery in a distributed storage system

A method, a computing device, and a non-transitory machine-readable medium for indexing data on a storage server are provided. A method includes providing, by a storage server, a plurality of sorted keys. A subset of the plurality of sorted keys that include a bit prefix is stored on a memory page. A trie representation is constructed that includes an entry corresponding to the memory page, wherein a position of the entry in the trie representation is determined based on a binary value of bits of the bit prefix. The trie representation is traversed, by the storage server, to access a record stored in the memory page, wherein the record corresponds to a key having the bit prefix.

Memory page indexing data structure

Systems and techniques for recovering a storage array are disclosed. These systems and techniques include determining a size corresponding to a storage stripe of the storage array. Pieces assigned to the storage stripe are identified. A storage configuration corresponding to the pieces assigned to the storage stripe is detected. Ordinal information and parity information are determined corresponding to the pieces assigned to the storage stripe. The size determined corresponding to the storage stripe, identification of the pieces assigned to the storage stripe, the storage configuration, the ordinal information, and the parity information is stored in a data store to reconstruct lost or corrupted metadata corresponding to the storage array.

System and method for recovering a storage array

Systems and methods for efficient input/output (I/O) workload capture are provided. For example, in one aspect, a machine implemented method includes: opening a network socket for listening to a connection request from a computing device; accepting the connection request from the computing device over the network socket; enabling selective data collection based on a network connection with the computing device over the network socket, where the network connection based selective data collection includes obtaining information regarding a plurality of input/output (I/O) requests and responses and performance information of a storage server for processing the I/O requests; sub-sampling the network connection based collected data; and sending at least a portion of the network connection based collected data over the network socket connection to the computing device.

Methods and systems for real-time activity tracing in a storage environment

A system and method of transposed storage in RAID arrays includes a storage manager with an interface for receiving storage requests associated with multi-segment blocks stored in a storage array, a controller for processing the received storage requests and controlling the storage manager, and a write request handler. The storage array includes a plurality of storage devices for which consecutive logical addresses are assigned to different ones of the storage devices. The write request handler is configured to process block write requests and send segment write requests to the storage array based on the write requests so that each segment of a given multi-segment block is written to a single respective one of the storage devices in the storage array. In some embodiments, the storage manager further includes a read request handler configured to determine logical addresses of requested segments and send segment read requests to the storage array.

System and method for transposed storage in RAID arrays

Growing community of inventors

Wichita, KS, United States of America

Joseph Moore