Netapp, Inc.

Nutanix, Inc.

Network Appliance, Inc.

Rickard Edward Faith

Blake H Lewis

Sandeep Yadav

Ashish Prakash

Subramaniam Periyagaram

Matti Antero Vanninen

Peter Scott Wyckoff

Ananthan Subramanian

Ram Kesavan

Douglas Michael Pase

Matthew D Hambrick

Jeffrey S Kimmel

Karan Gupta

Stephen William Daniel

Aman Nijhawan

Jeff Kimmel

Gary Jeffrey Little

Steve Daniel

Huapeng Yuan

Cyril Anthony Allen

Subramaniam Perivagaram

A method and apparatus for using cache size estimations for guiding hot-tier insertion decisions. The method and apparatus include an adaptive management element that determines what accesses of a resource should be logged and the parameters for logging. The determinations are used to configure an adaptive logging element to log only accesses corresponding to the selected resource(s) and to log only those accesses that match the identified parameters. The adaptive management element operates in a feedback loop: first determining what will be logged and second implementing that determination by an adaptive logging element. Upon a triggering event, the process returns to the first determination based on any then current parameters. In some embodiments, the parameters include a size estimate to achieve a given target hit rate (target hit rate size estimate) that is used in generating a score or weighting to identify the highest/best scoring/weighted disk(s) for logging.

Method and apparatus for using cache size estimations for guiding hot-tier insertion decisions

Disclosed is an improved approach to implement memory-efficient cache size estimations. A HyperLogLog is used to efficiently approximate an MRC with sufficient granularity to size caches.

Method and system for implementing cache size estimations

Embodiments serve to balance overall performance of a finite-sized caching system having a first cache of a first cache size and a second cache of a second cache size. A tail portion and a head portion of each of the caches are defined wherein incoming data elements are initially stored in a respective head portion and wherein evicted data elements are evicted from a respective tail portion. Performance metrics are defined wherein a performance metric includes a predicted miss cost that would be incurred when replacing an evicted data elements. A quantitative function is defined to include cache performance metrics and a cache reallocation amount. The cache performance metrics are evaluated periodically to determine a then-current cache reallocation amount. The caches can be balanced by increasing the first cache size by the cache reallocation amount and decreasing the second cache size by the cache reallocation amount.

Dynamic cache balancing

Partially overwriting a compression group without decompressing compressed data can consumption of resources for the decompression. A storage server partially overwrites the compression group when a file block identifier of a client's write request resolves to the compression group. The compression group remains compressed while the partial overwriting is performed.

Overwriting part of compressed data without decompressing on-disk compressed data

It is determined that a first data block contains the same data as a second data block. The first data block is associated with a first extent and the second data block is associated with a second extent. In response to determining that the first data block contains the same data as the second data block, the second data block is associated with the first extent and the first data block is disassociated with the second extent.

Deduplication in an extent-based architecture

An extent-based storage architecture is implemented by a storage server receiving a read request for an extent from a client, wherein the extent includes a group of contiguous blocks and the read request includes a file block number. The storage server retrieves an extent identifier from a first sorted data structure, wherein the storage server uses the received file block number to traverse the first sorted data structure to the extent identifier. The storage server retrieves a reference to the extent from a second sorted data structure, wherein the storage server uses the retrieved extent identifier to traverse the second sorted data structure to the reference, and wherein the second sorted data structure is global across a plurality of volumes. The storage server retrieves the extent from a storage device using the reference and returns the extent to the client.

Extent-based storage architecture

An extent-based storage architecture is implemented by a storage server. The storage server generates a new extent identifier for cloning a source extent identified by a source extent identifier and stored at a source data structure that includes a length value providing a length of the source extent, an offset value and a reference count value that provides a number of data containers that reference the source extent identifier. The storage server updates a data structure for a cloned version of the data container for storing the new extent identifier that points to the source extent identifier and includes an extent length value and offset value different from length value and the offset value of the source data structure.

Cloning using an extent-based architecture

Detection and reduction of unaligned input/output ('I/O') requests is implemented by a storage server determining an alignment value for data stored by the server within a storage system on behalf of a first client, writing the alignment value to a portion of the volume that stores the data for the first client, but not to a portion of the volume that stores data for a second client, and changing a location of data within the portion of the volume that stores the data for the first client, but not a location of data in the portion of the volume that stores data for the second client, to an alignment corresponding to the alignment value. The alignment value is applied to I/O requests directed to the portion of the volume that stores the data blocks for the first client after the location of the data blocks has been changed.

Dynamic detection and reduction of unaligned I/O operations

Overwriting part of compressed data without decompressing on-disk compressed data is implemented by receiving a write request for a block of data in a compression group from a client, wherein the compression group comprises a group of data blocks that is compressed, wherein the block of data is uncompressed. The storage server partially overwrites the compression group, wherein the compression group remains compressed while the partial overwriting is performed. The storage server determines whether the partially overwritten compression group including the uncompressed block of data should be compressed. The storage server defers compression of the partially overwritten compression group if the partially overwritten compression group should not be compressed. The storage server compresses the partially overwritten compression group if the partially overwritten compression group should be compressed.

A storage system tracks statistical behavior of client read requests directed to a storage device to form prediction about data that the client will require next. The storage system collects the size of read sequences for various streams into a data structure, which summarizes past behavior of read requests. This data structure reports the number of streams in each equivalence class of stream sizes that is tracked. The data structure is then used to determine expected size of a selected read stream. The data structure is also used to improve predictions about an expected size computed by a known technique.

Mechanism for determining read-ahead length in a storage system

A request is received to remove duplicate data. A log data container associated with a storage volume in a storage server is accessed. The log data container includes a plurality of entries. Each entry is identified by an extent identifier in a data structures stored in a volume associated with the storage server. For each entry in the log data container, a determination is made if the entry matches another entry in the log data container. If the entry matches another entry in the log data container, a determination is made of a donor extent and a recipient extent. If an external reference count associated with the recipient extent equals a first predetermined value, block sharing is performed for the donor extent and the recipient extent. A determination is made if the reference count of the donor extent equals a second predetermined value. If the reference count of the donor extent equals the second predetermined value, the donor extent is freed.

Example embodiments provide various techniques for initiating read-ahead requests. A rate at which applications is requesting data from a data storage device is identified. Additionally, a length of time in retrieving or servicing the data from the data storage device is also identified. The identified rate and length of time in retrieving the data are used to determine when read-ahead requests should be sent to pre-fetch data.

Initiation of read-ahead requests

Just-in-time segment cleaning obtains the location of blocks of a storage system predicted to be targeted by a future write operation and cleans the blocks targeted for the future write operation prior to the occurrence of the future write operation. In one aspect, just-in-time segment cleaning is performed in proportion to previous user-initiated write operations. In another aspect, just-in-time segment cleaning applies a cost analysis determination to minimize the work of cleaning.

Growing community of inventors

Hillsborough, NC, United States of America

Rickard Edward Faith