Netapp, Inc.

Jared Cantwell

Austino Nicholas Longo

James Philip Wittig

Mark David Olson

Alyssa Proulx

Mattias Fornander

Travis Gockel

Joe Roback

Matt Holiday

Daniel David McCarthy

Peter P Waskiewicz, Jr

Christopher Cason

Marshall McMullen

Forrest Trimbell

William Minckler

Bill Minckler

Austin Longo

Saxon Parker

Jim Wittig

Carl Jeffrey Seelye

Disclosed are systems, computer-readable mediums, and methods for managing client performance in a storage system. According to one embodiment, a total Input/Output Operations per Second (IOPS) pool and a read/write IOPS pool are managed for clients to ensure their write requests can be accommodated by both pools. In one example, a write request is received from a client by the storage system. A requested number of write IOPS is determined for a time period to accommodate the request. Based on the requested number of write IOPS exceeding a number of allocated write IOPS to the client for the time period, a target total IOPS for the client during the time period is determined by subtracting the number of allocated write IOPS from a number of allocated total IOPS to the client. At least a portion of the request is performed by executing the target total IOPS during the time period.

Quality of service policy sets

Disclosed are systems, computer-readable mediums, and methods for managing client performance in a storage system. In one example, the storage system receives a request from a client to write data to the storage system. The storage system estimates, based on a system metric associated with the storage system reflecting usage of the storage system, a requested write QoS parameter for storing the data by the storage system during a first time period. The storage system further determines a target write QoS parameter for the client based on the estimated requested write QoS parameter and an allocated write QoS parameter for the client. Then, the storage system independently regulates read performance and write performance of the client using a controller to adjust the write performance toward the determined target write QoS parameter within the first time period based on feedback regarding the estimated requested write QoS parameter.

Disclosed are systems, computer-readable mediums, and methods for receiving a start replication message to replicate a source volume to a replicated volume. A source system forwards I/O requests to the replica server. A data structure associated with the replicated volume is initialized. A write request is received from the source system. The write data is written to the replicated volume and the data structure is updated. Source metadata associated with the source volume is received. The source metadata is compared with prior metadata associated with a prior point-in-time image of the source volume to determine blocks of data that have changed since the prior point-in-time image of the source volume. A first block is determined to not be retrieved based upon the data structure. A second block is determined to be retrieved based upon the data structure. The second block is received and written to the replicated volume.

Data syncing in a distributed system

Disclosed are systems, computer-readable mediums, and methods for managing input/output operations within a system including at least one client and a storage system. A processor receives information regarding allocated input-output operations (IOPS) associated with a client accessing a storage system storing client data. The information includes a number of allocated total IOPS, a number of allocated read IOPS, and a number of allocated write IOPS. The processor also receives a requested number of write IOPS associated with the at least one client's request to write to the storage system. The processor determines a target write IOPS based on the number of allocated total IOPS, the number of allocated write IOPS and the requested number of write IOPS, and executes the determined target write IOPS within the first time period.

A technique maintains multiple copies of data served by storage nodes of a cluster during upgrade of a storage node to ensure continuous protection of the data served by the nodes. The data is logically organized as one or more volumes on storage devices of the cluster and includes metadata that describe the data of each volume. A data protection system may be configured to maintain two copies of the data in the cluster during upgrade to a storage node that is assigned to host one of the copies of the data but that is taken offline during the upgrade. As a result, a slice service of the node may become unavailable during the upgrade. In response to the unavailability of the slice service, the technique redirects replicated data targeted to the slice service to a standby slice service according to a degraded redundant metadata (DRuM) service of the cluster.

Degraded redundant metadata, DRuM, technique

A technique is configured to maintain multiple copies of data served by storage nodes of a cluster during upgrade of a storage node to ensure continuous protection of the data served by the nodes. The data is logically organized as one or more volumes on storage devices of the cluster and includes metadata that describe the data of each volume. A data protection system may be configured to maintain at least two copies of the data in the cluster during upgrade to a storage node that is assigned to host one of the copies of the data but that is taken offline during the upgrade. As a result, an original slice service of the node may be rendered unavailable during the upgrade. In response, the technique redirects replicated data targeted to the original slice service to a standby pool of slice services in accordance with a degraded redundant metadata service of the cluster. In the event the standby slice service itself subsequently becomes unavailable, another standby slice service from the standby pool is activated to receive the subsequent data. In this manner, cascading failure of secondary slice slices is handled.

Standby copies withstand cascading fails

Disclosed are systems, computer-readable mediums, and methods for managing input-output operations within a system including at least one client and a storage system. A processor receives information regarding allocated input-output operations (IOPS) associated with a client accessing a storage system storing client data. The information includes a number of allocated total IOPS, a number of allocated read IOPS, and a number of allocated write IOPS. The processor also receives a requested number of write IOPS associated with the at least one client's request to write to the storage system. The processor determines a target write IOPS based on the number of allocated total IOPS, the number of allocated write IOPS and the requested number of write IOPS, and executes the determined target write IOPS within the first time period.

Systems, computer-readable mediums, and methods are configured to receive a first request to write a first data block to a solid state storage device. A second request is received to associate the first data block with a first snapshot identifier (ID). The first data block is stored on the solid state storage device in response to the first request. The first data block is stored on a first segment of a plurality of physical segments of memory on the solid state storage device. A first data pointer that corresponds to the first snapshot ID is generated in response to the second request. The first data pointer points to the first data block stored on the solid state storage device.

Log-structured file system

Data traffic of different customers or tenants can be efficiently handled at a shared node while still being isolated from each other. An application instance can create multiple network stack that are isolated from each other and intelligently manage threads across the isolated network stack instances. To intelligently manage the threads across the network stack instances, each thread maintains data that identifies the network stack to which the thread is assigned. With this information, the application can intelligently use a thread already assigned to a network stack that will process the data traffic and avoid the performance impact of a system call to assign the thread to the network stack.

Intelligent thread management across isolated network stacks

Disclosed are systems, computer-readable mediums, and methods for incremental block level backup. An initial backup of a volume is created at a backup server, where creating the initial backup includes retrieving an original metadata file from a metadata server, and retrieving a copy of all data of the volume based on the original metadata file. A first incremental backup of the volume is then created at the backup server, where creating the first incremental backup includes retrieving a first metadata file, where the first metadata file was created separately from the original metadata file. A block identifier of the first metadata file is compared to a corresponding block identifier of the original metadata file to determine a difference between the first and original block identifiers, and a copy of a changed data block of the volume is retrieved based on the comparison of the first and original block identifiers.

Incremental block level backup

Disclosed are systems, computer-readable mediums, and methods for efficiency sets in a distributed system. A first efficiency set is determined for a first volume of data. Determining the first efficiency set includes selecting block identifiers for data blocks of the first volume, where each block identifier is used to access a particular data block corresponding to the first volume. Determining the first efficiency set further includes applying a mask to the selected block identifiers to mask at least one bit of each selected block identifier. The first efficiency set is compared to a second efficiency set for a second data store, and based on the comparison, an amount of unique data blocks of the first volume is approximated.

Growing community of inventors

Boulder, CO, United States of America

Jared Cantwell