Network Appliance, Inc.

Netapp, Inc.

Data General Corporation

Eric C Hamilton

Robert L Fair

John K Edwards

Robert M English

Ashish Prakash

Himanshu Aggarwal

Jeffrey S Kimmel

Peter Frank Corbett

Blake H Lewis

John A Scott

Richard Parvin Jernigan, Iv

Andy Kahn

Tianyu Jiang

Darrell Gene Suggs

Abhijeet Prakash Gole

John F Pilat

Douglas M Wells

Peter Griess

J Theodore Compton

John K Edwars

A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols. The file system layout extends the file system layout of a conventional write anywhere file layout system implementation, yet maintains performance properties of the conventional implementation.

Extension of write anywhere file system layout

A file system layout apportions an underlying physical volume into one or more virtual volumes (vvols) of a storage system. The underlying physical volume is an aggregate comprising one or more groups of disks, such as RAID groups, of the storage system. The aggregate has its own physical volume block number (pvbn) space and maintains metadata, such as block allocation structures, within that pvbn space. Each vvol has its own virtual volume block number (vvbn) space and maintains metadata, such as block allocation structures, within that vvbn space. Notably, the block allocation structures of a vvol are sized to the vvol, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the vvols.

In one embodiment of the present invention, a method and system are provided to control access to the non-volatile log (NVlog) of a storage server. By controlling access to the NVLog of a storage server the relative disk write bandwidth available to different client write requests can be controlled. The incoming write request can be categorized, and, during times of heavy load, only be permitted to use NVLog space as permitted based on the categorization of each write request. In one embodiment, the present invention includes receiving a write request from a client at a storage server, and determining whether the received write request can be presently logged in a NVlog based on a category of the write request.

Controlling write request access to a non-volatile log

The present invention provides a system and method for flexible space reservations in a file system. An administrator of a storage system selects a fractional reserve percentage, which defines the size of the overwrite reserve that is to be maintained to ensure file writeability.

System and method for flexible space reservations in a file system supporting persistent consistency point image

A storage controller is operable concurrently as an independent storage server and as at least a portion of a node of a storage server cluster. A network storage system comprises multiple storage controllers and multiple independent storage servers, each storage server implemented in a separate storage controller, each of the storage server configured to present to users a separate system image of stored data. The storage servers are independently operable and manageable. The system further includes a storage server cluster that includes cooperating storage server nodes, distributed among the storage controllers. The storage server cluster presents to users a single system image of data stored in the server cluster independently of the node or nodes in which said data resides. The storage server cluster is operable independently of the independent storage servers and is operable concurrently with operation of at least one of the independent storage servers.

Storage server cluster implemented in and operating concurrently with a set of non-clustered storage servers

An underlying physical volume of a storage system is an aggregate having a plurality of storage devices. The aggregate has its own physical volume block number (pvbn) space. A file system layout apportions the underlying physical volume into a plurality of virtual volumes of the storage system each having a virtual volume identification (vvid). Each virtual volume has its own virtual volume block number (vvbn) space. The block allocation structures of a virtual volume are sized to the virtual volume, and not to the underlying aggregate, to thereby allow operations that manage data served by the storage system (e.g., snapshot operations) to efficiently work over the virtual volumes. Each storage block in a virtual volume is identified by the triplet: pvbn, vvid, and vvbn.

A computer data storage system is described. A processor maintains a striped volume set by striping a data container over a plurality of storage nodes. A storage node determines whether space available on that node is below a predetermined threshold, the predetermined threshold indicating a low-in-space state. The storage node sends a message indicating that the storage node is in a low-in-space state. The processor accepts no further write messages to the data container as long as the storage node is in a low-in-space state.

System and method for providing space availability notification in a distributed striped volume set

A technique uniquely identifies block context signatures in a storage volume hierarchy of a storage system. In particular, the technique assigns unique volume identifiers ('IDs,' e.g., buffer tree or 'bufftree' IDs) to volumes of the storage volume hierarchy, and allows clone volumes to determine that data blocks belong to an appropriate ancestor volume (e.g., and were written by the ancestor volume prior to creation of the clone). In this manner, the novel technique may uniquely identify and accurately determine whether an accessed data block is a correct data block (e.g., alleviating occurrences of data ID aliasing).

Uniquely identifying block context signatures in a storage volume hierarchy

A storage system, such as a file server, creates persistent consistency point images (PCPI) or 'snapshots' that are point-in-time representations of the storage system. The storage system uses persistent consistency point images to backup or mirror data. Using data already found in the data block maps associated with these PCPI, the rate of change of data between persistent consistency point images can be determined.

Rate of change of data using on-the-fly accounting

A storage system, such as a file server, creates persistent consistency point images (PCPI) or 'snapshots' that are point-in-time representations of the storage system. The storage system uses persistent consistency point images to backup or mirror data. Using data already found in the data block maps associated with these PCPI, the amount of space used by a set of these consistency points can be determined.

Method and system for determining reclaimable space occupied by a set of snapshots

A system and method for managing data during consistency points in a storage system is provided. A buffer data control structure is modified to include a flags array that tracks various status flags for both a current and a next consistency point (CP). By utilizing multiple pointers within a buffer control structure, the storage system may permit write operations to continue to a data container undergoing write allocation. Received writes during a write allocation procedure are stored in raw data buffers and the buffer control structure is marked as being dirty for a next CP.

System and method for managing file data during consistency points

A system and method are provided to manage storage space. The method comprises detecting a free storage space threshold condition for a storage volume and automatically applying a space management technique to achieve a free storage space threshold condition. Space management techniques comprise deleting selected backup data (e.g., persistent consistency point images) and automatically increasing the size of the storage volume.

Method and system for automatic management of storage space

A system and method for managing space availability in a distributed striped file system is provided. A master data server is configured to send space availability detection messages to a plurality of data volumes servers hosting constituent volumes of a striped volume set. If one of the constituent volumes in the striped volume set has a low-in-space flag set, then the master data volume instructs all of the constituent volumes to set a low-in-space required flag, and no further writes are accepted for the striped volume set. The low-in-space and low-in-space required flags represent two states, and these states are returned in response to subsequent space availability detection messages from the master data server. A procedure for utilizing reserved space to complete an accepted cross stripe write operation is also provided.

A system and method for enabling write operations to files undergoing write allocation is provided. The system and method generate a shadow state entry of metadata associated with an inode of the file upon receipt of a write operation. During the write allocation process for the inode, the shadow state information is merged with the data stored in the inode to be written to disk.

System and method for managing file metadata during consistency points

A network storage filer implements a four-phase commit protocol to coordinate consistency points among a group of volumes (a 'group consistency point') such that, upon recovery, all volumes of the group reflect consistency points taken at the same time. The filer records progress at each phase while creating the consistency points of the volumes. Upon recovery, the same or a different filer uses the last recorded progress indicator to ascertain whether the consistency points were completed on all of the volumes in the group. If all the volume consistency points of the group consistency point were completed, the filer users the volume consistency points to mount the volumes. Otherwise, the filer uses an earlier group consistency point to mount the volumes. In either ease, the volumes are consistent with each other.

Growing community of inventors

Sunnyvale, CA, United States of America

Eric C Hamilton