Emc Corporation

Nasuni Corporation

Stephen A Fridella

Uday K Gupta

Sorin Faibish

Xiaoye Jiang

David M Shaw

Robert S Mason, Jr

Kevin William Baughman

James Theodore Compton

Fernando Oliveira

Rossen P Dimitrov

Patrick Eaton

Peter C Bixby

Stephen A Rago

Surrendranath R Doohkan

Nikolay Popov

Walter A O'Brien, Iii

John T Cardente

Roy E Clark

John Forecast

Ivan Basov

Jiannan Zheng

Christopher H Stacey

Michael B Evans

Eyal Zimran

Gang Ma

Rui Liang

Raymond Angelone

Christian J Chuba

Richard G Chandler

Rossen Dimitrov

Managing file objects in a data storage system includes providing a plurality of metadata servers, each metadata server having information for only a subset of file objects in the storage system, each subset for each of the metadata servers being less than a total number of file objects for the system and includes providing at least one metadata location server, where an entity accessing a specific file object of the system determines which of the metadata servers contains data for the specific file object by first accessing the at least one metadata location server. Information for the subset of file objects may include a table having file identifiers and metadata location information. The metadata location information may point to data storage for a metadata object for a file.

Managing file objects in a data storage system

A versioned file system comprises a set of structured data representations. Each structured data representation corresponds to a version, and each version comprises a tree of write-once objects. Each version in the versioned file system has associated therewith a borrow window. When it is desired to reconstruct the file system to a point in time, i.e., to perform a 'restore,' it is only required to walk a single structured data representation. During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is pulled back from the cloud when the user goes to open the file. As a result, the entire file system is restored to a previous time nearly instantaneously. A fast restore is performed if an object being restored exists within a borrow window of the version from which the system is restoring.

Versioned file system with fast restore

A method of data sharing among multiple entities is provided. Each entity exports to a data store a structured data representation comprising a versioned file system local to that entity. The method begins by forming a sharing group that includes two or more entities. Sharing of the structured data representations by members of the sharing group is enabled. The filers use a single distributed lock to protect each version of the file system. This lock is managed to allow each filer access to the shared file system volume to create its new version. To share a fully-versioned file system, asynchronous updates at each of the filers is permitted, and each node is then allowed to 'push' its individual changes to the store to form the next version of the file system. A mechanism also may be used to reduce the period during which filers in the group operate under lock.

Versioned file system with sharing

Tracking storage resources includes providing a table containing storage resources along with capabilities and statuses thereof, updating the table in response to a change of status of a storage resource, updating the table in response to a change in capabilities of a storage resource and, in response to an inquiry for a storage resource having a particular capability, searching the table for a storage resource having the particular capability. Tracking storage resources may also include adding an element to the table in response to a new resource being added to the system. The capabilities may include RAID striping, data deduplication, and green operation. The status may be one of: on-line, off-line, and full.

Exposing storage resources with differing capabilities

A versioned file system comprises a set of structured data representations, such as XML. Each structured data representation corresponds to a 'version,' and each version comprises a tree of write-once objects rooted at a root directory manifest. Each version in the versioned file system has associated therewith a 'borrow window.' When it is desired to reconstruct the file system to a point in time (or, more generally, a given state), i.e., to perform a &#x201c;restore,&#x201d; it is only required to walk (use) a single structured data representation (a tree). During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is then pulled back from the cloud if/when the user goes to open the file. As a result, the entire file system (or any portion thereof) can be restored to a previous time nearly instantaneously. A &#x201c;fast&#x201d; restore is performed if an object being restored exists within a &#x201c;borrow window&#x201d; of the version from which the system is restoring. A version is pruned from the versioned file system by deleting all objects in the tree (associated with the version) that, at the time of pruning: (i) are not being lent to any other version within the borrow window of the version being pruned, and (ii) are not referenced in any other version whose borrow window is sufficiently large enough such that an object in the version could have been restored from that other version.

Versioned file system with pruning

Extensions to the Fragment Mapping Protocol are introduced which protect a disk array from malicious client access by exporting file system access information to the storage device. FMP requests received at the storage device can be authorized at a block granularity prior to completion, thereby limiting the exposure of the disk array to malicious clients. Client authorizations can be cached at the storage device to enable the permissions to be quickly extracted for subsequent client accesses to pre-authorized volumes.

Method for extending the fragment mapping protocol to prevent malicious access to virtualized storage

A versioned file system comprises a set of structured data representations, such as XML. Each structured data representation corresponds to a 'version,' and each version comprises a tree of write-once objects rooted at a root directory manifest. Each version in the versioned file system has associated therewith a 'borrow window.' When it is desired to reconstruct the file system to a point in time (or, more generally, a given state), i.e., to perform a &#x201c;restore,&#x201d; it is only required to walk (use) a single structured data representation (a tree). During a restore, metadata is pulled back from the cloud first, so users can see the existence of needed files immediately. The remainder of the data is then pulled back from the cloud if/when the user goes to open the file. As a result, the entire file system (or any portion thereof) can be restored to a previous time nearly instantaneously. A &#x201c;fast&#x201d; restore is performed if an object being restored exists within a &#x201c;borrow window&#x201d; of the version from which the system is restoring.

Accordingly a method and interface allows an attribute data base used by an Information Manager to be quickly populated and accurately maintained. A single Bulk Attribute Retrieval Request triggers the primary storage device to collect object attribute information. The method allows for selective collection of objects and attributes by providing filters and attribute lists in the Requests. The Request may be used to provide an incremental scan with appropriate time stamp filtering. In addition, the size of the results can be controlled by the IM by eliminating attributes that are not of interest to the IM. The Request is advantageously issued over a FileMover interface, which is an HTTP connection, and encoded in XML, allowing the IM to easily customize the Request as desired.

Method and apparatus for performing bulk file system attribute retrieval

Methods, systems, and computer program products for managing quality of service for shared storage accessed by a plurality of nodes are disclosed. According to one method, at a node of a plurality of nodes sharing access to a first disk array, quality-of-service (QoS) metadata associated with a portion of the disk array are requested from a server, wherein the QoS metadata includes at least one performance criterion for the portion of the disk array. I/O performance is monitored for the portion of the disk array in response to receiving the QoS metadata. The monitored I/O performance is compared with the QoS metadata. Based upon the comparison of the monitored I/O performance with the QoS metadata, it is determined whether a QoS violation has occurred. In response to determining that a QoS violation has occurred, the server is requested to migrate data associated with the portion of the disk array.

Methods systems, and computer program products for managing quality-of-service associated with storage shared by computing grids and clusters with a plurality of nodes

Managing files using layout storage objects

An improved policy enforcement architecture includes several components that improve the performance of policy enforcement in a hierarchical storage environment. A File System Query instruction permits a richer set of queries to be expressed by a policy engine, thereby permitting more complex policies to be implemented with ease. The File System result is generated at the file server, and only files of interest are forwarded to the policy engine. The file system query advantageously may be applied against a pre-generated index having one more characterizing attributes to further reduce the processing required to retrieve policy data. An Index Build instruction, added to a programming interface, allows a policy engine generate the characterizing indices in advance of use. Index maintenance techniques maintain consistency between the index and the file system to ensure that the policy can be enforced using accurate file information.

Apparatus and method for maintaining a file system index

A method is used in managing metadata. Data of an object is stored in a magnetic hard disk set in an object addressable data storage system. Metadata for the object is stored in an SSD set in the object addressable data storage system. The metadata includes information necessary for determining the location of the data of the object in the magnetic hard disk set.

Managing metadata

In a data processing system, a first processor pre-allocates data blocks for use in a file system at a later time when a second processor needs data blocks for extending the file system. The second processor selectively maps the logical addresses of the pre-allocated blocks so that when the pre-allocated blocks are used in the file system, the layout of the file system on disk is improved to avoid block scatter and enhance I/O performance. The selected mapping can be done at a program layer between a conventional file system manager and a conventional logical volume layer so that there is no need to modify the data block mapping mechanism of the file system manager or the logical volume layer. The data blocks can be pre-allocated adaptively in accordance with the allocation history of the file system.

Growing community of inventors

Newton, MA, United States of America

Stephen A Fridella