Hewlett-packard Development Company, L.p.

Emc Corporation

Netapp, Inc.

Daniel Alan Muntz

Zheng Zhang

Christos Karamanolis

Mallik Mahalingam

Sairam Veeraswamy

David Noveck

Harumi Kuno

Ravi Wijayaratne

Thomas J Dwyer, Iii

Alex Burlyga

William Kevin Wilkinson

Michael J Lemon

Baila Ndiaye

Described are techniques for creating a snapshot of a distributed file system or a file thereof. The distributed file system includes metadata file systems storing metadata about files in the distributed file system and includes data file systems storing file content data. Quiescing is performed for requests affecting the distributed file system, or a file thereof, for which a snapshot is being created. A snapshot of metadata is obtained. For the distributed file system, this includes a snapshot of each metadata file system. For a file, this includes a snapshot of the file's metadata. A snapshot of file data is obtained. For distributed file systems, this includes a snapshot of each data file system. For a file, this includes a snapshot of the file's data. Unquiescing is performed for requests affecting the distributed file system, or the file thereof, upon successful completion of obtaining snapshots of metadata and file data.

Distributed file system having separate data and metadata and providing a consistent snapshot thereof

Described are techniques for performing quota management in a distributed file system. An allocation quota specifies an upper limit of storage for allocation for a quota set of the distributed file system. The distributed file system includes metadata file system(s) and data file system(s). For files in the distributed file system, metadata is stored in one of the metadata file systems and file content data is stored in the data file systems. A quota coordinator partitions the allocation quota among the data file systems. The data file systems are allocators of storage and each receives a portion of the allocation quota specifying an upper limit of storage that the data file system has available for allocation for the quota set. Storage is allocated for client requests. The storage allocation is performed by each of the data file systems in accordance with the portion of the allocation quota assigned.

Providing file system quota support for a file system having separated data and metadata

A method implements a hierarchical cache system with a parallel Network File System (pNFS) configuration for a storage system. Upon receiving a request by the hierarchical cache system to access data stored in the storage system, the method divides the data into a plurality of data segments and distributes the plurality of data segments to a plurality of cache servers of the cache system. The method responds to the request a metadata layout for the plurality of data segments distributed among the plurality of cache servers. Based on the metadata layout, the plurality of data segments can be concurrently retrieved from the plurality of cache servers.

Method and apparatus to implement a hierarchical cache system with pNFS

Providing file sytem quota support for a file system having separated data and metadata

A method and system of enforcing file authorization access. The method may include generating an authorization combination at a metadata server and encrypting the authorization combination. The authorization combination may include a block combination including a block list for accessing user requested data from a storage server system and an authorization prefix. The authorization prefix may indicate at least one operation which the user requesting data access is authorized to perform. The method may further include receiving the encrypted authorization combination at the storage server, and decrypting the encrypted authorization combination to recover the block list for retrieving the user requested data.

Enforcing file authorization access

Method and system for performing a namespace operation in a distributed file system. The file system is disposed on a plurality of partition servers, and each partition server controls access to a subset of hierarchically-related, shared storage objects. Each namespace operation involves a namespace object and a target object that are part of the shared storage objects. Namespace operations received at each partition server are serialized. In response to an unlink namespace operation, a reference in the namespace object to the target object is removed, and after removal the target object is modified in accordance with the unlink operation. In response to a link operation, the target object is modified consistent with the link operation. After modification of the target object, a reference to the target object is inserted in the namespace object.

Namespace management in a distributed file system

A system and method for resource handling provides for receiving a query and providing a response. The query is used to search resources locally available. Metadata is associated with each of the resources. The resources and the metadata are both analyzed with respect to the query such that a response indicative of a match between at least two of the resources and the query is a ranked search result included in the response. The invention is exemplified in an adaptation to peer-to-peer network applications.

Resource searching

A method for responding to search requests for stored documents. Each of the stored documents includes metadata, content, and a document type identifier for identifying a type of the document. A search request including document type criteria, metadata criteria, and content criteria, is received. A first set of documents based on the metadata criteria is identified. Documents in the first set that have a document type identifier that matches the document type criteria are identified. A search method is identified from a plurality of search methods for searching the content of the identified documents in the first set based on the document type identifier of the documents. The content of the identified documents in the first set is searched using the identified search method and the content criteria.

Apparatus and method for responding to search requests for stored documents

A system and method for providing a plurality of client applications access to data in a distributed file system. In various embodiments, read requests are separated from write requests and the read requests are processed by dedicated read servers. A plurality of read servers are coupled to the client applications and each read server reads file data from the distributed file system and returns the file data to the client applications. A write server writes data to the distributed file system. Various embodiments are described for separating read requests from write requests and transmitting read requests to the read servers write requests to the write server.

Separate read and write servers in a distributed file system

System and method for managing leases in a distributed file system. A meta-data server is employed to manage leases to the objects in the distributed file system, and a plurality of storage servers provide data storage for the objects. The meta-data server grants leases for objects to the clients. With each lease granted, the meta-data server provides data that indicate the time at which the lease will expire for the associated object. The lease expiration time is included in a subsequent request to the storage server for access to the object. The storage server determines whether the lease has expired. If the lease has expired, the requested access is denied. Otherwise, the storage server provides the requested access.

Lease enforcement in a distributed file system

Method and system for migrating a logical volumes in a distributed file system having multiple partitions servers. Each partition server owns one or more volumes, and each volume including a subset of logically related objects of the file system. In response to a migration request that requests migration of a selected logical volume from a source partition server to the target partition server, an ownership map is updated to indicate the selected logical volume is owned by the target partition server. At the source partition server, forwarding information is stored to reference the target partition server. When the source partition server receives a request for access to the object, the forwarding information is transmitted to the requester.

Logical volume-level migration in a partition-based distributed file system

Method and arrangement for updating references to a migrated object in a distributed file system. A migrated object is an object has moved from a source partition server to a target partition server. Each object has an associated parent object and each partition server owns a subset of objects of the file system. A set of forward pointers that reference one or more child objects of the parent object is maintained in each parent object. In the migrated object a set of back pointers is maintained that reference one or more parent objects having forward pointers to the migrated object. To each partition server that owns a parent object of the migrated object, an update request is transmitted. The forward pointers in each parent object are updated in response to the update request.

Updating references to a migrated object in a partition-based distributed file system

Method and system for moving an object from a source partition server to a target partition server in a distributed file system having multiple partition servers. Each object has at least one associated and linked parent object, and each partition server owns a subset of objects of the file system. The object migration is accomplished by creating a copy in a target partition server of a selected object from a source partition server. At the source partition server, a forwarding link is created that references the copy in the target partition server. The copy is linked to the parent object of the selected object, and requests for access to the selected object are responded to with the forwarding link.

Object-level migration in a partition-based distributed file system

Namespace service in a distributed file system using a database management system. A namespace database is configured on a namespace server with namespace identifiers and associated file location information. The namespace server is separate from the data servers in the distributed file system. A client proxy arrangement interfaces with client applications and with the namespace server to obtain from the namespace server location information associated with files referenced in file access requests and submit storage access requests to the appropriate data servers. The separate namespace server and data servers enhances scalability of the distributed file system.

Growing community of inventors

Cupertino, CA, United States of America

Daniel Alan Muntz