International Business Machines Corporation

Boaz Shmueli

Frank Bernhard Schmuck

James Christopher Wyllie

Craig Fulmer Everhart

Owen Theodore Anderson

Anthony J Zlotek

Robert J Curran

Jason Christopher Young

James John Seeger, Jr

William A Kish

Zvi Yosef Yehudai

John Michael Marberg

Benjamin Mandler

Irit Loy

Roman Talyansky

Daniel Lloyd McNabb

A method and system for allocating blocks of disk in persistent storage to requesting threads. A primary data structure is provided for organizing and categorizing blocks of disk space. In addition, a secondary data structure is provided for maintaining a list of all active file system processes and blocks of disk space used by those processes. Blocks of disk space are assigned to pages. At such time as a thread may request allocation of disk space, both data structures are reviewed to determine if the requested disk space is available and to limit access of available disk space to a single page of memory to a single thread at any one time.

Disk space allocation

Improved techniques are disclosed for accessing content in file systems, allowing file system clients to realize advantages of file system referrals even though a file access protocol used by the client is not specifically adapted for referral objects. (For example, the client may have a legacy file system protocol or a proprietary file system protocol which does not support referrals.) These advantages include a uniform name space view of content in a network file system, and an ability to locate content in a (nearly) seamless and transparent manner, even though the content may be dynamically moved from one location to another or replicated in different locations. A file system server returns a symbolic link in place of a referral, and an automated file mounting process on the client is leveraged to access the content using the link. Built-in crash recovery techniques of the file system client are leveraged to access moved content.

Uniform name space referrals with location independence

Method for improving disk space allocation

In a cluster of computing nodes having shared access to one or more volumes of data storage using a parallel file system, a method for managing the data storage includes initiating a data management application in the cluster using a data management application programming interface (DMAPI) of the parallel file system. When a request submitted to the parallel file system is received on one of the nodes to perform an operation on a file in one of the volumes of data storage, a data management access right is obtained from the DMAPI responsive to the request The operation is performed on the file using the access right.

Implementing data management application programming interface access rights in a parallel file system

In one form, in a method for mapping file handles, protocol data elements are created for respective file system protocols. Such a protocol data element identifies a structure of server handles for the data element's corresponding protocol. File system data elements are created for server file systems. Such a file system data element includes a file system identification (FSID) attribute. Responsive to accessing an object of one of the server file systems, a value for the FSID attribute of the corresponding file system data element is created for reconstructing the object's server handle. Creating the value includes parsing, responsive to one of the protocol data elements, an FSID of a server handle for the object.

Method, apparatus and computer program product for mapping file handles

A system provides referencing from one file system server to another through the use of a file system location database, improving movement and replication of file systems. When a file system is moved from a first file system server, a data object that references the file system remains in the first server and contains information used to find the current location of the file system. The actual location of the file system is stored in the separate file system location database, which contains the locations of file systems on a number of file system servers. This allows the data in a file system to be replicated or moved without requiring updates to the data in any redirecting or referencing servers.

Method, apparatus, and program for separate representations of file system locations from referring file systems

In one form, in a method for making file systems accessible, a referring file system attribute is created, responsive to a first file system having a referral object referring to a second file system. The referring file system attribute describes referring file system identification for the first file system. A path attribute is created, responsive to the first file system having the referral object, describing a path from a root of the first file system to the referral object. A referred file system attribute is created, responsive to the first file system having the referral object, describing a referred file system identification for the second file system. A parent attribute is created, responsive to the first file system having the referral object, describing a parent flag for the second file system.

Method, apparatus and computer program product for file system referrals

A technique for asynchronous computer disk cache management reduces the burstiness of disk I/O. It provides a periodic sync process to flush modified buffers, wherein only buffers exceeding a threshold age are selected to be flushed. In one variant the sync process only executes when the number of modified buffers in the cache exceeds a threshold, which can be dynamically adapted to changing patterns of cache usage. The age of the dirty buffers is measured by sync periods to minimize time computation. In some embodiments an additional cleaning process flushes old buffers between sync invocations.

Smoothing bursts of disk load in a file system

A computer system having a shared disk file system running on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. A metadata node manages file metadata for parallel read and write actions. Metadata tokens are used for controlled access to the metadata and initial selection and changing of the metadata node.

Parallel file system with method using tokens for locking modes

A computer system having a shared disk file system running on on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory service for the system with a stable cursor, a segmented allocation map. Dynamic prefetch and cached balance pools for multiple accesses improve the system. Extended file attributes are used for implementation of Access Control Lists in a parallel file system. Improvements to caching and cache performance developments balance pools for multiple accesses. A metadata node manages file metadata for parallel read and write actions. For our system, tokens are used for metadata node selection and identification, and we have enhanced token modes for controlling file size, as well as smart caching of byte range tokens using file access patterns and a byte range lock algorithm using a byte range token interface. Locking techniques reduce the overhead of a token manager which is also used in the file system recovery if a computer participating in the management of shared disks becomes unavailable or failed.

Parallel file system and method for multiple node file access

A computer system having a shared disk file system running on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Access to a file by a processor node is controlled by tokens transferred to the node from a token manager. To prevent another processor node from removing a token after the token has been received, but before it performs the operation on the file, each process can lock the token after it has been received. A node with a token can lock a byte range of a file, which byte range may include all or only some of byte range cornered by the token.

Parallel file system and method with byte range API locking

A computer system having a shared disk file system running on on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory, service for the system with a stable cursor, a segmented allocation map. Dynamic prefetch and cached balance pools for multiple accesses improve the system. Extended file attributes are used for implementation of Access Control Lists in a parallel file system, improvements to caching and cache performance developments balance pools for multiple accesses. A metadata node manages file metadata for parallel read and write actions. For our system, tokens are used for metadata node selection and identification, and we have enhanced token modes for controlling file size, as well as smart caching of byte range tokens using file access patterns and a byte range lock algorithm using a byte range token interface. Locking techniques reduce the overhead of a token manager which is also used in the file system recovery if a computer participating in the management of shared disks becomes unavailable or failed.

Parallel file system and method for parallel write sharing

A computer system having a shared disk file system running on multiple computers each having their own instance of an operating system and being coupled for parallel data disks. One of the computers serves as a metadata node and manages metadata for parallel read and write actions for a particular file. Metadata is transferred from and to this file from the other computers via this medadata node computer.

Parallel file system and method with a metadata node

A computer system having a shared disk file system running on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory service for the system with a stable cursor, a segmented allocation map. Dynamic prefetch and cached balance pools for multiple accesses improve the system. Extended file attributes are used for implementation of Access Control Lists in a parallel file system. Improvements to caching and cache performance developments balance pools for multiple accesses. A metadata node manages file metadata for parallel read and write actions. For our system, tokens are used for metadata node selection and identification, and we have enhanced token modes for controlling file size, as well as smart caching of byte range tokens using file access patterns and a byte range lock algorithm using a byte range token interface. Locking techniques reduce the overhead of a token manager which is also used in the file system recovery if a computer participating in the management of shared disks becomes unavailable or failed.

Growing community of inventors

Pittsburgh, PA, United States of America

Boaz Shmueli