International Business Machines Corporation

Marc M Eshel

Frank Bernhard Schmuck

Dean Hildebrand

Manoj Premanand Naik

Renu Tewari

Roger Lee Haskin

Wayne A Sawdon

Deepavali M Bhagwat

Rajagopol Ananthanarayanan

James Christopher Wyllie

Rajagopal Ananthanarayanan

Vasily Tarasov

Martin G Kienzle

Christopher Meyer

D Scott Guthridge

Juan C Gomez

Donald N Jones

Jeremy Andrew Cohn

Daniel Lloyd McNabb

Brian P Dixon

Guerney Douglass Hunt

Frederick A Schwartz

Raymond Edward Rose

Lei Pan

Carol D Hartman

A computer-implemented method, according to one approach, includes performing a predetermined first data merge process in response to a determination that a version conflict exists between a plurality of versions of a data file resulting from editing performed on different cloud devices. The predetermined first data merge process includes determining a first merging policy of the data file from a plurality of potential merging policies, determining a plugin associated with the data file, and calling the plugin. The first merging policy is used as an input for the plugin, and the plugin includes predetermined conditions for determining first contents of the versions of the data file to exclude from a merge operation performed on the data file and second contents of the versions of the data file to include in the merge operation.

Calling a plugin and using a merging policy for mitigating version conflicts

One embodiment provides a method for storage management in a hierarchical file system that includes creating a structure for the hierarchical file system including a top-file system and sub-file systems that each include files and directories. Directories for the sub-file systems of the hierarchical file system are tracked. A request for creation of a unique directory in the hierarchical file system is received. Upon a request by a particular sub-file system for more inodes without any inodes available, inode numbers are revoked from at least one other sub-file system having unused inodes.

Global namespace for a hierarchical set of file systems

A system facilitates access to data in a network and includes a cache that stores instructions. A processor executes the instructions including: caching processing configured to integrate caching into a local cluster file system, and cache local file data in the cache based on fetching file data on demand from a remote cluster file system. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system. Applications execute on a multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface. Data cache is locally and remotely consistent for updates.

Scalable caching of remote file data in a cluster file system

One embodiment provides a method for re-balancing data and metadata across multiple sub-file systems of a file system. The method includes determining sub-file systems including an amount of data that exceeds a threshold. At least one cell in the sub-file systems is identified as a candidate for re-balancing. A re-balance process is performed on the at least one cell that includes performing a flush operation to flush dirty data from file system buffers, copying an inode table for an independent set of files in the at least one cell to a destination sub-file system, notifying an allocation manager for the destination sub-file system of a new storage pool to manage, and performing an un-quiesce operation on I/O operations to each element in the at least one cell.

Storage space management in union mounted file systems

One embodiment provides a method for storage management in an aggregated file system. The method includes tracking inode numbers for sub-file systems of the aggregated file system. A request for a unique range of inode numbers in the aggregated file system is received. The unique range of inode numbers is provided to one or more sub-file systems of the aggregated file system while providing that each sub-file system of the aggregated file system has a required range of inode numbers and that no two sub-file systems have overlapping inode numbers.

Providing unique inodes across multiple file system namespaces

One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a corresponding directory in a top-file system and a number of symbolic pointers in the corresponding directory which each point from the corresponding directory to a sub-file system where any portion of the tracked directory resides. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory does not exist in the second sub-file system, the system creates the second portion of the directory in the second sub-file system and creates a link from the first portion of the directory to the second portion of the directory.

Space management for a hierarchical set of file systems

One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for a unique directory in the hierarchical file system is received. The sub-file system of the hierarchical file system containing the unique directory is provided while ensuring that each directory resides in only one sub-file system. The system also maintains, in a top-file system, a corresponding directory and a symbolic pointer which points from the corresponding directory to a sub-file system where a given directory resides. Creation of new directories in sub-file systems also includes creation of corresponding directories in the top-file system and symbolic pointers from corresponding directory to new directory.

One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical file system. A request for creation of an additional directory in the hierarchical file system is received. In response to the request, the additional directory is created in a sub-file system of the hierarchical file system while providing that each directory resides in only one sub-file system. If the additional directory is a child directory, the additional directory is created in the sub-file system containing the parent directory of the child directory.

One embodiment provides a method for storage management in a hierarchical file system. The method includes tracking directories for sub-file systems of the hierarchical set of file systems and maintaining a directory while providing the directory is capable of residing in more than one sub-file system within the hierarchical file system. If available space in a first sub-file system is below a threshold, the system identifies a second sub-file system with available space above the threshold, wherein the first sub-file system contains a first portion of the directory. If a second portion of the directory exists in the second sub-file system, the system creates or expands a file in the second portion of the directory, otherwise the system first creates the second portion of the directory in the second sub-file system and then creates a link from the first portion of the directory to the second portion of the directory.

A system facilitates access to data in a network. One implementations includes a caching layer function that is configured to integrate into a local cluster file system, to cache local file data in a cache based on fetching file data on demand from a remote cluster file system, and to operate on a multi-node cache cluster. The cache is visible to file system clients as a Portable Operating System Interface (POSIX) compliant file system, applications execute on the multi-node cache cluster using POSIX semantics via a POSIX compliant file system interface, and data cache is locally consistent for updates made at the multi-node cache cluster using distributed locking for the data cache.

Embodiments of the invention relate to supporting coordinated access to a clustered file system's shared disk storage subsystem by using dynamic creation of file access layout for different workloads and access patterns. Active file servers in the cluster have equivalent access to shared data of said file system. In one embodiment, a particular file server in the cluster is configured to coordinate the file system's layout. The particular file server, in response to a particular client's request for a layout of a file stored by said file system in said subsystem determines a file access layout for a parallel file access protocol to use when servicing an I/O request by said client, marks said layout as active for the client, and sends the layout to the client. The file access layout is determined based on one or more of current file system resources, a specific workload type and a specific byte-range of the file included in the client request, and properties of the file. The layout includes a set of file servers in said cluster and byte ranges of the file, and the byte-range being associated with a particular server among the set.

Coordinated access to a file system's shared storage using dynamic creation of file access layout

Embodiments of the invention relate to coordinated access to a clustered file system's shared storage subsystem using a shared-lock architecture. A particular file server is configured to coordinate the file system's file access layout particular server, in response to a client's file access layout request for byte-ranges of a file stored in the storage subsystem being received by a file server in cluster and sent to said particular server, generates a file access layout of a set of file servers for a parallel file access protocol to use and byte ranges of the file that particular individual file servers to use, to service an I/O request for the file. The file server (that received the request), sends the generated file access layout to said client, and requests coordinated access to said file from the particular server in response to receiving an I/O request to byte-ranges of said file.

Coordinated access to a clustered file system's shared storage using shared-lock architecture

Scalable caching of remote file data in cluster file systems is provided. One implementation involves maintaining a cache in a local cluster file system and caching local file data in the cache by fetching file data on demand from the remote cluster file system into the local cached file system over the network. The local file data and metadata corresponds to the remote file data and metadata in the remote cluster file system. Updates made to the local file data and metadata are pushed back to the remote cluster file system asynchronously.

Embodiments of the invention relate to supporting coordinated access to a file system's shared disk storage subsystem using an automatic alignment of a parallel file access protocol and metadata management on a per object basis. A file server (among servers in the cluster) receives a client's request to access a file stored by the file system in the storage subsystem using a parallel file access protocol. If the file system has selected a file server to serve as a metadata manager of said file, the file server uses the selected file server to coordinate access to the file's metadata. If the file system has not selected a file server to serve as the metadata manager of the file, the file system selects the file server that received said request as the metadata manager for said file until a file close request operation is received.

Growing community of inventors

San Jose, CA, United States of America

Marc M Eshel