International Business Machines Corporation

Hewlett-packard Development Company, L.p.

Ralph Attila Becker-Szendy

James Christopher Wyllie

D Scott Guthridge

Robert Michael Rees

Guillermo Alvarez

Veera W Deenadhayalan

Rajagopal Ananthanarayanan

Arif Abdulhusein Merchant

John Wilkes

Mustafa Uysal

Jason Christopher Young

Randal Chilton Burns

Darrell D E Long

David Macnaughton Wolfe

David Jones Craft

Richard Andrew Golding

Winfried W Wilcke

James J Seeger

Theodore Ming-Tao Wong

Jujjuri Venkateswararao

Scott D Guthridge

Fabian E Bustamante

Miriam Sivan-Zimet

Caixue Lin

Ahmed Zaki Omer

A method for performing stripe placement within a storage system is disclosed. After a set of failure domains within a storage system has been identified, the failure domains are then organized to form a hierarchy of failure domains. A failure domain is defined as a group of one or more disks that are more likely to fail together because a common component is shared by that group of disks. Stripe placement is performed across all active failure domains within the storage system using a greedy algorithm.

Failure domain based storage system data stripe layout

Embodiments of the invention relate to accessing data that spans multiple storage devices in a storage system. A first index records identifiers of data blocks and their storage locations, and a second index records storage locations. An update request is processed by writing multiple copies of a received data block identifier and the updated data block to storage locations recorded in the second index. A read request is processed by performing a quorum read of the storage locations recorded in the second index to retrieve the updated data blocks. The processing of the read request also includes performing a quorum read of the first index to determine storage locations that contain data blocks to be read as part of processing the read request. The data blocks to be read exclude the updated data blocks.

Accessing data in a storage system

A data validation method comprises associating a validation chunk with one or more data chunks stored on one or more storage mediums. The validation chunk comprises validation information for verifying whether data recorded in a subdivision of the one or more data chunks is lost. A request is received for an operation to be performed on first data stored in a first subdivision of a first data chunk from among said one or more data chunks. One or more portions of the validation information are examined to determine whether the data recorded in said first subdivision is deemed to be lost.

Monitoring lost data in a storage system

This disclosure relates to parity declustered storage device arrays having partition groups. In an exemplary embodiment, the storage system includes a storage device array, such as disk array. Each storage device is divided into partitions. Each partition includes stripe units, such as hundreds or thousands of stripe units in exemplary embodiments. The storage system also includes a physical array controller coupled to the storage device array. In an exemplary embodiment, the array controller includes a partition group lookup table and stores and retrieves data and parity in the storage devices based on the partition group lookup table. In this exemplary embodiment, the array controller also includes a stripe lookup table and/or a log. In an exemplary embodiment, the partition group lookup table and the stripe lookup table take up less memory (e.g., by an order of magnitude) than a single-level stripe map conveying the same information.

Parity declustered storage device array with partition groups

A method and apparatus for storing data is provided. One implementation involves providing a fiber medium for storing data, wherein the fiber medium has a characteristic configured to irreversibly change when exposed to write irradiation. The fiber medium is logically partitioned into cells along the length of the fiber medium. Data is stored in a cell of the fiber medium by exposing the cell to write irradiation to irreversibly change characteristic of the bulk of the cell.

Storing data on fiber data storage media

A scheduler selects an I/O from a session of a pool and updates token buckets associated with resource limits and reserves for the session and the pool and statistics used in determining fair sharing. To select an I/O, the scheduler identifies sessions with a non-empty queue, identifies head I/Os in the queues, computes for the head I/O a deadline using session and pool reserve buckets and a release time using session and pool limit buckets, and selects a head I/O with an earliest deadline that is past the release time. If the deadline of the selected candidate head I/O is in the past, the scheduler transfers the selected head I/O to the tail of the storage device queue. Otherwise, the scheduler selects the pool with the least amount of I/O traffic according to a session fair share estimator.

System and method for managing storage system performance as a resource

A technique for programmatically obtaining experimental measurements for model construction. A user provides criteria for the model, such as computational algorithms which characterize behavior of the real system, specifications of experiments to be performed on the real system for collecting experimental data from the real system, an identification of sought parameters which are to be derived from results of the experiments and desired tolerance constraints on the sought parameters. From experimental data collected from the real system and from the provided criteria, the inventive method and apparatus programmatically determines in an iterative loop which additional experiments are to be performed in order to achieve the desired tolerance constraints. After one or more iterations of the loop, the values for the sought parameters are determined within the desired tolerance constraints.

Technique for programmatically obtaining experimental measurements for model construction

Indirect access to local file systems is provided using storage tank protocols, allowing federation of a local file system into a distributed file system while preserving local access to the existing data in the local file system. The goal of the present system is to federate and migrate the data on a computer system with minimum disruption to applications operating on the computer system. Existing applications on a computer system continue to operate during data federation and migration and require little or no reconfiguration either when the data migration starts or when it ends. Data consistency is maintained: existing applications may modify data in the file system during migration or federation. During federation, other computer systems (or hosts) may modify the data in the file system if access control information allows them to do so. All changes in the file system are seen consistently on all hosts. Minimal downtime is required to install the present system and reconfigure the existing applications to communicate with the present system.

System, method, and service for federating and optionally migrating a local file system into a distributed file system while preserving local access to existing data

A system and method for managing logical versions of a filesystem made through a near-instantaneous copy process is provided. Each logical version of the filesystem, including the primary version, is assigned a unique epoch number or other identifying value, and each filesystem object is assigned two epoch numbers that define the space of logical versions for which the object exists. A list of all valid epoch numbers are maintained in a version table. The near-instantaneous creation of a logical version of the filesystem requires inserting a unique epoch number entry in the version table () and the next epoch number to the new logical version. An original state of the system is preserved through the use of a copy-on-write procedure for filesystem objects that are reference by that logical filesystem version (). Accordingly, filesystem versions are maintained and original states of previous states of the logical filesystem versions are preserved.

Managing filesystem versions

A method and system for asserting a lock in a distributed file system is provided. All distributed locks have a lease for a limited time period, wherein the lease may be renewed periodically. A lock manager data structure is provided to maintain mode compatibility of locks granted to different client nodes for the same object. The process of acquiring or reasserting a lock includes determining whether there are other valid locks in use, as well as whether a valid lock is in operation in a conflicting or compatible mode with a requested lock. A new lock lease may be granted and a lock lease which has expired may be reasserted if a conflicting lease is not present.

Discipline for lock reassertion in a distributed file system

Various copy-on-write implementations for a computing environment are presented. One copy-on-write implementation includes employing a read mapping table to perform a first virtual block to physical block mapping for use in reading a block of data of a file to be copy-on-written from physical storage for modification; and employing a different, write mapping table to perform a second virtual block to physical block mapping for use in writing a modified block of the file data to physical storage, wherein copy-on-write of the block of data is achieved using a single write operation. In another implementation, a distributed copy-on-write of a file for a client server environment is presented. This distributed copy-on-write includes performing, by a first client, copy-on-write of at least one block of data of the file to be copy-on-written, and performing, by a second client, copy-on-write of at least one other block of data of the file, wherein multiple clients perform the copy-on-write of the file.

Method, system and computer program product for implementing copy-on-write of a file

A method predicts performance of a system that includes a plurality of interconnected components defining at least one data flow path. The method references a workload specification for the system. The method models the system using one or more component models. Each component model represents selected one or more of the components. Each component model is arranged in like relationship to the data flow path as the selected one or more of the components represented by the component model. Each component model is (a) a constraint upon the workload specification input to that component model or (b) a transformer of the workload specification input to that component model so as to result in one or more output workload specifications that are input workload specifications to subsequent component models along the data flow path or (c) both a constraint and a transformer. At least one of the component models is a constraint. At least some of the component models along the data flow path operate on the workload specification. In one preferred form, operating on the workload specification involves arranging the component models in a hierarchy corresponding to the data flow path; using the specified workload specification as input to the topmost component model in the hierarchy; and applying one or more of the component models to its input workload specification, starting with the topmost component model and then component models at progressively lower levels in the hierarchy. Output workload specification at one level is input workload specification at the next lower level. If the component model comprises a constraint, the method evaluates whether the input workload specification satisfies or violates the constraint. If the component model comprises a workload specification transform, the method modifies the input workload specification so as to produce one or more output workload specifications.

Method and apparatus for morphological modeling of complex systems to predict performance

A method of and apparatus for determining whether a multi-component target system meets a given multi-part performability requirement is provided. A description of the target system, failure probabilities for components of the target system and a multi-part performability requirement for the target system are obtained. The multi-part performability requirement indicates desired performance levels and corresponding fractions of time. One or more failure-scenarios are successively computed that represent one or more states of the target system having zero or more components failed and a corresponding probability of occurrence of the one or more of the states of the target system. Performance of the target system is modeled under the failure scenarios using a performance predictor module for generating a multi-part performability function. The multi-part performability function is compared with said multi-part performability requirement to determine whether the target system meets the multi-part performability requirement.

Growing community of inventors

Los Gatos, CA, United States of America

Ralph Attila Becker-Szendy