International Business Machines Corporation

Kevin F Smith

Aare Onton

Juan Li

Dung Kim Nguyen

Hai-Fang Yun

Jeffrey Mark Nick

David Arlen Elko

Jaishankar Moothedath Menon

Jeffrey Alan Frey

Michael D Swanson

Bruce McNutt

Brian B Moore

Gary M King

Kelly Scott Carpenter

Chia-Hon Chien

Thomas R Engelmann

A shared system memory, such as a cache, buffers Input/Output (I/O) requests between one or more host computers and one or more data storage servers or devices. The cache may be configured to operate natively as a least-recently-used (LRU)-only cache and may be optimized for random data accesses. Data buffered by the cache may be part of a sequential data stream for which prefetching data is desirable. A remote prefetch module is provided between the cache and the host to conduct prefetching without internally modifying the cache. The remote prefetch module maintains a model of the cache. Using the model, the prefetch module anticipates whether data is likely to be part of a sequential steam of data passed between a host and a data storage device. If so, the prefetch module schedules a prefetch of the data. The prefetch may be achieved by sending an I/O request to the data server or device. The remote cache model minimizes impacts to random access data hits by minimizing the likelihood of prefetching data which is not used and further enhances the efficiency of the successful identification of likely prefetch candidates.

Prescheduling sequential data prefetches in a preexisting LRU cache

In a storage system comprising an array of storage devices, including a processor and memory, a plurality of logical tracks are organized on the storage devices in segments comprising columns striped across the storage devices. A system and method are provided for storing logical tracks in the storage devices. Sequentially logically related logical tracks are stored together in neighborhoods. Logical tracks of the same neighborhood destaged at the same time are assigned to a single open segment with other logical tracks of the same neighborhood type. The time at which open segments are designated as closed segments to be written to the storage devices is based on performance, disk utilization and memory utilization criteria. Logical tracks are never split between segment columns. Also, attempts are made not to split a neighborhood of logical tracks being destaged together between segment columns.

System and method for organizing data stored in a log structured array

In a storage system comprising an array of storage devices, including a processor and memory, a plurality of logical tracks are organized on the storage devices in segments comprising columns striped across the storage devices. A system and method are provided for storing logical tracks in the storage devices. Sequentially logically related logical tracks are stored together in neighborhoods. Logical tracks of the same neighborhood A0 destaged at the same time are assigned to a single open segment with other logical tracks of the same neighborhood type. The time at which open segments are designated as closed segments to be written to the storage devices is based on performance, disk utilization and memory utilization criteria. Logical tracks are never split between segment columns. Also, attempts are made not to split a neighborhood of logical tracks being destaged together between segment columns.

A data processing system dynamically balances allocation of storage areas in a shared coupling facility that is devoted to storage of directory entries and data blocks. Each directory entry includes information regarding the validity of a data block that is locally stored by one or more processor modules in the data processing system. The system includes a coupling facility having a cache memory wherein a first portion is allocated to storage of data blocks and a second portion is allocated to storage of directory entries. Each directory entry, associated with a data block, indicates the validity or invalidity of data contained in a copy of the data block maintained by a connected computer module in its local memory. Each computer module, upon requiring a first data block and determining that (i) the first data block is present in its local memory (i.e., a buffer 'hit'), but (ii) is not marked valid and (iii) is not present in the coupling facility (i.e., a cache 'miss'), accesses the first data block from a disk store, even though the first data block in its local memory may be valid. A memory allocation procedure controls the sizes of the first portion and second portion of the cache memory. The procedure (i) maintains a record, per unit of time, of a number of cache 'miss' occurrences, (i.e., a cache 'miss' rate) and an estimate of the number of false invalidation occurrences (i.e., a false invalidation rate); derives a first order derivative of the sum of cache miss-rate and the estimated false invalidation rate and (iii) employs the derivative to adjust the sizes of the first portion and second portion of the cache memory so as to minimize the sum of the expected miss and false invalidation rate values.

Method and apparatus for determining a desirable directory/data block

A system and method are provided for managing persistent and non-persistent data in a log structured disk array. In a log structured array, data are first collected in a log stored in a buffer. When the buffer is full, the data are written to a disk drive. Persistent data are continually accessed over a long period of time such that, most disk activity is associated with persistent data. Preferably, the persistent data are stored in the part of the disk where access times are minimal (active region), such as in the middle cylinders of a disk. When writing data (including during garbage collection procedure) persistent data is written to the active region and non-persistent data is written to the other regions (inactive region). The persistence of the data is monitored. Persistent data is migrated to the active region and non-persistent is migrated to the inactive region.

System and method for management of persistent data in a log-structured

Methods and systems are disclosed which utilize Little's law to improve cache management in a computer based storage system in which the cache is a partitioned or non partitioned write-back cache. In a first aspect, a method and system for managing a write-back cache in a computer based system to accommodate for deferred write operations is disclosed. The method .and system comprises determining an average cast out delay for a cache entry. In a second aspect the method and system comprises utilizing the average castout delay for estimating the number of dirty cache entries which will be needed by the cache if the write operations rate and the average cast out delay does not change significantly. In a third aspect, a method and system for managing a partitioned and prioritized cache in a computer based system to provide sensitivity to a local maximum is disclosed. The method and system comprises determining an average residency time for a cache entry in the cache and then determining the minimum number of clean pages which should be made available to a particular cache partition based upon that average residency time. In yet another aspect the above aspects can be combined to estimate the minimum requirements on the cache to ensure that a cache management system operates in an efficient manner.

Method and system for estimating minumun requirements on a cache in a

Apparatus and method for operating a data processing system (10) having a data processor (12), a disk data storage subsystem (14), and a disk manager (16). The method includes the steps of (a) partitioning the disk data storage system into multiple partitions including a first partition (28) and a second partition (30); (b) managing at least the first partition as a log-structured file system (LSFS) for storing segments comprised of active data units, for example logical tracks, each having, when stored, an access activity value that exceeds a first predetermined threshold; and (c) storing, within the second partition, segments comprised of inactive data units each having an access activity value, when stored, that is less than the first predetermined threshold.

Partitioned log-structured file system and methods for operating the same

Storage is managed in a shared electronic store (SES) by assigning storage classes (STCs) to each directory entry having a data item stored in SES. The assignments of directory entries and data elements to the respective STCs can be changed at any time by any CPC. Eventually, no free space remains in the SES cache, and then space for new directory entries and data items must be obtained by reclaiming space occupied by directory entries and associated unchanged data items. The reclaiming of SES space is controlled on a STC basis. Any specified STC may reclaim from itself or from another STC using reclaiming software/microcode in SES, which includes a reclaim vector, a reclaim counter, a queue, and reclaiming controls. The vector and counter have respective elements for all possible STCs to controls how a specified STC may reclaim space from any or all target STC. Any enabled target STC reclaims its space according to an LRU algorithm maintained by a queue for the STC. A CPC can issue a command to load different values in target STC elements in the SES vector for a specified STC to control how reclaiming is apportioned for the specified STC. In SES, associated target counter elements are loaded from its vector. Reclaiming is done automatically in SES upon each directory miss in the current target STC having a non-zero counter value, when no free space exists. The counter is decremented for each reclaimed directory entry until its count reaches zero, and then the counter element for the next non-zero target STC is used for reclaiming until it reaches zero. When all STC elements in the counter are zero for the specified STC, the counter elements are reloaded from the vector elements to repeat the reclaiming operation, as long as a repeat factor for the specified STC has not reached zero. The repeat factor is decremented each time the counters are loaded from the vector. When the counters and repeat factor have all reached zero, reclaiming is disabled for the specified STC.

Software cache management of a shared electronic store in a supplex

Methods for managing a Least Recently Used (LRU) cache in a staged storage system on a prioritized basis permitting management of data using multiple cache priorities assigned at the data set level. One method uses the signed difference between an assigned and actual hit ratio to dynamically allocate cache size to a class of data and its correlation with priority of that class. Advantageously, the hit ratio performance of a class of data can not be degraded beyond a predetermined level by a class of data having lower priority. Another method dynamically reallocates cache space among partitions asymptotically to the general equality of a weighted hit rate slope function attributable to each partition. This function is the product of the slope of a weighting factor and the partition hit rate versus partition space allocation function. Cache space is dynamically reallocated among the partitions in a direction that forces the weighted hit rate slope for each partition into equality over the entire cache. Partition space is left substantially unchanged in those partitions where the weighted hit rate slope is non-positive. Hit rate is defined as hit ratio times I/O rate.

Growing community of inventors

San Jose, CA, United States of America

Kevin F Smith