International Business Machines Corporation

Jeffrey Allen Berger

Matthew J Kalos

Harry M Yudenfriend

Beth Ann Peterson

Susan Kay Candelaria

Laura Michiko Kunioka-Weis

Haakon Roberts

Thomas Majithia

William James Franklin

Brent Cameron Beardsley

Terence Patrick Purcell

You-Chin Gene Fuh

Stanley C Kurtz

Balakrishna Raghavendra Iyer

Hong Min

David Alan Burton

Bruce McNutt

James Z Teng

James Zu-Chia Teng

Sauraj Goswami

David L Levish

Cecilia Carranza Lewis

Michael R Shadduck

Stephen Walter Turnbaugh

W James Kincaid

John G Aschoff

Yue Huang

Kou-Horng A Yang

Provided are a computer program product, system, and method for generating a chain of a plurality of write requests including a commit wait flag and plurality of write requests. The commit wait flag is set to one of an indicated first value or a second value. The commit wait flag is set to the first value to cause a storage server to process the write requests by requiring a current write request being processed to complete before transferring data for a next write request following the current write request. The commit wait flag is set to the second value to cause the storage server to process the write requests by transferring data for the next write request before completing the current write request preceding the next write request. The write request chain is sent to the storage server to apply the write requests to the storage.

Generating a chain of a plurality of write requests

Provided are a computer program product, system, and method for processing a chain of a plurality of write requests including a commit wait flag and plurality of write requests, wherein each write request group includes write transactions directed to the storage. A determination is made as to whether the commit wait flag has a first value or a second value. The write requests are processed by requiring a current write request comprising one of the write requests being processed to complete before beginning to write data for a next write request following the current write request in the write request chain in response to the commit wait flag having the first value. The write requests are processed by processing the next write request before completing the current write request in response to the commit wait flag having the second value.

Processing a chain of a plurality of write requests

A database recovery and index rebuilding method involves reading data pages for a database to be recovered as recovery bases; retrieving all log records from stored post-backup updates and sorting the retrieved log records; as the data pages to be recovered are read, applying the sorted log records to their respective data pages; as the applying completes for individual data pages, extracting and sorting index keys from the individual data pages for which the applying is complete, until all index keys have been extracted from all individual data pages and sorted; on an individual recovered page basis, writing the recovered individual data pages into the database; and when all index keys have been extracted and sorted from all of the recovered individual data pages, rebuilding indexes of the database using the sorted index keys and writing the rebuilt indexes to the non-transitory storage.

Relational database recovery

A database recovery and index rebuilding system involves at least one processor coupled to memory, storage, a database, and sets of image copies and post-backup updates. When database reconstruction is required, the processor will i) access the image copies; ii) access post-backup updates to be re-applied; iii) without regard to 'ii)', read data pages to be recovered into objects; iv) concurrently with 'iii)', retrieve and sort log records from the post-backup updates; v) as the data pages to be recovered are read in &#x201c;iii)&#x201d;, apply the sorted log records; vi) as &#x201c;e)&#x201d; completes and concurrently with &#x201c;v)&#x201d;, extract and sort index keys; vii) concurrently with &#x201c;vi)&#x201d; and on an individual recovered page basis, write the recovered data pages into the database; viii) when all index keys have been extracted and sorted, initiate database index rebuilds; and ix) when rebuilt, store the rebuilt indexes without regard to whether &#x201c;vii)&#x201d; has completed.

A database recovery method involves: a) accessing image copies; b) accessing updates to be re-applied to reconstruct the database; c) reading into objects, from the image copies, data pages to be recovered; d) concurrently with 'c)' retrieving and sorting log records from the updates; e) as the data pages are read into the objects in 'c)', applying the sorted log records, in timestamp order; f) as &#x201c;e)&#x201d; completes for individual pages, and concurrently with &#x201c;e)&#x201d; with respect to pages for which applying is not complete, extracting and sorting index keys, from pages for which the applying is complete, until all index keys have been extracted and sorted; g) concurrently with &#x201c;f)&#x201d;, writing recovered pages into the database; h) initiating a rebuild of database indexes using the extracted and sorted index keys; and i) when the database indexes have been rebuilt, writing them to storage without regard to whether &#x201c;g)&#x201d; completed.

Prefetching data using a piecemeal list prefetching method. This is achieved by various means, including building a plurality of data pages, sorting the plurality of data pages into sequential data pages and a list of non-sequential pages, prefetching the sequential data pages using a first prefetching technique, and prefetching the non-sequential list of data pages using a second prefetching technique.

Piecemeal list prefetch

A method, system, and computer program product are provided for retrieving records into a main memory. A first number of gaps and a first total gap size are received for a list of records from a database subsystem. A determination is made of a first average gap size using the first number of gaps and the first total gap size. A determination is made as to whether the first average gap size is greater than a prestaging threshold value. Responsive to the first average gap size being equal to or less than the prestaging threshold value, a prestaging flag is set for the list of records. Then, the list of records is retrieved into the main memory using prestaging.

System and method for an adaptive list prefetch

Various embodiments of a computer-implemented method, system and computer program product maintain a logical page having a predetermined size. Data is added to an uncompressed area of the logical page. The uncompressed area of the logical page is associated with an uncompressed area of a physical page. The logical page also has a compressed area associated with a compressed area of a physical page. In response to exhausting the uncompressed area, data in the uncompressed area is included in the compressed area. The uncompressed area is adjusted.

Compressibility checking avoidance

Disclosed is a method, system, and program for prestaging data into cache from a storage system in preparation for data transfer operations. A first processing unit communicates data transfer operations to a second processing unit that controls access to the storage system. The first processing unit determines addressable locations in the storage system of data to prestage into cache and generates a data structure capable of indicating contiguous and non-contiguous addressable locations addressable locations in the storage system including the data to prestage into the cache. The first processing unit transmits a prestage command to the second processing unit. The prestage command causes the second processing unit to prestage into cache the data at the addressable locations indicated in the data structure. The first processing unit then requests data at the addressable locations indicated in the data structure. In response, the second processing unit returns the requested data from the cache.

Prestaging data into cache in preparation for data transfer operations

The present invention provides a system and method for dynamic management of buffers by the buffer manager for a hierarchical file system. A system according to the present invention includes a buffer pool which is comprised of both fixed and virtual storage. The buffer pool is logically partitioned into three states, fixed, pageable and released, wherein each of the plurality of buffers in the buffer pool exists in one of these states. The system further comprises the buffer manager, which dynamically manages the buffers in the system by moving them between these three states in a non-disruptive manner based on a comparison between the actual usage of fixed and virtual storage compared with target usage values for the fixed and virtual storage.

System and method for performing dynamic buffer management in an HFS (hierarchical file system DF/SMS)

A method and means for dynamically managing access by data sets to an LRU disciplined DASD cache staging data sets. A threshold utilizes the inverse of slowly varying global hit ratios (ST) taken over a large number of data sets to control the access by individual data sets in cache as measured by their local hit ratios (DSHR). After an initial trial period, a data set is allowed access to the cache only where DSHR&gt;ST. The method and means are self adjusting in that as I/O demand changes, ST also changes in phase such that an increase in aggregate demand for cache results in elimination of data sets whose DSHRs were marginally equal to the old ST and now less than the new ST. Utilization of a DASD Fast Write and non-volatile store (NVS) can be managed similarly if coordinated with the cache and maintenance of both a global WRITE threshold (WR), local data set write hit ratio (DSHRW), and invocation of Fast Write only where DSHRW&gt;WT (write threshold).

Growing community of inventors

San Jose, CA, United States of America

Jeffrey Allen Berger