International Business Machines Corporation

Zvi Yosef Yehudai

Frank Bernhard Schmuck

James Christopher Wyllie

Roger Lee Haskin

Elliot Karl Kolodner

Alain Charles Azagury

Robert J Curran

John Michael Marberg

Mila Keren

Irit Loy

Erez Petrank

Boaz Shumeli

Lev Kozakov

Benjamin Mandler

Richard Kirchhofer

Yuval Ishai

Shmuel Gal

Paul K-w Lee

Christopher Thomas Kello

Boaz Shmueli

William A Kish

Dafna Sheinwald

Anthony J Zlotek

Sibylle Schaller

John M Marberg

Alan Hartmann

Igal Galperin

In a cluster of computing nodes having shared access to one or more file systems in data storage using parallel file system software, a method for managing the data storage includes initiating a session of a data management application on a first one of the nodes, while running a user application on a second one of the nodes. A request is submitted to the parallel file system software by the user application on the second node to mount one of the file systems in the data storage. A mount event message is sent from the second node to the first node responsive to the request, for processing by the data management application on the first node. When the file system is to be unmounted, preunmount and unmount events are sent for processing by the data management application on the first node.

Data management application programming interface handling mount on multiple nodes in a parallel file system

In a cluster of computing nodes having shared access to one or more volumes of data storage using a parallel file system, a method for managing the data storage includes initiating a session of a data management application on a session node selected from among the nodes in the cluster. The session node receives an event message in a session queue for processing by the data management application, responsive to a request submitted to the parallel file system by a source node among the nodes in the cluster to perform a file operation on a file in the data storage. Following a failure at the session node, the session queue is reconstructed so that processing of the event message by the data management application can continue after recovery from the failure, and the request can be fulfilled at the source node.

Data management application programming interface failure recovery in a parallel file system

In a cluster of computing nodes having shared access to one or more volumes of data storage using a parallel file system, a method for managing the data storage includes initiating a session of a data management application on a first one of the nodes, while running a user application on a second one of the nodes. A request is submitted to the parallel file system by the user application on the second node to perform a file operation on a file in one of the volumes of data storage. A data management event message is sent from the second node to the first node responsive to the request, for processing by the data management application on the first node.

Data management application programming interface for a parallel file system

A graphical user interface permits a user to graphically designate a range within data files as input to operations that manipulate the data and permits the user to graphically represent operations that thereafter can be selected for execution. In this way, a user can graphically designate a range of records within a data file on which a desired data manipulation operation will occur and can graphically represent the desired data manipulation operation with a single icon that can be selected for execution.

Object oriented data arranger graphical user interface

A system for garbage collection of memory objects in a memory heap. The system includes a memory heap that is partitioned into few generations, each generation being partitioned into cards. The generations are associated with remembered sets and card markings data structure. Each card in the card markings data structure includes scan generation related data that represent generations for which the card has or has not to be scanned. The system further includes a processor communicating with the memory for scanning the cards according to the scan generation related data, and in response to identifying updated inter-generational pointers, the processor is capable of updating the remember set with the identified inter-generational pointers. The processor is further capable of updating the scan generation related data.

Method for combining card marking with remembered set for generational garbage collection with more than two generations

A system for garbage collection of memory objects in a memory heap. The system includes memory heap that is partitioned into respective old and young areas. The old area is partitioned into cars and is further associated with card markings and remembered sets data structures. The card markings include for each card, a card time stamp that represents the time that the card was updated. The car includes, for each car, a car time entry stamp that represents the time the remembered set of the car was updated. The system further includes a processor communicating with the memory, and being capable of identifying all cards that were updated later than the remembered set of a selected car. In response to the event, it performs identifying change in pointers that refer from the card to a memory object in the selected car and in response to identified change in pointers, updating the remembered set of the car with the identified pointers. The process is further capable of updating the car time stamp of the selected car.

Method for combining card marking with remembered sets for old area of a

In a computer application having a counter for setting first and second counter values in respect of first and second objects. The counter has a range of n possible values and being incremented or decremented in accordance with update criterion. The application taking an action according to whether the first counter value is larger or equal to the second counter value. A method for handling a courter overflow event whereby the counter is incremented beyond said range by executing the steps of: compacting the first and second counter values to m values (m&lt;n) so as to constitute first and second compacted counter values, such that if the first counter value id larger than or equal to the second counter value then the compacted first value is not smaller than the compacted second value, and setting the counter to a value from among n-m values within the specified range.

Method for handling overflow of counters in comparison based actions

A method is described for operating a data processing system having a plurality of processors to sort a set of data records each having an associated key for governing the sort process. The method comprises determining a range for the key values by sampling the key values. The range is divided into a plurality of quantiles, one for each processor, each quantile having a respective index. At each processor, a plurality of buckets are defined, each bucket corresponding to a respective one of a plurality M.sub.p of subintervals in the quantile, each subinterval having a respective index. The index of the quantile in which the key value lies and the index of the subinterval in which the key value lies are determined directly from the key values using fast operations. Each key is distributed to the processor corresponding to the quantile in which the key value lies. At each processor, the keys falling in the quantile corresponding to the processor are distributed into the buckets according to the indices of the subintervals in which the key values lie, the buckets being processed in sequence in order to sort the records and the keys in each bucket sorted if the bucket contains more than one key. Finally, the sorted keys from each processor are concatenated.

Data sorting

A computer system having a shared disk file system running on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. A metadata node manages file metadata for parallel read and write actions. Metadata tokens are used for controlled access to the metadata and initial selection and changing of the metadata node.

Parallel file system with method using tokens for locking modes

A computer system having a shared disk file system running on on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory service for the system with a stable cursor, a segmented allocation map. Dynamic prefetch and cached balance pools for multiple accesses improve the system. Extended file attributes are used for implementation of Access Control Lists in a parallel file system. Improvement to caching and cache performance development balance pools for multiple accesses. A metadata node manages file metadata for parallel read and write actions. For our system, tokens are used for metadata node selection and identification, and we have enhanced token modes for controlling file size, as well as smart caching of byte range tokens using file access patterns and a byte range lock algorithm using a byte range token interface. Locking techniques reduce the overhead of a token manager which is also used in the file system recovery if a computer participating in the management of shared disks becomes unavailable or failed.

Parallel file system and method with allocation map

A computer system having a shared disk file system running on on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory service for the system with a stable cursor, a segmented allocation map. Synchronous and asynchronous takeover of a metadata node occurs for correction of metadata which was under modification and a new computer node to be a metadata node for that file. Locks are not constantly required to allocate new blocks on behalf of a user. We have provided recoverable local shares for Quota Management. A quota server and a quota client are provided for each file system, and a quotacheck utility allows parallel environments with minimal interference to other applications which require the use of quotas.

Parallel file system and method with quota allocation

A computer system having a shared disk file system running on on multiple computers each having their own instance of an operating system and being coupled for parallel data sharing access to files residing on network attached shared disks. Methods are provided for use as a parallel file system in a shared disk environment by use of a scalable directory service for the system with a stable cursor, a segmented allocation map. Dynamic prefetch and cached balance pools for multiple accesses improve the system. Extended file attributes are used for implementation of Access Control Lists in a parallel file system. Improvements to caching and cache performance developments balance pools for multiple accesses. A metadata node manages file metadata for parallel read and write actions. For our system, tokens are used for metadata node selection and identification, and we have enhanced token modes for controlling file size, as well as smart caching of byte range tokens using file access patterns and a byte range lock algorithm using a byte range token interface. Locking techniques reduce the overhead of a token manager which is also used in the file system recovery if a computer participating in the management of shared disks becomes unavailable or failed.

Parallel file system and method anticipating cache usage patterns

A method is described for operating a computer to sort a set of data records each having an associated key for governing the sort process, the method comprising determining a range for the key values by sampling the key values; defining a plurality of buckets, each bucket corresponding to a respective one of a plurality M of subintervals in the range, and two edge buckets for key values outside the range, each subinterval having a respective index; distributing the keys among the buckets by determining directly from each key value the index of the subinterval into which the key value falls; and processing the buckets in sequence in order to sort the records, sorting the keys in each bucket if the bucket contains more than one key.

System and method for sorting data in a computer system using edge

A graphical user interface (GUI) application program implemented in an object oriented programming language permits a user to instantiate new program objects by dragging an instantiation object from one GUI window to a new GUI window and dropping the instantiation object in the new window. When the instantiation object is dropped into the new window, an object specified by the new window type is automatically instantiated. If needed, the user is automatically prompted to provide parameters needed for the object instantiation according to the window in which the object is dropped.

Dragging and dropping with an instantiation object

A system for storing sortwork data in a storage subsystem to maximize parallelism during subsequent merging operations. One implementation of the invention employs a computing system with a sort/merge application module and a storage subsystem such as a RAID arrangement. Each DASD unit preferably includes multiple disks in a vertically-stacked array, and each disk is configured with at least one track. Each disk corresponds to one disk of each of the other DASD units. A group of identically-sized locations from identically-positioned tracks of corresponding disks from each DASD unit forms. And, the aggregate of all identically-positioned tracks from all disks in all DASD units makes up a logical array cylinder. To perform sorting and merging operations in accordance with the invention, the sort/merge application module first internally sorts a data packet to generate a string. The string is then stored in the storage subsystem, with a particular storage scheme that maximizes the parallelism of subsequent merge operations. Specifically, the string is stored such that, after all strings have been stored, each string will be evenly striped across disks of all DASD units after all data has been sorted and stored in the disk storage subsystem. Striping is most advantageously made in a diagonal pattern with wrap-around. Moreover, string size permitting, striping is preferably confined to a single logical array cylinder to minimize actuator movement in reading and writing to the disks. Storage is accomplished such that each logical block contains one string-block from each string, and each string-block within a logical block has the same order relative to its string as the other string-blocks to their corresponding strings.

Growing community of inventors

Haifa, Israel

Zvi Yosef Yehudai