International Business Machines Corporation

Enci Zhong

Frank Bernhard Schmuck

Felipe Knop

Owen Theodore Anderson

Deepavali M Bhagwat

John Lewars

D Scott Guthridge

Alexander Druyan

Richard J Coppinger

Hai Zhong Zhou

Abhishek Jain

David Paul Brelsford

Huzefa Pancha

Computer technology for use with a computer data storage system including a plurality of client computers and a plurality of collector nodes. A file consists of data blocks with each block resides on one of the persistent storage device. Data received at client computers are buffered and then transferred to selected collector nodes depending on which data blocks the data belong to. Location-aware collector node selection is performed by selecting collector nodes, from the plurality of collector nodes, by querying a file system allocation subsystem to determine selected collector nodes for the given data block that owns a persistent storage device on which the given block is allocated. Data for each data block is then sent to and stored in its selected collector nodes.

Write sharing method for a cluster filesystem

Provided are a computer program product, system, and method for switching serialization techniques for handling concurrent write requests to a shared file. A first node serializes write requests from client nodes to write to the shared file. The first node determines whether to switch to a second node to manage write quests to the shared file based on a pattern of write requests to the shared file. The client nodes are notified to direct write requests to the shared file to the second node in response to determining to switch to the second node. The second node processes write requests to the shared file to serialize writes to the shared file after the client nodes are notified to submit the write requests to the shared file to the second node.

Switching serialization techniques for handling concurrent write requests to a shared file

A method for executing operation requests includes formatting one or more storage devices and selecting one or more labeled replicas and one or more distinguished replicas, receiving an operation request with respect to a set of data blocks, identifying a preferred replica corresponding to the received operation request, determining whether the replication-pending bits for the preferred replica are set, and executing the received operation request with respect to the corresponding distinguished replica. A method for executing a write operation request additionally includes setting replication-pending bits with respect to the labeled replica, wherein the replication-pending bits indicate an incomplete write request, writing data to both a labeled replica and a distinguished replica, and clearing the replication-pending bits with respect to the labeled replica to indicate the completion of the write operation. A method for executing a read operation additionally includes identifying a preferred replica with respect to the received read request.

Interrupted replicated write recognition

A method and a computer system for asymmetric replication of data are provided. Storage of a set of data is organized as a first copy and as a second copy in non-volatile storage. The second copy is reliable and stored so as to be readable at a speed slower than for the first copy. A read instruction regarding the set of data is received and performed preferentially via the first copy such that the asymmetric replication achieves enhanced performance speed. A request to execute a write operation is received. The write operation is executed to the first copy and to the second copy. In response to determining that the write operation to the first copy was unsuccessful, a label for the first copy is set as stale.

Efficient use of optional fast replicas via asymmetric replication

A technique for determining scheduling priorities of jobs of different types to facilitate workload management in a clustered processing environment having a plurality of managed resource types is provided. The technique includes allocating shares of multiple managed resource types to users and user groups of the clustered processing environment. The technique also includes assigning a job priority for a subsequent job submitted by a user or group, wherein the calculated job priority is based on a type attribute of the subsequent job and the number of the user's or group's remaining shares of at least one managed resource type of the multiple of managed resource types.

Fair share scheduling for mixed clusters with multiple resources

A method, information processing system, and computer program storage product for associating jobs with resource subsets in a job scheduler. At least one job class that defines characteristics associated with a type of job is received. A list of resource identifiers for a set of resources associated with the job class is received. A set of resources available on at least one information processing system is received. The resource identifiers are compared with each resource in the set of resources available on the information processing system. A job associated with the job class with is scheduled with a set of resources determined to be usable by the job based on the comparing.

Association and scheduling of jobs using job classes and resource subsets

Fair share scheduling to divide the total amount of available resource into a finite number of shares and allocate a portion of the shares to an individual user or group of users as a way to specify the resource proportion entitled by the user or group of users. The scheduling priority of jobs for a user or group of users depends on a customizable expression of allocated and used shares by that individual user or group of users. The usage by the user or group of users is accumulated and an exponential decay function is applied thereto in order to keep track of historic resource usage for a user or group of users by one piece of data and an update timestamp.

Fair share scheduling based on an individual user's resource usage and the tracking of that usage

Disclosed are a method, information processing system, and computer readable medium for reserving resources in a massively parallel processing system comprising. The method includes receiving a reservation request for reserving at least one resource within a massively parallel processing system. The massively parallel processing system includes compute nodes, base partitions, switches, wires, and node cards. The reservation type associated with the reservation is determined. The reservation type is at least one of a number of compute nodes, a base partition list, a particular shape configuration, a particular job, and a particular partition. A set of requirements is generated based on the reservation type. A set of resources is identified for satisfying the set of requirements and the set of resources is reserved.

Resource reservation for massively parallel processing systems

A fault tolerant capability for the reliable collection and aggregation of disparate data from multiple processing units. Data of a processing unit is collected and locally aggregated at that processing unit. The locally aggregated data is stored in a local database, and periodically forwarded to a central manager. In response to receiving the data, the central manager determines whether the data should be included in a total aggregate of data for the processing environment.

Fault tolerant facility for the aggregation of data from multiple processing units

Methods and systems are disclosed for determining whether circular dependency exists in a plurality of dependency rules. The method includes generating for at least some of the plurality of dependency rules a plurality of exclusion lists. Each exclusion list is associated with at least one independent object of a dependency rule and lists as an item a dependent object of the dependency rule. Where an item listed in at least one of the generated exclusion lists corresponds to an independent object having its own exclusion list with at least one item, the at least one item is incorporated into the generated exclusion list. Where at least one of the generated exclusion lists for an independent object includes an item corresponding to the independent object, circular dependency is found. If not, and if all of the dependency rules have been read, circular dependency is not found.

Growing community of inventors

Poughkeepsie, NY, United States of America

Enci Zhong