International Business Machines Corporation

Piyush Chaudhary

Ravi Kumar Arimilli

Steven J Martin

Jason E Goscinski

Rama Krishna Govindaraju

John Scott Houston

Fu Chung Chang

Peter H Hochschild

Leonard W Helmer, Jr

Claude Basso

Chulho Kim

Robert Sidney Blackmore

Jean Louis Calvignac

Edward Joseph Seminaro

Hanhong Xue

Donald G Grice

Rajeev Sivaram

Sandeep R Patil

Lakshminarayana Baba Arimilli

Colin Beaton Verrilli

Christoph Raisch

Bernard Charles Drerup

Jody B Joyner

Yu-Cheng Hsu

Sridhar R Muppidi

Jay Robert Herring

Jang-Soo Lee

Kevin J Gildea

Smita J Raut

Jan-Bernd Themann

Shajeer K Mohammed

Deryck X Hong

Patricia E Heywood

Yuqing Zhu

Jason S Houston

Jennifer M Doxtader

A computer-implemented method, according to one embodiment, includes: determining, for each pair of HCI systems where each pair includes a first HCI system coupled to another HCI system, a federation relationship setting that corresponds to each pair. The federation relationship settings are used to control a flow of data, as well as to control a flow of workload scheduling, between the first HCI system and the other HCI systems in the respective pairs. Moreover, determining a federation relationship setting that corresponds to a pair includes: determining whether a risk score which corresponds to the pair is outside a predetermined range. In response to determining that the risk score is outside the predetermined range, a restrictive federation relationship setting is assigned to the pair, and in response to determining that the risk score is not outside the predetermined range, a permissive federation relationship setting is assigned to the pair.

Utilizing federation relationship settings among different systems

A mechanism is provided for managing an input/output device communication request. A first operating system passes a call from a first application intended for an input/output device in a second data processing system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call to determine the second data processing system with which the call is associated. The first host fabric interface initiates a connection to a second host fabric interface in the second data processing system and transfers the call to a second operating system associated with the input/output device in the second data processing system via the connection to the second host fabric interface. The second operating system then processes the call intended for the input/output device without assistance from any application running on the second data processing system.

Management of application to I/O device communication requests between data processing systems

A mechanism is provided for managing a process-to-process inter-cluster communication request. A call from a first application is received in a first operating system in a first data processing system. The first operating system passes the call from the first operating system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call to determine a second data processing system in the plurality of data processing systems with which the call is associated, wherein the call is processed by the first host fabric interface without intervention by the first operating system. The first host fabric interface initiates an inter-cluster connection to a second host fabric interface in the second data processing. The call is then transferred to the second host fabric interface in the second data processing system via the inter-cluster connection.

Management of process-to-process inter-cluster communication requests

Disclosed are a method, a system and a computer program product for automatically allocating and de-allocating resources for jobs executed or processed by one or more supercomputer systems. In one or more embodiments, a supercomputing system can process multiple jobs with respective supercomputing resources. A global resource manager can automatically allocate additional resources to a first job and de-allocate resources from a second job. In one or more embodiments, the global resource manager can provide the de-allocated resources to the first job as additional supercomputing resources. In one or more embodiments, the first job can use the additional supercomputing resources to perform data analysis at a higher resolution, and the additional resources can compensate for an amount of time the higher resolution analysis would take using originally allocated supercomputing resources.

Event-based dynamic resource provisioning

A mechanism is provided for managing a process-to-process communication request. A call is received in an operating system from an application in the data processing system. The operating system passes the call to a host fabric interface controller in the data processing system without processing the call. The host fabric interface controller processes the call using state information associated with the call. The call is processed by the host fabric interface controller without intervention by the operating system.

Management of process-to-process communication requests

A mechanism is provided for managing an application communication request. A first operating system passes a call from a first application in a first data processing system intended for a second application in a second data processing system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface processes the call using state information associated with the call to determine the second data processing system with which the call is associated. The first host fabric interface initiates a connection to a second host fabric interface in the second data processing system and transfers the call to a second operating system in the second data processing system via the connection to the second host fabric interface. The second data processing system then processes the call intended for the second application without assistance from the second application.

Management of application to application communication requests between data processing systems

Mechanisms are provided for processing streaming data at high sustained data rates. These mechanisms receive a plurality of data elements over a plurality of non-sequential communication channels and write the plurality of data elements directly to the file system of the data processing system in an unassembled manner. The mechanisms determining whether to perform a data scrubbing operation or not based on history information indicative of whether data elements in the plurality of data elements are being received in a substantially sequential manner. The mechanisms perform a data scrubbing operation, in response to a determination to perform data scrubbing, to identify any missing data elements in the plurality of data elements written to the file system and assemble the plurality of data elements into a plurality of data streams in response to results of the data scrubbing indicating that there are no missing data elements.

Dynamic monitoring of ability to reassemble streaming data across multiple channels based on history

A mechanism is provided for managing a process-to-process intra-cluster communication request. A call from a first application is received in a first operating system in a first data processing system. The first operating system passes the call from the first operating system to a first host fabric interface controller in the first data processing system without processing the call. The first host fabric interface controller processes the call without intervention by the first operating system to determine a second data processing system in the plurality of data processing systems with which the call is associated. The first host fabric interface controller initiates an intra-cluster connection to a second host fabric interface controller in the second data processing system. The first host fabric interface controller then transfers the call to the second host fabric interface controller in the second data processing system via the intra-cluster connection.

Management of process-to-process intra-cluster communication requests

In a multinode data processing system in which the nodes communicate with one another via communication adapters over a network or switch, the adapters are provided with a dual register mechanism for tracking microcode task status. Upon the issuance of a disruptive command that requires attention from one of the nodes, the task status maintained in one register is copied to the snapshot register. As tasks within the adapter are completed, both registers are updated, thus providing a mechanism for the nodes to determine that all tasks active at the time of the disruptive command have completed. This means that the nodes now have a mechanism for determining, as soon as possible, that all tasks that are active when a disruptive command occurs have completed, thus allowing the data processing node to perform such operations as releasing system memory that is associated with the disruptive command, thus eliminating temporal overhead that can affect performance.

Snapshot interface operations

Mechanisms are provided for processing streaming data at high sustained data rates. These mechanisms receive a plurality of data elements over a plurality of non-sequential communication channels and write the plurality of data elements directly to the file system of the data processing system in an unassembled manner. The mechanisms further perform a data scrubbing operation to determine if there are any missing data elements that are not present in the plurality of data elements written to the file system and assemble the plurality of data elements into a plurality of data streams associated with the plurality of non-sequential communication channels in response to results of the data scrubbing indicating that there are no missing data elements. In addition, the mechanisms release the assembled plurality of data streams for access via the file system.

Reassembling streaming data across multiple packetized communication channels

A synchronization optimized queuing method and device to minimize software/hardware interaction in network interface hardware during an end-of-initiative process, including network adapter queue implementations for network interface hardware for optimized communication in a computer system. An end-of-initiative procedure to ensure that the network interface hardware has received an interrupt enable and to recheck the interrupt queue is unnecessary in the present invention.

Synchronization optimized queuing system

Mechanisms for performing a backend operation in a file system are provided. A backend operation on a portion of the file system is initiated. At least one indirect transition table data structure is created for performing the backend operation. Metadata corresponding to the portion of the file system is linked to the at least one indirect transition table data structure. The backend operation is performed on data in a sub-portion of the portion of the file system and the at least one indirect transition table data structure is updated with pointers to new locations of the data in the sub-portion as transitions of the data are completed. At least one data access operation is performed to the portion of the file system at substantially a same time as performing the backend operation on the data in the sub-portion of the portion of the file system.

Consolidating file system backend operations with access of data

A barrier synchronization register, accessible to the nodes in a distributed data processing system, has portions thereof allotted to threads which are present in multiple groups. The barrier synchronization register portion allotted to a given thread has stored therein, over time, group identifier numbers. In this way the state space of a barrier synchronization register is shared over more than one group of process threads.

Growing community of inventors

Highland, NY, United States of America

Piyush Chaudhary