International Business Machines Corporation

Guang Lei Li

Jun He

Hua Chai

Rong Li

Chuan Wei Guo

Wei Liu

Qi Chen

Huo Ding Li

Embodiments of the inventive subject matter include receiving, in a distributed computing environment, a plurality of files for execution. Embodiments further include identifying, by parsing the plurality of files, code segments contained in each of the plurality of files. Embodiments further include determining, based on a comparison of the code segments and definitions contained in a distributed computing basic function library, a first group of the code segments that include configuration tasks and a second group of the code segments that include computational tasks. Embodiments further include combining the first group of the code segments to form a super configuration task. Embodiments further include creating an executable code, wherein the executable code comprises the super configuration task and the second group of code segments. Embodiments further include allocating the executable code to one or more nodes. Embodiments further include executing the executable code on the one or more nodes.

Framework to improve parallel job workflow

Specified data is distributed in a High-Performance Computing cluster comprising a Management node and M computation nodes where M is an integer greater than one, by the method comprising: dividing the M computation nodes into m layers where m is an integer greater than one; dividing the specified data into k shares where k is an integer greater than one; distributing, by the Management node, the k shares of data to a first layer of computation nodes as sub-nodes thereof, each node of the first layer obtaining at least one share of data therein; distributing, by each of the computation nodes, the share(s) of data distributed by a parent node thereof to sub-nodes thereof; and requesting, by each of the computation nodes, the remaining specified data to other computation nodes, to thereby obtain all the specified data.

Method and system for data distribution in high performance computing cluster

Growing community of inventors

Beijing, China

Guang Lei Li