Beijing Oneflow Technology Co., Ltd

Xinqi Li

Juncheng Liu

Jinhui Yuan

Cheng Cheng

Yipeng Li

The present disclosure provides a logical node distributed signature decision system for a distributed data processing system, including: an initial logical node generating assembly, configured to receive task configuration data input by a user, and generate an initial logical node topology for the distributed data processing system, wherein a source logical node has a specified logical distributed signature, each initial logical node is attached with a candidate logical distributed signature set based on the task configuration data; and a logical distributed signature selecting assembly, configured to, according to a distributed descriptor of an output end of each upstream logical node for which a logical distributed signature is already determined, for each candidate logical distributed signature of a current logical node, compute a cost of data transmission required to transform the distributed descriptor of the tensor of the output end of each upstream logical node into the distributed descriptor.

Logical node distributed signature decision system and a method thereof

The present disclosure provides a parallel decision system and method for distributed data processing. The system includes: an initial logical node generation assembly, a logical node traversal assembly, a predetermined configuration cost computation assembly, and a parallel decision assembly. The initial logical node generation assembly is configured to receive task configuration data input by a user to generate an initial logical node topology for the distributed data processing system. The logical node traversal assembly is configured to traverse the initial logical node topology to obtain a predetermined configuration in the initial logical node topology. The predetermined configuration cost computation assembly is configured to compute a transmission cost of each predetermined configuration and a cost sum. The predetermined configuration transformation assembly is configured to, based on the result of the predetermined configuration, reduce an initial logical node, and a connection edge, a combined initial logical node.

Growing community of inventors

Beijing, China

Xinqi Li