Distributed processing system and distributed processing method

Abstract

Individual distributed processing nodes packetize distributed data for each weight of a neural network of a learning object in an order of a number of the weight, transmit the distributed data to an aggregation processing node, acquire aggregation data transmitted from the node in order, and update the weight of the neural network. The node acquires the transmitted distributed data, packetizes the aggregation data for which the distributed data of all the distributed processing nodes is aggregated for each weight, and transmits the aggregation data to the individual nodes. The individual nodes monitor an unreceived data amount which is a difference between data amounts of the transmitted distributed data and the acquired aggregation data, and when the unreceived data amount becomes equal to or larger than a threshold Ma, stops transmission of the distributed data until the unreceived data amount becomes equal to or smaller than a threshold Mb (Mb<Ma).