Distributed deep learning system, distributed deep learning method, and computing interconnect device

Abstract

A distributed deep learning system that can achieve speeding-up by processing learning in parallel at a large number of learning nodes connected with a communication network and perform faster cooperative processing among the learning nodes connected through the communication network is provided. The distributed deep learning system includes: a plurality of computing interconnect devicesconnected with each other through a ring communication networkthrough which communication is possible in one direction; and a plurality of learning nodesconnected with the respective computing interconnect devicesin a one-to-one relation, and each computing interconnect deviceexecutes communication packet transmission-reception processing between the learning nodesand All-reduce processing simultaneously in parallel.