Kunlunxin Technology (Beijing) Company Limited

Baidu USA LLC

Baidu.com Times Technology (Beijing) Co. Ltd.

Beijing Baidu Netcom Science and Technology Co., Ltd.

Qingshu Chen

Hefei Zhu

Zhibiao Zhao

Jian Ouyang

Xiaozhang Gong

Yong Wang

Wei Qi

Xianglun Leng

A data processing system includes a central processing unit (CPU) and accelerator cards coupled to the CPU over a bus, each of the accelerator cards having a plurality of data processing (DP) accelerators to receive DP tasks from the CPU and to perform the received DP tasks. At least two of the accelerator cards are coupled to each other via an inter-card connection, and at least two of the DP accelerators are coupled to each other via an inter-chip connection. Each of the inter-card connection and the inter-chip connection is capable of being dynamically activated or deactivated, such that in response to a request received from the CPU, any one of the accelerator cards or any one of the DP accelerators within any one of the accelerator cards can be enabled or disabled to process any one of the DP tasks received from the CPU.

Distributed AI training topology based on flexible cable connection

According to various embodiments, methods and systems are provided to accelerate artificial intelligence (AI) model training with advanced interconnect communication technologies and systematic zero-value compression over a distributed training system. According to an exemplary method, during each iteration of a Scatter-Reduce process performed on a cluster of processors arranged in a logical ring to train a neural network model, a processor receives a compressed data block from a prior processor in the logical ring, performs an operation on the received compressed data block and a compressed data block generated on the processor to obtain a calculated data block, and sends the calculated data block to a following processor in the logical ring. A compressed data block calculated from corresponding data blocks from the processors can be identified on each processor and distributed to each other processor and decompressed therein for use in the AI model training.

Accelerating AI training by an all-reduce process with compression over a distributed system

Methods, apparatuses, devices, and storage media for performing a processing task are provided. A portion of portions of the processing task can include a group of operations that are to be performed at a processing unit of processing units. The group of operations can include operations of a first type and operations of a second type. In the method, a first queue for performing the operations of the first type and a second queue for performing the operations of the second type can be built, respectively. Based on a definition of the processing task, a dependency relationship between a group of operations to be performed at the processing unit and a group of operations to be performed at other processing units in the plurality of processing units can be obtained. Operations in the first queue and operations in the second queue can be performed respectively based on the dependency relationship.

Method, apparatus, device, and storage medium for performing processing task

Embodiments of the present disclosure relate to a method, an apparatus, an electronic device and a computer readable storage medium for determining connection relationships among a plurality of chips. The method includes determining identity information of a plurality of chips managed by a host, the plurality of chips being connected by respective inter-chip communication interfaces for inter-chip communication. The method further includes allowing one or more of the plurality of chips to acquire identity information of other chips connected to the inter-chip communication interface of the one or more chips. The method further includes reading identity information of the other chips by means of a management interface of the one or more chips with regard to communicating with the host, so as to determine connection relationships among the plurality of chips.

Method, apparatus, electronic device and computer readable storage medium for supporting communication among chips

Growing community of inventors

Shanghai, China

Qingshu Chen