Beijing Baidu Netcom Science and Technology Co., Ltd.

Beijing Baidu Netcom Science Technology Co., Ltd

Beijing Bytedance Network Technology Co., Ltd.

Zhihua Wu

Dianhai Yu

Wenhui Zhang

Haifeng Wang

Yanjun Ma

Tian Wu

Xin Huang

Daxiang Dong

Shentao Wang

Weibao Gong

Jiaqi Fan

Yulong Ao

The present disclosure provides a method and apparatus for performing distributed training on a deep learning model. The method may include: generating a distributed computation view based on data information of a to-be-trained deep learning model; generating a cluster resource view based on property information of a cluster hardware resource corresponding to the to-be-trained deep learning model; determining a target segmentation strategy of a distributed training task based on the distributed computation view and the cluster resource view; and performing distributed training on the to-be-trained deep learning model based on the target segmentation strategy.

Method and apparatus for performing distributed training on deep learning model, device and storage medium

The invention discloses a method and an apparatus for updating parameters of a multi-task model. The method includes: obtaining a training sample set, in which the training sample set comprises a plurality of samples and a task to which each sample belongs; putting each sample into a corresponding sample queue sequentially according to the task to which each sample belongs; training a shared network layer in the multi-task model and a target sub-network layer of tasks associated with the sample queue with samples in the sample queue in case that the number of the samples in the sample queue reaches a training data requirement, so as to generate a model parameter update gradient corresponding to the tasks associated with the sample queue; and updating parameters of the shared network layer and the target sub-network layer in a parameter server according to the model parameter update gradient.

Method and apparatus for updating parameter of multi-task model, and storage medium

A method and an apparatus for generating a shared encoder are provided, which belongs to a field of computer technology and deep learning. The method includes: sending by a master node a shared encoder training instruction to child nodes, so that each child node obtains training samples based on a type of a target shared encoder included in the training instruction; sending an initial parameter set of the target shared encoder to be trained to each child node after obtaining a confirmation message returned by each child node; obtaining an updated parameter set of the target shared encoder returned by each child node; determining a target parameter set corresponding to the target shared encoder based on a first preset rule and the updated parameter set of the target shared encoder returned by each child node.

Method and apparatus for generating shared encoder

Growing community of inventors

Beijing, China

Zhihua Wu