The patent badge is an abbreviated version of the USPTO patent document. The patent badge does contain a link to the full patent document.
The patent badge is an abbreviated version of the USPTO patent document. The patent badge covers the following: Patent number, Date patent was issued, Date patent was filed, Title of the patent, Applicant, Inventor, Assignee, Attorney firm, Primary examiner, Assistant examiner, CPCs, and Abstract. The patent badge does contain a link to the full patent document (in Adobe Acrobat format, aka pdf). To download or print any patent click here.
Patent No.:
Date of Patent:
Jun. 30, 2026
Filed:
Sep. 20, 2022
Beijing Baidu Netcom Science Technology Co., Ltd., Beijing, CN;
Liang Shen, Beijing, CN;
Haifeng Wang, Beijing, CN;
Huachao Wu, Beijing, CN;
Weibao Gong, Beijing, CN;
Zhihua Wu, Beijing, CN;
Dianhai Yu, Beijing, CN;
BEIJING BAIDU NETCOM SCIENCE TECHNOLOGY CO., LTD., Beijing, CN;
Abstract
The present disclosure provides a mixture-of-experts (MoE) model implementation method and system, an electronic device, and a storage medium, and relates to the field of artificial intelligence (AI) such as deep learning and distributed storage. The method includes: constructing a communication group, the communication group including a tensor-parallelism communication group, the tensor-parallelism communication group including at least two computing devices, tensor-parallelism segmentation being adopted for sparse parameters of each of the computing devices in a same tensor-parallelism communication group; and training an MoE model based on the communication group. By use of the solutions of the present disclosure, normal operation of model training can be guaranteed.