International Business Machines Corporation

Changchang Liu

Shiqiang Wang

Wei-Han Lee

Bong Jun Ko

Tiffany Tuor

Dinesh C Verma

Seraphin Bernard Calo

Supriyo Chakraborty

Thai Franck Le

Yuang Jiang

A computer-implemented method, a computer program product, and a computer system for incremental and decentralized pruning of a machine learning model in federated learning. A federated learning system determines a serial sequence of participating in model pruning by agents in the federated learning system. A server in the federated learning system sends, to a first agent in the serial sequence, an initial model to trigger a federated pruning process for the machine learning model. The each of agents in the serial sequence prunes the machine learning model. The each of agents in the serial sequence generates an intermediately pruned model for an immediately next agent to prune. A final agent in the serial sequence prunes the machine learning model and generates a finally pruned model. The final agent sends the finally pruned model to the server.

Incremental and decentralized model pruning in federated machine learning

Techniques for adaptive asynchronous federated learning are described herein. An aspect includes providing a first version of a global parameter to a first client and a second client. Another aspect includes receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client. Another aspect includes determining whether the first version of the global parameter matches a most recent version of the global parameter. Another aspect includes, based on determining that the first version of the global parameter does not match the most recent version of the global parameter, selecting a version of the global parameter. Another aspect includes aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

Adaptive asynchronous federated learning

A computer-implemented method, a computer program product, and a computer system for enhanced distributed machine learning. A fusion server in a distributed machine learning system determines correlation relationships across agents in the distributed machine learning system, based on auxiliary information. The fusion server clusters the agents to form one or more communities, based on the correlation relationships. The fusion server selects, from the one or more communities, participating agents that participate in the enhanced distributed machine learning.

Leveraging correlation across agents for enhanced distributed machine learning

A method, a computer program product, and a computer system determine when to perform a federated learning process. The method includes identifying currently available contributors among contributors of a federated learning task for which the federated learning process is to be performed. The method includes determining a usefulness metric of the currently available contributors for respective datasets from each of the currently available contributors used in performing the federated learning process. The method includes, as a result of the usefulness metric of the currently available contributors being at least a usefulness threshold, generating a recommendation to perform the federated learning process with the datasets of the currently available contributors. The method includes transmitting the recommendation to a processing component configured to perform the federated learning process.

Federated learning of clients

Distributed machine learning employs a central fusion server that coordinates the distributed learning process. Preferably, each of set of learning agents that are typically distributed from one another initially obtains initial parameters for a model from the fusion server. Each agent trains using a dataset local to the agent. The parameters that result from this local training (for a current iteration) are then passed back to the fusion server in a secure manner, and a partial homomorphic encryption scheme is then applied. In particular, the fusion server fuses the parameters from all the agents, and it then shares the results with the agents for a next iteration. In this approach, the model parameters are secured using the encryption scheme, thereby protecting the privacy of the training data, even from the fusion server itself.

Growing community of inventors

White Plains, NY, United States of America

Changchang Liu