International Business Machines Corporation

Shiqiang Wang

Bong Jun Ko

Theodoros Salonidis

Changchang Liu

Dinesh C Verma

Mudhakar Srivatsa

Supriyo Chakraborty

Wei-Han Lee

Tiffany Tuor

Seraphin Bernard Calo

Raghu Kiran Ganti

Ramya Raghavendra

Nirmit V Desai

Georgios Kollias

Petr Novotny

Qi Zhang

Venkatraman Ramakrishna

Thai Franck Le

Christian Makaya

Keith William Grueneberg

Joshua M Rosenkranz

Mandis S Beigi

Irene Lizeth Manotas Gutiérrez

David A Wood

Yuang Jiang

Stacy Elizabeth Patterson

Timothy John Castiglia

Anirban Das

For a plurality of client computing devices of a federated learning system, obtain initial compressed embeddings, compressed by clustering, and including output of initial local models for a current minibatch, and initial cluster labels corresponding to the initial embeddings. Recreate an initial overall embedding based on the initial embeddings and the initial labels. At a server of the federated learning system, send a current version of a server model to each of the client computing devices; and obtain, from the client computing devices: updated compressed embeddings, compressed by clustering, and updated cluster labels corresponding to the updated embeddings. Based on local training by the plurality of clients with the overall embedding and the current server model, at the server, recreate an updated overall embedding based on the updated embeddings and the corresponding updated labels, and locally train the server model based on the updated overall embedding.

Vertical federated learning with compressed embeddings

Tensor comparison across a network by determining a first parameter of a first vector representation of a first tensor object associated with a first processor, receiving a second parameter associated with a second vector representation of a second tensor object associated with a second processor, determining a first difference between the first parameter and the second parameter, and sending the first vector representation from the first processor to the second processor, according to the first difference.

Tensor comparison across a distributed machine learning environment

A computer-implemented method, a computer program product, and a computer system for incremental and decentralized pruning of a machine learning model in federated learning. A federated learning system determines a serial sequence of participating in model pruning by agents in the federated learning system. A server in the federated learning system sends, to a first agent in the serial sequence, an initial model to trigger a federated pruning process for the machine learning model. The each of agents in the serial sequence prunes the machine learning model. The each of agents in the serial sequence generates an intermediately pruned model for an immediately next agent to prune. A final agent in the serial sequence prunes the machine learning model and generates a finally pruned model. The final agent sends the finally pruned model to the server.

Incremental and decentralized model pruning in federated machine learning

A training process of a machine learning model is executed at the edge node for a number of iterations to generate a model parameter based at least in part on a local dataset and a global model parameter. A resource parameter set indicative of resources available at the edge node is estimated. The model parameter and the resource parameter set are sent to a synchronization node. Updates to the global model parameter and the number of iterations are received from the synchronization node based at least in part on the model parameter and the resource parameter set of edge nodes. The training process of the machine learning model is repeated at the edge node to determine an update to the model parameter based at least in part on the local dataset and updates to the global model parameter and the number of iterations from the synchronization node.

Distributed machine learning at edge nodes

An example operation may include one or more of collecting, by a lead peer, state and quality of service (QoS) data from a plurality of peers of a blockchain network, building, by the lead peer, a network graph (NG) based on the state and the QoS data from the plurality of the peers, and mapping, by the lead peer, the NG to a block deliver graph (BDG), wherein edges of the BDG represent a sequence of blocks to be sent from a source peer to a destination peer.

Optimization of delivery of blocks

Using locality sensitive hashing in federated machine learning. A server receives from clients locality sensitive hash (LSH) vectors. In one embodiment, the server groups the clients into clusters, based on the LSH vectors; the server selects a subset of the clients, by choosing at least one client from each of the clusters. In another embodiment, the server finds a subset of the clients, by minimize gradient divergence for the subset of the clients. The server receives from selected clients LSH vectors computed based on parameter vectors of updated models, and based on LSH vectors the server determines whether the updated models are sufficiently different from a model being trained; in response to determining that the updated models are sufficiently different from the model, the server requests the selected clients to send the parameter vectors to the server.

Federated machine learning using locality sensitive hashing

Techniques for adaptive asynchronous federated learning are described herein. An aspect includes providing a first version of a global parameter to a first client and a second client. Another aspect includes receiving, from the first client, a first gradient, wherein the first gradient was computed by the first client based on the first version of the global parameter and a respective first local dataset of the first client. Another aspect includes determining whether the first version of the global parameter matches a most recent version of the global parameter. Another aspect includes, based on determining that the first version of the global parameter does not match the most recent version of the global parameter, selecting a version of the global parameter. Another aspect includes aggregating the first gradient with the selected version of the global parameter to determine an updated version of the global parameter.

Adaptive asynchronous federated learning

Method and apparatus for exchanging corpora via a data broker are provided. One example method generally includes receiving, at the data broker from a holder of a first corpus application, a coreset for the first corpus and transmitting the coreset to a set of data providers. The method further includes receiving, from a first data provider of the set of data providers, a value with respect to the coreset of a second corpus associated with the first data provider and transmitting, from the data broker to the holder of the first corpus, the value. The method further includes receiving, at the data broker from the holder of the first corpus, a request to receive the second corpus and receiving the second corpus from the first data provider. The method further includes validating the value of the second corpus and transmitting the second corpus to the holder of the first corpus.

Determining value of corpora for machine learning using coresets

A model requester node, which is an edge node of a cloud computing network, generates a specification of a machine learning model, distributes the specification to a plurality of other edge nodes, and receives replies to the specification from the plurality of other edge nodes. In response to the replies, the model requester node identifies a set of participating edge nodes based on a learning utility and a cost estimate of each of the plurality of other edge nodes. The model requester node then trains the machine learning model, without exchanging training data among the model requester node and the participating edge nodes, by repeatedly: distributing most recent parameters of the machine learning model to the participating edge nodes; receiving updates to the most recent parameters from the participating edge nodes; and establishing new parameters for the machine learning model by aggregating the updates from the participating edge nodes.

Collaborative distributed machine learning

A computer-implemented method, a computer program product, and a computer system for enhanced distributed machine learning. A fusion server in a distributed machine learning system determines correlation relationships across agents in the distributed machine learning system, based on auxiliary information. The fusion server clusters the agents to form one or more communities, based on the correlation relationships. The fusion server selects, from the one or more communities, participating agents that participate in the enhanced distributed machine learning.

Leveraging correlation across agents for enhanced distributed machine learning

A method, a computer program product, and a computer system determine when to perform a federated learning process. The method includes identifying currently available contributors among contributors of a federated learning task for which the federated learning process is to be performed. The method includes determining a usefulness metric of the currently available contributors for respective datasets from each of the currently available contributors used in performing the federated learning process. The method includes, as a result of the usefulness metric of the currently available contributors being at least a usefulness threshold, generating a recommendation to perform the federated learning process with the datasets of the currently available contributors. The method includes transmitting the recommendation to a processing component configured to perform the federated learning process.

Federated learning of clients

Aspects of the invention include methods of performing beam management in a base station of a cellular network. A method includes obtaining, at the base station, information indicating whether one or more terminals of the cellular network have data to transmit, and determining, using the base station, during a sweep by the base station if an upcoming terminal has data to transmit based on the information. The sweep by the base station is a sequential movement of the beam over a coverage area. The method also includes foregoing, by the base station, any communication with the upcoming terminal during the sweep based on the information indicating that the upcoming terminal has no data to transmit, and communicating, using the base station, with the upcoming terminal during the sweep based on the information indicating that the upcoming terminal has data to transmit.

Base station beam management based on terminal transmit data indication

A computer-implemented method for delivering network connectivity includes receiving, by an edge server, a set of communication packets from a communication device. The method further includes storing, by the edge server, the set of communication packets as part of outbound data. The method further includes determining, by the edge server, that a mobile access point is within a communicable range of the edge server. The mobile access point travels back and forth between the edge server and a base station. The method further includes transmitting, by the edge server, the outbound data to the mobile access point that is within the communicable range.

Cost effective delivery of network connectivity to remote areas

A user input is analyzed during labeling of a first portion of data to determine a behavior of a user. A labeling accuracy rate of the user is forecasted. A characteristic of a second portion of data to be presented to the user at a second time is determined. The second portion of data is presented to the user. A second label corresponding to the second portion of data is received. A second user input is analyzed to determine a second behavior of the user. The accuracy prediction model is refined. A second labeling accuracy rate of the user is forecasted. The recommendation model is refined.

User adapted data presentation for data labeling

A domain description is received, by a processor, the domain description identifying a domain associated with a sensor input. The domain description is formatted according to a hierarchical naming structure. A training data set is selected from a plurality of training data sets based upon the received domain description and sensor input. A combination of a subset of classifiers for classifying the sensor input is selected from a set of classifiers based upon the selected training data set.

Growing community of inventors

White Plains, NY, United States of America

Shiqiang Wang