Training distributed machine learning with selective data transfers

Abstract

Presented herein are techniques for training a central/global machine learning model in a distributed machine learning system. In the data sampling techniques, a subset of the data obtained at the local sites is intelligently selected for transfer to the central site for use in training the central machine learning model. In the model merging techniques, distributed local training occurs in each local site and copies of the local machine learning models are sent to the central site for aggregation of learning by merging of the models. As a result, in accordance with the examples presented herein, a central machine learning model can be trained based on various representations/transformations of data seen at the local machine learning models, including sampled selections of data-label pairs, intermediate representation of training errors, or synthetic data-label pairs generated by models trained at various local sites.