System and methods for distributed machine learning with multiple data sources, multiple programming languages or frameworks, and multiple devices or infrastructures

Abstract

Methods and systems are presented for consuming different data sources, and deploying artificial intelligence and machine learning programs on different target devices or infrastructures. Many data types can be transformed into machine learning data shards (MLDS) while many machine learning programs written in various programming languages or frameworks are transformed to common operator representations. Operator representations are transformed into execution graphs (EG) for a chosen target device or infrastructure. The MLDS and EG are input to the targeted devices and infrastructures, which then execute the machine learning programs (now transformed to EGs) on the MLDS to produce trained models or predictions with trained models.