Systems and methods for applying semi-discrete calculus to meta machine learning

Abstract

A method and system for building and implementing a meta-machine learning (meta-ML) optimization engine for a neural network (NN) or a machine learning (ML) connective model. A computer processor may iteratively simulate a backpropagation algorithm by executing a sequence of optimization steps. At each optimization step a position of a loss function may be determined that may be closer than a previously determined position of the loss function to a local minimum. A computer processor may compute and store after each iteration a detachment of the loss function, learning rate, and optimal learning rate. A computer processor may train a machine learning connective model to model the optimal learning rates of the simulated backpropagation algorithm. The meta-ML optimization engine may be implemented for a NN or ML connective model by generating a modified backpropagation algorithm in which algorithmic features of gradient descent may be replaced by the meta-ML optimization engine.