Machine learning and optimization techniques for job scheduling

Abstract

A method includes training a recurrent neural network by monitoring data in a memory of a first server as the first server executes jobs and by determining an amount of computing resources used by the first server while executing the jobs and applying the recurrent neural network to data in the memory to predict an amount of computing resources that the first server will use when executing a first future job. The method also includes, in response to determining that execution of the first future job did not meet a performance criterion, making a change to the first server. The method further includes further training the recurrent neural network using a reinforcement learning technique, applying the recurrent neural network to determine that the change should be made to a second server, and in response, making the change to the second server before the second server executes a second future job.