Method and system for evicting and reloading a cache for machine learning training data streams

Abstract

Techniques described herein relate to a method for managing training data. The method includes monitoring, by a training data stream manager (TDSM), a cache comprising a plurality of training data examples associated with streams of mini-batch sequences scheduled to be transmitted to a machine learning training environment; making a first determination that a cache eviction is required; in response to the first determination: selecting a training data example of the plurality of training data examples; making a second determination that the training data example is eligible for cache eviction; in response to the second determination: evicting the training data example from the cache; and updating a training data example database entry to indicate that the training data example is evicted from the cache.