Systems and methods for providing flexible, multi-capacity models for use of deep neural networks in mobile devices

Abstract

Systems and methods are disclosed which allow mobile devices, and other resource constrained applications, to more efficiently and effectively utilize deep learning neural networks using only (or primarily) local resources. These systems and methods take the dynamics of runtime resources into account to enable resource-aware, multi-tenant on-device deep learning for artificial intelligence functions for use in tasks like mobile vision systems. The multi-capacity framework enables deep learning models to offer flexible resource-accuracy trade-offs and other similar balancing of performance and resources consumed. At runtime, various systems disclosed herein may dynamically select the optimal resource-accuracy trade-off for each deep learning model to fit the model's resource demand to the system's available runtime resources and the needs of the task being performed by the model. In doing so, systems and methods disclosed herein can efficiently utilize the limited resources in mobile systems to maximize performance of multiple concurrently running neural network-based applications.