Systems and methods for deep neural networks on device learning (online and offline) with and without supervision

Abstract

An artificial neural network (ANN) that learns at the Edge (e.g., on a smart phone) can be faster and use less network bandwidth than an ANN trained on a server and distributed to the Edge. Learning at the compute edge can be accomplished by executing Lifelong Deep Neural Network (L-DNN) technology at the compute edge. L-DNN technology uses a representation-rich, DNN-based subsystem with a fast-learning subsystem to learn new features quickly without forgetting previously learned features. Compared to a conventional DNN, L-DNN uses much less data to build robust networks, has dramatically shorter training time, and learns on-device instead of on servers without re-training or storing data. An edge device with L-DNN can learn continuously after deployment, eliminating costs in data collection and annotation, memory, and compute power. This fast, local, on-device learning can be used in unsupervised mode to make personal assistants more intelligent and enhance frequently used apps.