System and method for language-guided video analytics at the edge

Abstract

Training a classifier using embeddings and building a latent space is disclosed. The embeddings may be based on weights in a trained machine learning model. Also, operation of the classifier to process video segments in real-time using the using the weights and the latent space is disclosed. The embeddings and the latent space allow the classification to be performed at an overall reduced dimensionality. The latent space is designed to efficiently scale with an increasing number of queries to permit fast search through the space. Embodiments permit real-time operation on video with dynamic features. The classifier reduces the bandwidth demand of video camera-equipped devices at a network edge by setting aside, accurately, non-informative video sequences rather than uploading video too many things over the network. Applications include security cameras, robots and augmented reality glasses.