Western Digital Technologies, Inc.

Damien Kah

Toshiki Hirano

Shaomin Xiong

Rajeev Nagabhirava

David Berman

Ramy Ayad

Qian Zhong

Systems and methods described herein provide for the use and storage of intermediate layer data within a neural network processing system. A first neural network, such as an object detection neural network may receive and process raw video image data to generate output utilized for metadata creation. Secondary neural networks may be configured to accept input data from one or more intermediate layers of the first neural network instead of the raw video image data. In this way, the initial data processed by the intermediate layers of the first neural network can be stored and utilized as a shortcut for processing additional features or attributes within the video image data. This alleviates the need to process video image data multiple times in different neural networks. The intermediate layer data can be stored in a different and often cheaper storage system and recalled faster and with fewer resources for future use.

Storage and processing of intermediate features in neural networks

A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture identifies objects according to a first confidence threshold of the model and identifies candidate objects according to the first confidence threshold and a second confidence threshold. The model may track the motion of the candidate objects within a visual field, separate the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, and present at least a portion of the false positive candidate objects and false negative candidate objects for further annotation.

Data collection and user feedback in edge video devices

A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture may identify objects according to a confidence threshold of a model. The confidence threshold may be monitored over time, and the model may be updated if the confidence threshold drops below an acceptable level. The data for retraining is ideally generated substantially internal to the camera. A filter is generated to process the entire field data set stored on the camera to create a field data subset also stored on the camera. The field data subset may be run through the model to generate cases that may be used in further monitoring, training, and updating of the model.

Data collection and retraining in edge video devices

Systems and methods for processing surveillance video streams using image classification and object detection are described. Video data from a video image sensor may be processed using an image classifier to determine whether an object type is present in a video frame. If the object type is present, the video frame and/or subsequent video frames may be processed using an object detector to provide additional object data, such as position information, for use in other video surveillance processes. In some examples, an event message may be generated and sent to a video surveillance application in response to selective object detection.

Processing of surveillance video streams using image classification and object detection

Systems and methods for site-based calibration of object detection values, such as for surveillance video cameras, are described. Video data from a video image sensor may be processed using an object detector to determine object data and a confidence score for a detected object. The object data and confidence score may be post-processed to apply calibration values based on the camera location to one or more parameters used for determining detection events. Event notifications may be sent for detection events. The calibration values may be determined from a calibration period where a verification object detector is used to verify the object detections and failure analysis is applied to determine calibration values for the camera location.

Growing community of inventors

San Jose, CA, United States of America

Damien Kah