Western Digital Technologies, Inc.

Shaomin Xiong

Toshiki Hirano

Haoyu Wu

Na Wang

Damien Kah

Erhard Theodor Schreck

William Bernard Boyle

Rajeev Nagabhirava

Ramy Ayad

Qian Zhong

Guoxiao Guo

David Berman

Kei Yasuna

Duc H Banh

Masaru Furukawa

Robert Lee Smith

Sripathi Vangipuram Canchi

Jeff Wilke

Pritam Das

Oscar Jaime Ruiz

Satoshi Yamamoto

Qing Dai

Sukumar Rajauria

Hussam Zebian

Timothy Skarzynski

Dave Niss

Vipin A Vitikkate

Systems and methods described herein provide for the use and storage of intermediate layer data within a neural network processing system. A first neural network, such as an object detection neural network may receive and process raw video image data to generate output utilized for metadata creation. Secondary neural networks may be configured to accept input data from one or more intermediate layers of the first neural network instead of the raw video image data. In this way, the initial data processed by the intermediate layers of the first neural network can be stored and utilized as a shortcut for processing additional features or attributes within the video image data. This alleviates the need to process video image data multiple times in different neural networks. The intermediate layer data can be stored in a different and often cheaper storage system and recalled faster and with fewer resources for future use.

Storage and processing of intermediate features in neural networks

A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture identifies objects according to a first confidence threshold of the model and identifies candidate objects according to the first confidence threshold and a second confidence threshold. The model may track the motion of the candidate objects within a visual field, separate the candidate objects into false positive candidate objects and false negative candidate objects according to their tracked motions, and present at least a portion of the false positive candidate objects and false negative candidate objects for further annotation.

Data collection and user feedback in edge video devices

A digital video camera architecture for updating an object identification and tracking model deployed with the camera is disclosed. The invention comprises optics, a processor, a memory, and an artificial intelligence logic which may further comprise artificial neural networks. The architecture may identify objects according to a confidence threshold of a model. The confidence threshold may be monitored over time, and the model may be updated if the confidence threshold drops below an acceptable level. The data for retraining is ideally generated substantially internal to the camera. A filter is generated to process the entire field data set stored on the camera to create a field data subset also stored on the camera. The field data subset may be run through the model to generate cases that may be used in further monitoring, training, and updating of the model.

Data collection and retraining in edge video devices

Systems and methods for audio privacy in network video surveillance systems are described. A video camera may include an image sensor and a microphone to generate a video stream. Responsive to detecting a human speaking condition in the video stream, the audio data may be selectively modified to mask a human voice component of the audio data for storing and/or displaying the surveillance video stream.

Audio privacy protection for surveillance systems

A data storage device including, in one implementation, a non-volatile memory device having a memory block including a number of memory dies, and a controller coupled to the non-volatile memory device. The controller is configured to monitor a temperature of the data storage device and determine whether the monitored temperature exceeds a first temperature threshold. The controller is also configured to perform a default thermal throttling operation based on the monitored temperature exceeding the first temperature threshold, determine whether the monitored temperature exceeds a second temperature threshold, and perform a customized thermal throttling operation based on the monitored temperature exceeding the second temperature threshold.

Customized thermal throttling using environmental conditions

Systems and methods for site-based estimation of storage requirements, such as for surveillance video cameras, are described. Product information, a data retention policy, and an intended recording mode about a camera of a surveillance system may be received through a user interface of a user device. A baseline storage value is determined based on the received product information and the data retention policy. A storage requirement may be calculated based on the baseline data storage value and a determined recording co-efficient value based on a sample scene at the camera location. Scene descriptors may be generated based on the sample scene to retrieve the recording co-efficient value from a lookup table.

Usage-based assessment for surveillance storage configuration

A surveillance system includes at least one image or video capture device and a controller configured to determine a change in location for the at least one image or video capture device from a first location to a second location. First image or video data is received from the at least one image or video capture device at the second location, and in response to the location change, a preconfigured neural network is obtained or weights for a neural network are obtained based at least in part on the received first image or video data. Second image or video data is received from the at least one image or video capture device at the second location and an inference operation is performed on the second image or video data by processing the second image or video data using the obtained weights or the obtained preconfigured neural network.

Smart video surveillance system using a neural network engine

In one embodiment, a sensor device includes a sensor configured to obtain sensor data and a communication interface configured to communicate with a set of secondary sensor devices. The set of secondary sensor devices comprise a set of secondary sensors and a set of secondary processing devices. The set of secondary sensors obtain additional sensor data. The set of secondary processing devices process the additional sensor data based on a set of secondary machine learning models, and generate activation data based on the additional sensor data. The sensor device also includes a processing device. The processing device is also configured to receive the activation data from the set of secondary sensor devices and cause the sensor to obtain first sensor data based on the activation data. The processing device is further configured to generate one or more inferences based on the first sensor data and a machine learning model.

Sensor system with low power sensor devices and high power sensor devices

Systems and methods for processing surveillance video streams using image classification and object detection are described. Video data from a video image sensor may be processed using an image classifier to determine whether an object type is present in a video frame. If the object type is present, the video frame and/or subsequent video frames may be processed using an object detector to provide additional object data, such as position information, for use in other video surveillance processes. In some examples, an event message may be generated and sent to a video surveillance application in response to selective object detection.

Processing of surveillance video streams using image classification and object detection

Systems and methods for site-based calibration of object detection values, such as for surveillance video cameras, are described. Video data from a video image sensor may be processed using an object detector to determine object data and a confidence score for a detected object. The object data and confidence score may be post-processed to apply calibration values based on the camera location to one or more parameters used for determining detection events. Event notifications may be sent for detection events. The calibration values may be determined from a calibration period where a verification object detector is used to verify the object detections and failure analysis is applied to determine calibration values for the camera location.

Site-based calibration of object detection parameters

This disclosure relates to an apparatus for object detection. The apparatus comprises a video camera, an object detector, and a controller. The video camera may be configured to generate a video stream of frames. The object detector may be configured to accept the video stream as input data and to perform object detection. The controller may be coupled to the video camera and the object detector. The controller may be configured to manage object detection in order to satisfy a performance metric and/or operate within an operational constraint.

Object detection using multiple neural network configurations

The present disclosure generally relates to using a data timestamp and/or a read counter to prevent data from being re-read and signal that the data has been accessed. The write assist element in the write head can be utilized to degrade the data or an allocated marker after the data has been read. The degradation functions as a read counter to indicate how many times the data has been read. Additionally and/or alternatively, a timestamp can be utilized. The timestamp is updated each time that the data has been accessed. In so doing, it is possible to determine whether data in a data storage device has been accessed.

Data timestamp and read counter for magnetic recording devices

Various illustrative aspects are directed to a data storage device, comprising: one or more disks; an actuator assembly comprising a head, and configured to position the head over a corresponding disk surfaces; and one or more processing devices, wherein the head comprises: a write element; a laser unit; and a fly height control element, and wherein the one or more processing devices are configured to: iteratively perform write start operations of write start patterns with the head on the corresponding disk surface, at a plurality of values of at least one of laser pre-bias current, and write backoff; detect pattern signal amplitudes of the write start patterns on the corresponding disk surface; and determine a relation of write backoff to laser pre-bias current for the head, based at least in part on the pattern signal amplitudes.

Determining laser induced thermal protrusion and laser threshold in heat-assisted magnetic recording (HAMR) data storage

Various illustrative aspects are directed to a data storage device, comprising: one or more disks; an actuator assembly comprising a head, and configured to position the head over a corresponding disk surface; and one or more processing devices, the head comprising: a write element; a laser unit; and a fly height control element, and wherein the one or more processing devices are configured to: iteratively perform spiral write operations of spiral patterns comprising a plurality of sync marks with the head on the corresponding disk surface, wherein the spiral write operations are performed at: a plurality of values of laser pre-bias current, write backoff, and/or start disk phase; detect pattern signal amplitudes of the spiral patterns on the corresponding disk surface; and determine a relation of write backoff to laser pre-bias current for the head, based on the pattern signal amplitudes of the spiral patterns.

Method to measure thermal protrusion by sensing spacing change due to pre-lasing

Systems and methods for a low light surveillance system with dual video streams are described. A video camera may include an image sensor and an infrared blocking filter. Responsive to detecting low light conditions, the infrared blocking filter may be disabled and the full spectrum video data from the camera may be sent in one video stream that is processed for user display, such as rendering in gray scale, and another video stream that is used for analytics processing, such as event detection.

Growing community of inventors

Fremont, CA, United States of America

Shaomin Xiong