Qualcomm Incorporated

Amazon Technologies, Inc.

Jinglun Gao

Ying Hai Chen

Lei Wang

Ning Bi

Mashhour Solh

Lelin Zhang

Yingyong Qi

Dashan Gao

Xin Zhong

A system configured to improve human presence detection and/or localization by generating aggregate confidence values. The system may aggregate confidence values corresponding to overlapping regions of interest. The system may perform human presence detection by comparing the aggregate confidence values to a universal threshold, with aggregate confidence values above the universal threshold indicating that human presence is detected. The system may use the aggregate confidence values to generate a heatmap, may identify a strongest cluster of pixels in the heatmap and determine a bounding box surrounding the strongest cluster. To distinguish the strongest cluster from a second strongest cluster of pixels, the system may apply a Gaussian kernel with varying threshold values. The system may store the threshold value at which the strongest cluster separates from the second strongest cluster and may use the threshold value to generate the bounding box.

Presence detection and detection localization

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for performing content-adaptive morphology operations. A first erosion function can be performed on a foreground mask of a video frame, including setting one or more foreground pixels of the frame to one or more background pixels. A temporary foreground mask can be generated based on the first erosion function being performed on the foreground mask. One or more connected components can be generated for the frame by performing connected component analysis to connect one or more neighboring foreground pixels. A complexity of the frame (or of the foreground mask of the frame) can be determined by comparing a number of the one or more connected components to a threshold number. A second erosion function can be performed on the temporary foreground mask when the number of the one or more connected components is higher than the threshold number. The one or more connected components can be output for blob processing when the number of the one or more connected components is lower than the threshold number.

Methods and systems of performing adaptive morphology operations in video analytics

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for maintaining blob trackers for one or more video frames. A blob tracker can be associated with a blob generated for a video frame. The blob includes pixels of at least a portion of one or more foreground objects in the video frame. The blob tracker can be determined to be a first type of tracker or a second type of tracker. A first type of tracker has a first bounding box and a second bounding box with an overlapping ratio greater than an alignment threshold for the first type of tracker. A second type of tracker has an irregular size change or an irregular motion change over a threshold duration. The blob tracker can be removed from the plurality of blob trackers maintained for the one or more video frames when the blob tracker is the first type of tracker or the second type of tracker.

Methods and systems of maintaining object trackers in video analytics

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for performing context-aware object or blob tracker updates (e.g., by updating a motion model of a blob tracker). In some cases, to perform a context-aware blob tracker update, a blob tracker is associated with a first blob. The first blob includes pixels of at least a portion of one or more foreground objects in one or more video frames. A split of the first blob and a second blob in a current video frame can be detected, and a motion model of the blob tracker is reset in response to detecting the split of the first blob and the second blob. In some cases, a motion model of a blob tracker associated with a merged blob is updated to include a predicted location of the blob tracker in a next video frame. The motion model can be updated by using a previously predicted location of blob tracker as the predicted location of the blob tracker in the next video frame in response to the blob tracker being associated with the merged blob. The previously predicted location of the blob tracker can be determined using a blob location of a blob from a previous video frame.

Methods and systems of updating motion models for object trackers in video analytics

A system configured to improve human presence detection and/or localization by generating aggregate confidence values. The system may aggregate confidence values corresponding to overlapping regions of interest. The system may perform human presence detection by comparing the aggregate confidence values to a universal threshold, with aggregate confidence values above the universal threshold indicating that human presence is detected. The system may use the aggregate confidence values to generate a heatmap, may identify a strongest cluster of pixels in the heatmap and determine a bounding box surrounding the strongest cluster. To reduce false positive detections, the system may track the false positive detections as heatmap base data and may remove the heatmap base data from the heatmap. Thus, pixel intensity values corresponding to the false positive detections may be reduced to improve an accuracy of the human presence detection.

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for determining blob size thresholds. Blob sizes of blobs generated for a video frame can be determined. A lower boundary of a category of blob sizes can then be determined that corresponds to a minimum blob size of the video frame. The lower boundary is determined from a plurality of possible blob sizes including the blob sizes of the blobs and one or more other possible blob sizes. One of the possible blob sizes is determined as the lower boundary when one or more lower boundary conditions are met by characteristics of the possible blob size. A blob size threshold for the video frame is assigned as the minimum blob size corresponding to the lower boundary.

Methods and systems of determining a minimum blob size in video analytics

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for determining costs for blob trackers and blobs. A blob can be detected in a video frame. The blob includes pixels of at least a portion of a foreground object. A physical distance between a blob tracker and the blob can be determined. A size ratio between the blob tracker and the blob can also be determined. A cost between the blob tracker and the blob can then be determined using the physical distance and the size ratio. In some cases, a spatial relationship between the blob tracker and the blob is determined, in which case the physical distance can be determined based on the spatial relationship. Blob trackers can be associated with blobs based on the determined costs between the blob trackers and the blobs.

Methods and systems of determining costs for object tracking in video analytics

Techniques and systems are provided for processing video data. For example, techniques and systems are provided for matching a plurality of bounding boxes to a plurality of trackers. In some examples, a first association is performed, in which case one or more of the plurality of bounding boxes are associated with one or more of the plurality of trackers by minimizing distances between the one or more bounding boxes and the one or more trackers. A set of unmatched trackers are identified from the plurality of trackers after the first association. The set of unmatched trackers are not associated with a bounding box from the plurality of bounding boxes during the first association. A second association is then performed, in which case each of the set of unmatched trackers is associated with an associated bounding box from the plurality of bounding boxes that is within a first pre-determined distance. A set of unmatched bounding boxes is identified from the plurality of bounding boxes after the second association. The set of unmatched bounding boxes are not associated with a tracker from the plurality of trackers during the second association. A third association is then performed, in which case each of the set of unmatched bounding boxes is associated with an associated tracker from the plurality of trackers that is within a second pre-determined distance.

Multi-to-multi tracking in video analytics

An electronic device is described. The electronic device includes a processor. The processor is configured to obtain a plurality of images. The processor is also configured to obtain global motion information indicating global motion between at least two of the plurality of images. The processor is further configured to obtain object tracking information indicating motion of a tracked object between the at least two of the plurality of images. The processor is additionally configured to perform automatic zoom based on the global motion information and the object tracking information. Performing automatic zoom produces a zoom region including the tracked object. The processor is configured to determine a motion response speed for the zoom region based on a location of the tracked object within the zoom region.

Growing community of inventors

Milpitas, CA, United States of America

Jinglun Gao