Samsung Electronics Co., Ltd.

Ran Vitek

Ishay Goldin

Alexandra Dana

Netanel Stein

Nathan Henri Levy

Omer Shabtai

Michael Dinerstein

Roy Josef Jevnisek

Tomer Peleg

Yotam Platner

David Tsidkiahu

Yotam Perlitz

Zuf Pilosof

Yaron Ukrainitz

Matan Shoef

Tal Heller

Yonatan Dinai

Maor Shutman

Alon Intrater

One or more aspects of the present disclosure enable high accuracy computer vision and image processing techniques with decreased system resource requirements (e.g., with decreased computational load, shallower neural network designs, etc.). As described in more detail herein, one or more aspects of the described techniques may leverage key layers (e.g., certain key layers of a neural network) and compressed tensor comparisons to efficiently exploit temporal redundancy in videos and other slow changing signals (e.g., to efficiently reduce neural network inference computational burden, with only minor increase in data transfer power consumption). For example, key layers of a neural network may be identified, and temporal/spatial redundancy across frames may be efficiently leveraged such that only a computation region in a subsequent frame n+1 is re-computed in layers between identified key layers, while remaining feature-map calculations may be disabled in the layers between the identified key layers.

Low-power change-based neural network inference for image processing

Techniques and apparatuses enabling high accuracy video object detection using reduced system resource requirements (e.g., reduced computational load, shallower neural network designs, etc.) are described. For example, a search domain of an object detection scheme (e.g., a target object class, a target object size, a target object rotation angle, etc.) may be separated into subdomains (e.g., such as subdomains of object classes, subdomains of object sizes, subdomains object rotation angles, etc.). Specialized, subdomain-level object detection/segmentation tasks may then be separated across sequential video frames. As such, different subdomain-level processing techniques (e.g., via specialized neural networks) may be implemented across different frames of a video sequence. Moreover, redundancy information of consecutive video frames may be leveraged, such that specialized object detection tasks combined with visual object tracking across consecutive frames may enable more efficient (e.g., more accurate, less computationally intensive, etc.) full domain object detection and object segmentation schemes.

Fast object detection in video via scale separation

An object detection system may generate regions of interest (ROIs) from an input image that can be processed by a wide range of object detectors. According to the techniques described herein, an image is processed by a light-weight neural network (e.g., a heatmap network) that outputs object center and object scale heat-maps. The heatmaps are processed to define ROIs that are likely to include objects. Overlapping ROIs are then merged to reduce the aggregate size of the ROIs, and merged ROIs are downscaled to a reduced set of pre-defined resolutions. Fully-convolutional, high-accuracy object detectors may then operate on the downscaled ROIs to output accurate detections at a fraction of the computations by operating on a reduced image. For example, fully-convolutional, high-accuracy object detectors may operate on a subset of the entire image (e.g., cropped images based on ROIs) thus reducing computations otherwise performed over the entire image.

Region of interest selection for object detection

Growing community of inventors

Hasharon, Israel

Ran Vitek