Samsung Electronics Co., Ltd.

Nathan Henri Levy

Lior Zamir

Ariel Orfaig

Sebastien Derhy

Yoel Yaffe

Yonathan Aflalo

Shachaf Amir

Omer Shabtai

Uzi Hizi

Ishay Goldin

Mickey Bahar

Alexandra Dana

Ran Vitek

Netanel Stein

Yotam Platner

Tal Heller

Alon Intrater

Yaron Ukrainitz

David Tsidkiahu

Itai Lang

Zuf Pilosof

Ayal Keisar

Sagi Tzur

Elad Granot

Dor Barber

Yuri Friedman

Nir Dagani

A method for simultaneous localization and mapping (SLAM) employs dual event-based cameras. Event streams from the cameras are processed by an image processing system to stereoscopically detect surface points in an environment, dynamically compute pose of a camera as it moves, and concurrently update a map of the environment. A gradient descent based optimization may be utilized to update the pose for each event or for each small batch of events.

Simultaneous location and mapping (SLAM) using dual event cameras

Techniques and apparatuses enabling high accuracy video object detection using reduced system resource requirements (e.g., reduced computational load, shallower neural network designs, etc.) are described. For example, a search domain of an object detection scheme (e.g., a target object class, a target object size, a target object rotation angle, etc.) may be separated into subdomains (e.g., such as subdomains of object classes, subdomains of object sizes, subdomains object rotation angles, etc.). Specialized, subdomain-level object detection/segmentation tasks may then be separated across sequential video frames. As such, different subdomain-level processing techniques (e.g., via specialized neural networks) may be implemented across different frames of a video sequence. Moreover, redundancy information of consecutive video frames may be leveraged, such that specialized object detection tasks combined with visual object tracking across consecutive frames may enable more efficient (e.g., more accurate, less computationally intensive, etc.) full domain object detection and object segmentation schemes.

Fast object detection in video via scale separation

Simultaneous localization and mapping (SLAM) using dual event cameras

An event-based sensor includes a photoarray and a processing circuit. The photoarray includes an array of cells. Each cell includes a photosensor generating a sensor signal dependent on an intensity of light casted on the cell, and an intensity monitoring circuit outputting an ON signal when the light intensity is increasing and an OFF signal when the light intensity is decreasing. The processing circuit is configured to generate an event in response to the ON and OFF signals, filter out the events caused by flickering light, and pass the events caused by motion.

Event-based sensor that filters for flicker

Methods, apparatuses, and systems are described for positional tracking (PT) in Augmented Reality (AR) applications. The method may include creating different images for AR preview and PT applications. In some cases, the PT information utilizes key point information. The AR preview and the PT images may be generated simultaneously using time division, space division, split timing, or any combination thereof. This may result in PT with significantly less motion blur. Accordingly, the PT will be more robust, and there will be more good frames available for key components of a PT algorithm (i.e., using visual PT to calibrate the biases of an inertial measurement unit). Thus, the methods and apparatus of the present disclosure may enable more effective management of the trade-off between motion-blur, noise and resolution for PT.

Sensor auto-configuration

A pulsed image capturing system includes a pulse generation circuit, a dynamic vision sensor (DVS), and a DVS scanning and readout circuit. A global reset signal is transmitted to pixels of the DVS to set a reference illumination level for the pixels, and then a pulsed light pattern is generated. A global hold signal is transmitted to the pixels, after the pulsed light pattern is generated, to hold an illumination change event indicating a change with respect to the reference illumination level. A DVS scanning and readout operation is performed on event signals received from the pixels, after the global hold signal is transmitted to the pixels. A reflection of the pulsed light pattern is extracted from the DVS output after cleaning artifacts from the DVS output without additional image processing.

System and method for pulsed light pattern capturing using a dynamic vision sensor

Event-based image feature extraction includes reducing an accumulated magnitude of a leaky integrate and fire (LIF) neuron based on a difference between a current time and a previous time; receiving an event input from a dynamic vision sensor (DVS) pixel at the current time; weighting the received input; adding the weighted input to the reduced magnitude to form an accumulated magnitude of the LIF neuron at the current time; and, if the accumulated magnitude reaches a threshold, firing the neuron and decreasing the accumulated magnitude.

Event-based image feature extraction

In various embodiments of an image processing method and apparatus, first and second point clouds representing respective images of a scene/object from different viewpoints are obtained. Extracted features points from the first point cloud are matched with extracted feature points from the second point cloud, using depth based weighting, as part of an ICP initiation process. The first and second point clouds are then further ICP processed using results of the initiation process to generate at least one coordinate-transformed point cloud.

Image processing with iterative closest point (ICP) technique

A method includes determining blocks of speckle-free regions in a first image; storing information about the blocks in a memory; and processing at least one second image based on the stored information.

Image processing devices and methods for operating the same

A method for acquiring intensity images through a dynamic vision sensor is provided, the method including resetting an inner state of each of a plurality of pixels in the dynamic vision sensor to a predetermined value; receiving from at least one of the plurality of pixels an event signal based on a change in its inner state indicative of a difference in amount of light incident upon the at least one of the plurality of pixels versus that corresponding to the predetermined value; and integrating over a time period the received event signal for the at least one of the plurality of pixels to determine the intensity of light incident upon the at least one of the plurality of pixels.

Intensity image acquisition from dynamic vision sensors

The disclosure provides for globally detecting and correcting unwanted sensor artifacts arising from processing techniques combining multiple visual images into an output image while possessing information relating to a localized area of the image that can be accomplished during scanning of the image and independent of the number of images or the computer processing method used to acquire them.

Image artifact detection and correction in scenes obtained from multiple visual images

A first image and a second image of an object taken with different viewing directions are received. The first image and the second image are downscaled in a ratio of a downscale factor DF to generate a first downscaled image and a second downscaled image, respectively. An edge map is generated by detecting an edge pixel from the first downscaled image. An initial cost volume matrix is generated from the first downscaled image and the second downscaled image according to the edge map. An initial disparity estimate is generated from the initial cost volume matrix. The initial disparity estimate is refined using the initial disparity estimate to generate a final disparity set. A depth map is generated from the first image and the second image using the final disparity set.

Stereo matching system and method of operating thereof

Embodiments of a virtual frame buffer (VFB) for buffering asynchronous data originating from an imaging source such as a dynamic vision sensor. The VFB may include a hash table memory, a data memory, and a control unit. The control unit receives an input address of an asynchronous data event and determines a pseudo-randomly computed hash of the hash table memory at which to store the input address. The control unit links the hash to a handle corresponding to a data memory cell that stores pixel data of the event. The VFB may interface like a full frame buffer, but use significantly less memory. The VFB may be used with a pixel processing algorithm to search for recent events in a localized environment of a pixel.

Growing community of inventors

Givatayim, Israel

Nathan Henri Levy