Mesa Imaging Ag

Heptagon Micro Optics Pte. Ltd.

Ams Sensors Singapore Pte. Ltd.

Carl Zeiss Microimaging Gmbh

Csem Centre Suisse D'electronique Et De Microtechnique Sa&#x2014;recherche Et D&#xe9;veloppement

Thierry Oggier

Bernhard Buettgen

Michael Lehmann

Felix Lustenberger

Simon Neukom

Matthias Schweizer

Rolf Kaufmann

Jonas Felber

Mathias Deschler

Stéphane Kaloustian

Cornelia Prott

Michael Richter

Laurent Lamesch

Vincent Hayward

Bernhard Büttgen

Guido Becker

Roger Gassert

Guy Voirin

Yeongmi Kim

Markus Riesch

Stefan Beat Schneller

Thierry Zamofing

Kaspar Cottier

Alessandro Esposito

Fred Silvester Wouters

Rino E Kunz

Stefan Schneller

The present invention relates to vision sensors based on an active illumination. An imaging system includes an imaging sensor and is adapted to process an image of a scene being illuminated by at least two different illumination sources each having a wavelength in the near infrared range. In a variant, the imaging system is adapted to use an illumination source having a modulation frequency below the modulation frequency used to perform a three dimensional time of flight measurement. In a variant, the imaging system is adapted to acquire a reduced number of samples per frame than used in time of flight measurements.

Sensor system with active illumination

The invention relates to a depth sensor module and depth sensing method. The depth sensor module and method is adapted to include a light detector part and emitting part with a least two light sources spatially offset in the direction of the triangulation baseline. In some of the embodiments, the pixel field of the image sensor in the light detector part consists of time-of-flight pixels. Depth measurements derived by triangulation can be used to calibrate depth maps generated by the time-of-flight measurements.

Depth sensor module and depth sensing method

A method and system to compensate for stray light errors in time of flight (TOF) camera systems uses reference targets in the in the field of view (FOV) that can be used to measure stray light. In different embodiments, one or more reference targets are used.

Stray light compensation method and system for time of flight camera systems

A modulation technique for 3D time-of-flight (TOF) cameras allows the operation of fully autonomous operated 3D TOF cameras. The method subdivides the exposure time into several sub-exposure intervals, for which the signal control unit adds a preferably pseudo-random common phase delay to the illumination and the sensor.

System and method for multi TOF camera operation using phase hopping

A method to compensate for multi-path in time-of-flight (TOF) three dimensional (3D) cameras applies different modulation frequencies in order to calculate/estimate the error vector. Multi-path in 3D TOF cameras might be caused by one of the two following sources: stray light artifacts in the TOF camera systems and multiple reflections in the scene. The proposed method compensates for the errors caused by both sources by implementing multiple modulation frequencies.

Multi-path compensation using multiple modulation frequencies in time of flight sensor

A demodulation pixel improves the charge transport speed and sensitivity by exploiting two effects of charge transport in silicon in order to achieve the before-mentioned optimization. The first one is a transport method based on the CCD gate principle. However, this is not limited to CCD technology, but can be realized also in CMOS technology. The charge transport in a surface or even a buried channel close to the surface is highly efficient in terms of speed, sensitivity and low trapping noise. In addition, by activating a majority carrier current flowing through the substrate, another drift field is generated below the depleted CCD channel. This drift field is located deeply in the substrate, acting as an efficient separator for deeply photo-generated electron-hole pairs. Thus, another large amount of minority carriers is transported to the diffusion nodes at high speed and detected.

Demodulation pixel incorporating majority carrier current, buried channel and high-low junction

A demodulation image sensor, such as used in time of flight (TOF) cameras, extracts all storage- and post-processing-related steps from the pixels to another array of storage and processing elements (proxels) on the chip. The pixel array has the task of photo-detection, first processing and intermediate storage, while the array of storage and processing elements provides further processing and enhanced storage capabilities for each pixel individually. The architecture can be used to address problems due to the down-scaling of the pixel size. Typically, either the photo-sensitivity or the signal storage capacitance suffers significantly. Both a lower sensitivity and smaller storage capacitances have negative influence on the image quality. The disclosed architecture allows for keeping the storage capacitance unaffected by the pixel down-scaling. In addition to that, it provides a high degree of flexibility in integrating more intelligence into the image sensor design already on the level of the pixel array. In particular, if applied to demodulation pixels, the flexibility of the architecture allows for integrating on sensor-level concepts for multi-tap sampling, mismatch compensation, background suppression and so on, without any requirement to adjust the particular demodulation pixel architecture.

Demodulation sensor with separate pixel and storage arrays

A time of flight (TOF) based camera system includes an illumination module that illuminates only portion of the sensor's field of view that translates to a given region of the pixels of the imaging sensor. The acquired data of the pixel region is processed and/or readout, typically. After the exposure time of the first pixel region is completed, a second pixel region is illuminated and the second pixel region is processed. This procedure can be repeated a couple of times up to a few hundred even thousand times until the entire pixel array is readout and possibly read-out a number of times. The full depth image is then reconstructed based on the results from the different pixel region acquisitions. This system can be used to reduce stray light. Compared to state-of-the-art TOF camera, the presented method and device show improvements in background light stability and a reduction in multiple reflections.

Time of flight camera with stripe illumination

A method and system for reducing multi-path effects (stray light and multiple reflections) in 3D time of flight cameras by partly masking the camera's illumination elements.

3D TOF camera with masked illumination

The modulation scheme disclosed in this invention report allows for utilizing multiple 3D time-of-flight cameras at the same time by exploiting the inherent pseudo noise properties of the optical modulation signals. Compared to recent systems based on pure pseudo noise modulation signals, the stochastic measurement error in a single-camera environment is significantly reduced. The basic concept relies on the generation of a three level optical modulation signal that includes two pseudo noise sequences.

Multi-level digital modulation for time of flight method and system

Improved field-of-illumination (FOI) and field-of-view (FOV) matching for 3D time-of-flight cameras is provided using light emitters with rectangular reflectors. A better adjustment of the FOI with the camera's FOV has the following advantages: optimal use of emitted light and reduced multi-path problems. Furthermore, embodiments bring the benefit for rather low-cost customization of the illumination to match the FOI to the specified FOV.

Time of flight camera with rectangular field of illumination

Due to their parallel illumination and acquisition for all the pixels, today's state-of-the-art time-of-flight (TOF) range cameras suffer from erroneous measurements caused by multiple reflections in the scene. The invention proposes to compensate for the multi-path fusing the results obtained by applying two spatially different illumination schemes, typically one to achieve highest possible lateral resolution and for the second one structuring the emitted light and by doing so lowering the lateral resolution but limiting the impact of multiple reflections.

Method to compensate for errors in time-of-flight range cameras caused by multiple reflections

A pixel is formed in a semiconductor substrate (S) with a plane surface for use in a photodetector. It comprises an active region for converting incident light (In) into charge carriers, photogates (PGL, PGM, PGR) for generating a lateral electric potential (&#x3a6;(x)) across the active region, and an integration gate (IG) for storing charge carriers generated in the active region and a dump site (Ddiff). The pixel further comprises separation-enhancing means (SL) for additionally enhancing charge separation in the active region and charge transport from the active region to the integration gate (IG). The separation-enhancing means (SL) are for instance a shield layer designed such that for a given lateral electric potential (&#x3a6;(x)), the incident light (In) does not impinge on the section from which the charge carriers would not be transported to the integration gate (IG).

Solid-state photodetector pixel and photodetecting method

A demodulation image sensor, such as used in time of flight (TOF) cameras, extracts all storage- and post-processing-related steps from the pixels to another array of storage and processing elements (proxels) on the chip. The pixel array has the task of photo-detection, first processing and intermediate storage to create subframes, while the array of storage and processing elements provides accumulation into frames. Particularly, sampled values of several subframes are summed in a compressed manner. Possible compression is to use exponential function.

Time of flight sensor with subframe compression and method

A high-speed, high-sensitivity demodulation sensor usable for e.g. time-of-flight application uses a back side illuminated (BSI) image sensor chip, in which the photo-generated charges are first transferred to a demodulation area, from which the charges are then sampled and stored on at least one specific storage node. The storage node is electrically isolated from the sensitive area. Such a pixel might find its use specifically in 3D time-of-flight imaging given its improvements in sensitivity because the presented invention allows to design pixel with up to 100% fill factor and enables charge detection even if the charge generation by the photon occurs deep in the silicon substrate.

Growing community of inventors

Zurich, Switzerland

Thierry Oggier