Microsoft Technology Licensing, LLC

Denis Demandolx

Michael Bleyer

Raymond Kirk Price

Jian Zhao

Steven J White

Thomas Lund Dideriksen

Douglas Alan Ricard

Sing Bing Kang

Ravi Kiran Nalla

Charbel Khawand

Drew Edward Steedly

Quentin Simon Charles Miller

Zhiqiang Liu

Steve Glenner

Todd Michael Lyon

Youding Zhu

Qi Kuan Zhou

Robert Milausnic

Junghwan Moon

Kevin Geeyoung Chiu

Michael Dean Stokes

Glenn Frederick Evans

Antonio Criminisi

Ashraf A Michail

Juan Vicente Esteve Balducci

Jeffrey Alan Kohler

David R Blythe

Karthik G Anbalagan

Jingyang Xu

Eric Paul Bennett

Anantha Rao Kancherla

Eric W Richards

Bradley Phillip Gibson

Christopher Stephen Messer

Michael John Schoenberg

Donald M Marsh

Steve J White

Sergei S Shinkevich

Will Guyman

Minshik Park

Justin Nafziger

Benjamin Ellett

Vladimir Farbman

An illumination module and a depth camera on a near-eye-display (NED) device used for depth tracking may be subject to strict power consumption budgets. To reduce power consumption of depth tracking, the illumination power of the illumination module is controllably varied. Such variation entails using a previous frame, or previously recorded data, to inform the illumination power used to generate a current frame. Once the NED determines the next minimum illumination power, the illumination module activates at that power level. The illumination module emits electromagnetic (EM) radiation (e.g. IR light), the EM radiation reflects off surfaces in the scene, and the reflected light is captured by the depth camera. The method repeats for subsequent frames, using contextual information from each of the previous frames to dynamically control the illumination power. Thus, the method reduces the overall power consumption of the depth camera assembly of the NED to a minimum level.

Active illumination management through contextual information

A stereoscopic 3D imaging system includes multiple imaging sensors with adjustable optics. The adjustable optics are variable to alter the FOV of each of the multiple imaging sensors to improve angular resolution of the imaging system.

Passive and active stereo vision 3D sensors with variable focal length lenses

Disclosed is a technique for 3D reconstruction through a combination of time-of-flight (ToF) and stereoscopy. Use of a single modulation frequency in ToF computation provides a set of ambiguous distances. To determine a single accurate distance out of this candidate set, a stereoscopic comparison of an image pair provides a disambiguating distance. The stereoscopic comparison uses a stored virtual image of at least part of the emitted light, and the detected image of light reflected from an object. The stereoscopic distance is used to determine which of the multiple accurate distances is correct based on proximity. The closest of the multiple distances to the disambiguating distance is taken as the actual distance.

Single-frequency time-of-flight depth computation using stereoscopic disambiguation

A method for active eye-tracking comprises pulsing on and off a plurality of infrared optical sources configured to emit infrared light with a narrow spectral linewidth toward an eye of a user, such that a pulse-on duration is less than a duration needed to fully thermalize each optical source. One or more shuttered optical sensors are configured to receive infrared light reflected off the eye of the user. The shuttered optical sensors are opened for a detection duration based on the pulse-on duration, the shuttered optical sensors. A conformation of the user's eye is indicated based on infrared light received at the shuttered optical sensor during the detection duration.

Infrared eye-tracking in high ambient light conditions

Disclosed are a device and a method of depth sensing that handle light leakage issues. In some embodiments, the depth sensing device includes a light emitter that illuminates an environment of the depth sensing device. The device identifies a first portion of the emitted light that is prevented from reaching the environment of the device due to being redirected by an optical component located in proximity to the light emitter. An imaging sensor of the device detects a second portion of the emitted light that reaches and is reflected by a surface in the environment of the device other than a surface of the optical component. The device generates, based on the second portion of the emitted light, a depth map that includes a plurality of values corresponding to distances relative to the device, wherein said generating excludes from consideration the identified first portion of the emitted light.

Depth camera light leakage avoidance

An illumination source is operated to illuminate an operating environment and an optical sensor is periodically operated for a detection period to detect illumination reflected from one or more subjects within the operating environment. Upon recognizing a source of interfering pulsed illumination within the operating environment, the timing of a subsequent detection period may be varied. In this way, sensing of the interfering pulsed illumination may be averted.

Interference mitigation via adaptive depth imaging

A method for three-dimensional imaging includes emitting an output light with a structured light illuminator in a structured light pattern, receiving a trigger command, changing a field of illumination of the illuminator, and changing a field of view of an imaging sensor. The field of view and the field of illumination are linked, such that the field of view of the imaging sensor is the same as the field of illumination of the illuminator at a short throw field of view and a long throw field of view. The method further includes detecting a reflected light with the imaging sensor and measuring a depth value by calculating a distortion of the structured light pattern.

Structured light 3D sensors with variable focal length lenses and illuminators

A device and method use multiple light emitters with a single, multi-spectrum imaging sensor to perform multi-modal infrared light based depth sensing and visible light based Simultaneous Localization and Mapping (SLAM). The multi-modal infrared based depth sensing may include, for example, any combination of infrared-based spatial mapping, infrared based hand tracking and/or infrared based semantic labeling. The visible light based SLAM may include head tracking, for example.

Multi-spectrum illumination-and-sensor module for head tracking, gesture recognition and spatial mapping

A three dimensional imaging system includes a hybrid imaging sensor with infrared wavelength photoreceptors and visible wavelength photoreceptors integrated in one photoreceptor array. The three dimensional imaging system includes a bandpass filter to filter incoming light before the light is received at the hybrid imaging sensor to reduce crosstalk between the infrared wavelength photoreceptors and visible wavelength photoreceptors. The infrared photoreceptors receive infrared light provided by a structured light source and the visible wavelength photoreceptors receive ambient visible light. The hybrid imaging sensor collects infrared image data and visible light image data concurrently and/or simultaneously.

Hybrid imaging sensor for structured light object capture

Disclosed are an apparatus and a method for detecting deformation of a structured light depth imaging system and automatic re-calibration for the depth imaging system. In one embodiment, a depth imaging device includes a light projector, a camera and a processor. The light projector emits light corresponding to a projected pattern image having a plurality of features with known locations in the projected pattern image. The camera captures the light reflected by an environment of the depth imaging device and generates a reflected pattern image as a two-dimensional (2D) projection of the environment. The reflected pattern image includes a plurality of features that correspond to the features of the projected pattern image. The processor detects a misalignment for a distance or an orientation between the light projector and the camera based on a relationship between the features in the projected pattern image and the features in the reflected pattern image.

Deformation detection and automatic calibration for a depth imaging system

A device for detecting light includes a photoreceptor array that is sensitive to the visible light spectrum and has a first wavelength sensor, a second wavelength sensor, and a third wavelength sensor. The first wavelength sensor has a first peak sensitivity at a first wavelength. The second wavelength sensor has a second peak sensitivity at a second wavelength. The third wavelength sensor has a third peak sensitivity at a third wavelength. The first wavelength sensor has a first base sensitivity at least 30% of the first peak sensitivity at the second wavelength and third wavelength. The second wavelength sensor has a second base sensitivity at least 30% of the second peak sensitivity at the first wavelength and third wavelength. The third wavelength sensor has a third base sensitivity at least 30% of the third peak sensitivity at the first wavelength and second wavelength.

Systems and methods of increasing light detection in color imaging sensors

A method including receiving an image of a scene illuminated by both a predetermined structured light pattern and a flood fill illumination, generating an active brightness image of the scene based on the received image of the scene including detecting a plurality of dots of the predetermined structured light pattern, and removing the plurality of dots of the predetermined structured light pattern from the active brightness image, and generating a depth map of the scene based on the received image and the active brightness image.

Depth map with structured and flood light

Disclosed are an apparatus and a method of compensating temperature shifts of a structured light pattern for a depth imaging system. In some embodiments, a depth imaging device includes a light source, an imaging sensor and a processor. The light source emits light corresponding to a pattern. A temperature drift of the light source can cause a shift of the pattern. The imaging sensor receives the light reflected by environment in front of the depth imaging device and generates a depth map including a plurality of pixel values corresponding to depths of the environment relative to the depth imaging device. The processor estimates the shift of the pattern based on a polynomial model depending on the temperature drift of the light source. The processor further adjusts the depth map based on the shift of the pattern.

Temperature compensation for structured light depth imaging system

A sensor device is provided that includes an array of light sensitive pixels configured in a predetermined pattern of light sensitive pixels that alternates between broadband visible light sensitive pixels and infrared light sensitive pixels, each broadband visible light sensitive pixel being configured to have increased sensitivity to light of the visible light spectrum and to have reduced sensitivity to light of the infrared light spectrum compared to each infrared light sensitive pixel, and each infrared light sensitive pixel being configured to have increased sensitivity to light of the infrared light spectrum and to have reduced sensitivity to light of the visible light spectrum compared to each broadband visible light sensitive pixel.

Sensor with alternating visible and infrared sensitive pixels

Growing community of inventors

Bellevue, WA, United States of America

Denis Demandolx