Microsoft Technology Licensing, LLC

Michael Scott Fenton

Sergio Ortiz Egea

John Peter Godbaz

Onur Can Akkaya

Cyrus Soli Bamji

Avdhesh Chhodavdia

Abdelrehim Ahmed

Sheethal Somesh Nayak

Michael Bleyer

Christopher Douglas Edmonds

Mark James Finocchio

Maria Esther Pace

Travis Perry

Erroll William Wood

Mirko Schmidt

Andrew Dean Payne

Arrigo Benedetti

Andrew David Daniel

Jayachandra Gullapalli

Riaz Imdad Ali

John Albert Judnich

The technology described herein recalibrates a structured light sensor in the field using time-of-flight sensor data. Structured light sensors are sensitive to mechanical changes that result in decreased accuracy. A structured light system calculates the range to an object by comparing a reference image to the actual image of the scene. The reference image is what the projected light pattern would look like on a flat object at a known distance. When the projected image changes, the reference image no longer matches the projected pattern. The calibration technology described herein captures a new reference image based on the current sensor characteristics using a time-of-flight capable sensor as the structured light imaging sensor.

Field calibration of a structured light range-sensor

A camera system is configured to automatically monitor an area. Depth image(s) of the area are acquired based on active IR light emitted by the camera system and reflected from the area to a sensor array of the camera system. The depth image(s) are computer analyzed to identify a human subject. For each spectral illuminator of the camera system, spectral light image(s) of the area are acquired based on active spectral light in the spectral light sub-band of the spectral illuminator reflected from the area to the sensor array. The spectral light image(s) for the spectral illuminators are computer analyzed to identify an interaction between the human subject and an object in the area. In response to identifying the interaction between the human subject and the object in the area, an action to be performed for the object in the area is computer issued.

Monitoring activity with depth and multi-spectral camera

A camera system includes one or more spectral illuminators, a tunable optical filter, and a sensor array. Active spectral light emitted from the one or more spectral illuminators towards a scene is dynamically tuned to an illumination sub-band selected from a plurality of different illumination sub-bands. Sequentially for each of a plurality of fluorescing light sub-bands different than the selected illumination sub-band, the tunable optical filter is adjusted to block light from being transmitted from the scene to the sensor array in all but a tested fluorescing light sub-band from the plurality of different fluorescing light sub-bands, and the sensor array is addressed to acquire one or more image of the scene in the tested fluorescing light sub-band.

Multi-spectral fluorescent imaging

A camera includes a time-of-flight illuminator configured to emit active IR light and a plurality of spectral illuminators, each spectral illuminator configured to emit active spectral light in a different spectral light sub-band. The camera further includes a sensor array that includes a plurality of differential sensors. Each differential sensor is configured to differentially measure both 1) the active IR light, and 2) the active spectral light in each of the different spectral light sub-bands. The camera further includes an output machine operatively connected to the sensor array. The output machine is configured to output a matrix of pixels based on time-multiplexed measurements of the sensor array. Each pixel of the matrix includes 1) a depth value, and 2) a plurality of spectral values. Each of the plurality of spectral values corresponds to a spectral light sub-band of one of the plurality of spectral illuminators.

Depth and multi-spectral camera

A method to identify one or more depth-image segments that correspond to a predetermined object type is enacted in a depth-imaging controller operatively coupled to an optical time-of-flight (ToF) camera; it comprises: receiving depth-image data from the optical ToF camera, the depth-image data exhibiting an aliasing uncertainty, such that a coordinate (X, Y) of the depth-image data maps to a periodic series of depth values {Z}; and labeling, as corresponding to the object type, one or more coordinates of the depth-image data exhibiting the aliasing uncertainty.

Subject tracking with aliased time-of-flight data

A camera is configured to output a test depth+multi-spectral image including a plurality of pixels. Each pixel corresponds to one of the plurality of sensors of a sensor array of the camera and includes at least a depth value and a spectral value for each spectral light sub-band of a plurality of spectral illuminators of the camera. A face recognition machine is previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image. The face recognition machine is configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a face.

Face recognition using depth and multi-spectral camera

The technology described herein is a depth camera that uses structured light and modulated light to produce a depth image. The modulated and structured light allows a time-of-flight depth to be calculated for each unit (e.g., a single dot) of the reflected structured light image captured by the depth camera's image sensor. The time-of-flight depth can be used to accurately identify each unit of the reflected structured light image. Once each unit of the reflected structured light image is identified, a structured light triangulation algorithm can be used to calculate a depth for each unit of the reflected structured light image. The present technology also allows a confidence score to be generated for the assigned depths within the depth image by comparing the structured light depth for each unit to the TOF depth for each unit.

Time-of-flight augmented structured light range-sensor

The time-of-flight system disclosed herein includes a frequency unwrapping module configured to generate an input phase vector with M phases corresponding to M sampled signals from an object, determine an M&#x2212;1 dimensional vector of transformed phase values by applying a transformation matrix (T) to the input phase vector, determine an M&#x2212;1 dimensional vector of rounded transformed phase values by rounding the transformed phase values to a nearest integer, and determine a one dimensional lookup table (LUT) index value by transforming the M&#x2212;1 dimensional rounded transformed phase values. The index value is input into the one dimensional LUT to determine a range of the object.

Multi-frequency unwrapping

A camera is configured to output a test depth+multi-spectral image including a plurality of pixels. Each pixel corresponds to one of the plurality of sensors of a sensor array of the camera and includes at least a depth value and a spectral value for each spectral light sub-band of a plurality of spectral illuminators of the camera. An object recognition machine is previously trained with a set of labeled training depth+multi-spectral images having a same structure as the test depth+multi-spectral image. The object recognition machine is configured to output a confidence value indicating a likelihood that the test depth+multi-spectral image includes a specified object.

Object recognition using depth and multi-spectral camera

A camera includes a sensor array including a plurality of individually addressable sensor elements, each of the plurality of sensor elements responsive to incident light over a broad wavelength band. Covering the sensor array is a light valve switchable electronically between closed and open states. The light valve is configured to, in the closed state, block light of a stopband and transmit light outside the stopband, and, in the open state, transmit the light of the stopband. An electronic controller of the camera is configured to switch the light valve from the closed to the open state and, synchronously with switching the light valve, address the sensor elements of the sensor array.

Camera with light valve over sensor array

A system includes operational circuit blocks associated with configurable counter circuits. A configurable counter circuit is configured to control event signal when counting expires and includes a mode input configured to receive a setting of a programmable control event asynchronous mode and a programmable control event synchronous mode. Depending on the programmed mode and whether a control event has occurred in a previous synchronization period, the configurable counter circuit processes an associated operation responsive to issuance of a synchronization instruction or to issuance of a subsequent control event.

Configurable synchronized processing of multiple operations

Data compression is described herein. The encoder transmits a coded word having replacement bits, as well as a code that defines the starting location of the replacement bits in a data sample. The replacement bits may be actual bits from a selected location in the new data sample. The selected location of the replacement bits can vary from data sample to data sample. The encoder may select the location based on the most significant bit that has changed. Thus, reconstructed data will be bit-accurate from the replaced bits all the way to the highest-order bit. A limited number of key values can be transmitted losslessly. Moreover, the data compression does not need forward error correction (FEC), which is a necessary part of many lossy delta encoding schemes. Furthermore, the encoding and decoding can be done very efficiently in terms of hardware and/or software.

Data compression

An electronic device includes a configurable pulse generator configured to generate a programmable master pulse train. One or more functional circuits of the electronic device includes a programming interface to receive one or more a programmable slave pulse parameters for the one or more functional circuits. The programmable slave pulse parameters are dependent upon the programmable master pulse train. A slave pulse generator generates a slave pulse for one of the functional circuits based on the one or more programmable slave pulse parameters corresponding to the functional circuits relative to the programmable master pulse train.

Growing community of inventors

Sunnyvale, CA, United States of America

Michael Scott Fenton