Microsoft Technology Licensing, LLC

Other

Christopher Douglas Edmonds

Michael Bleyer

Raymond Kirk Price

Sudipta Narayan Sinha

Mark James Finocchio

Constantin Dulu

Christian Markus Maekelae

Alexandru Octavian Balan

Dominik Alexander Konik

Arthur Charles Tomlin

Michael C Fanning

Adam G Poulos

Yuri Pekelny

Timothy Scott Rice

Andrew R Sterland

Felix Gerard Torquil Ifor Andrew

Simon Calvert

Casey L Miller

Elliot H Omiya

Michael Edward Samples

Gregg Bernard Miskelly

Erroll William Wood

Harold L Pierson, Iii

Robert Alan Paveza

Daniel James Moseley

Francisco Javier Wong

Antonios Matakos

William Chao-Hui Hwang

Jeffrey T Fisher

Taylor C Woll

Karthik Vivek Nadig

Donald John Patrick O'Neil

Zhaochuan Gao

Andriy Stasyuk

Mikael Froejk Figueroa

Adam James Muff

Daniel Joseph McCulloch

John Allen Tardif

Andrew William Fitzgibbon

Jeffrey Alan Kohler

Anthony James Ambrus

Susan Elizabeth Carrie

Nicholas Fajt

Jonathan Gustav Paulovich

Thomas Joseph Cashman

Lev Cherkashin

Michael Scott Fenton

Kyungsuk David Lee

Michael Grabner

Mark Plagge

James Dack

Evan Gregory Levine

Eric Scott Rehmeyer

Federica Bogo

Brent Michael Wilson

Varun Ramesh Mani

Matthew B Karr

Benjamin Brown

Brent Charles Allen

Muhammad Jabir Kapasi

Paul Malcolm McIlroy

Edward Daniel Parker

Cecilia Bong

Miguel Angel Susffalich

John Albert Judnich

Randy Crane

Melissa Arde Lobb

Casey Lee Miller

Harold L Iii Pierson

Gregg Miskelly

A system for generating depth information from low-resolution images is configured to access a plurality of image frames capturing an environment, identify a first group of image frames from the plurality of image frames, and generate a first image comprising a first composite image of the environment using the first group of image frames as input. The first composite image has an image resolution that is higher than an image resolution of the image frames of the first group of image frames. The system is also configured to obtain a second image of the environment, where parallax exists between a capture perspective associated with the first image and a capture perspective associated with the second image. The system is also configured to generate depth information for the environment based on the first image and the second image.

Systems and methods for generating depth information from low-resolution images

A system for facilitating dark current compensation by weighted filtering is configurable to (i) receive an input dark current image; (ii) generate a corrected dark current image at least by scaling pixel values of the input dark current image based upon ambient light conditions; and generate a weight map comprising a weight value for each pixel of the corrected dark current image. For each pixel of the corrected dark current image, the weight value of the weight map may be based upon a light level of the pixel of the corrected dark current image. The system is also configurable to generate an output image by utilizing the weight map to filter an input image.

Systems and methods for facilitating dark current compensation by weighted filtering

Disclosed herein are techniques for providing an illumination system that emits illumination into an environment while also enabling that system to be undetectable to certain types of external light detection systems. The system includes a single photon avalanche diode (SPAD) low light (LL) detection device and a light emitting device. The light emitting device provides illumination having a wavelength of at least 950 nanometers (nm). An intensity of the illumination is set to a level that causes the illumination to be undetectable from a determined distance away based on the roll off rate of the light. While the light emitting device is providing the illumination, the SPAD LL detection device generates an image of an environment in which the illumination is being provided.

External illumination with reduced detectability

A system for modifying a dark current image is configurable to receive an input image depicting a dark current state for one or more pixels of the input image, The dark current state for one or more pixels of the input image comprises one of: a faulty state or a non-faulty state. The system is configurable to partition the input image into a plurality of partitions and generate an updated input image by imposing at least one quantity constraint or at least one severity constraint to the plurality of partitions in association with at least one type of dark current state.

Systems and methods for selectively constraining and implementing cross talk templates for dark current images

A system for updating continuous image alignment of separate cameras identifies a previous alignment matrix associated with a previous frame pair captured at one or more previous timepoints by a reference camera and a match camera. The previous alignment matrix is based on visual correspondences in the previous frame pair. The system also identifies a current matrix associated with a current frame pair captured at one or more current timepoints by the reference camera and the match camera. The current matrix is based on visual correspondences in the current frame pair. The system also identifies a difference value associated with the reference camera or the match camera relative to the one or more previous timepoints and the one or more current timepoints. The system also generates an updated alignment matrix by using the previous alignment matrix, the current matrix, and the difference value as inputs.

Systems and methods for updating continuous image alignment of separate cameras

Techniques for generating a high resolution full color output image from lower resolution sparse color input images are disclosed. A camera generates images. The camera's sensor has a sparse Bayer pattern. While the camera is generating the images, IMU data for each image is acquired. The IMU data indicates a corresponding pose the camera was in while the camera generated each image. The images and IMU data are fed into a motion model, which performs temporal filtering on the images and uses the IMU data to generate a red-only image, a green-only image, a blue-only image, and a monochrome image. The color images are up-sampled to match the resolution of the monochrome image. A high resolution output color image is generated by combining the up-sampled images and the monochrome image.

Generate super-resolution images from sparse color information

Techniques for updating a position of overlaid image content using IMU data to reflect subsequent changes in camera positions to minimize latency effects are disclosed. A 'system camera' refers to an integrated camera that is a part of an HMD. An “external camera” is a camera that is separated from the HMD. The system camera and the external camera generate images. These images are overlaid on one another and aligned to form an overlaid image. Content from the external camera image is surrounded by a bounding element in the overlaid image. IMU data associated with both the system camera and the external camera is obtained. Based on that IMU data, an amount of movement that the system camera and/or the external camera have moved since the images were originally generated is determined. Based on that movement, the bounding element is shifted to a new position in the overlaid image.

Low motion to photon latency rapid target acquisition

A system for power efficient image acquisition is configurable to capture, using an image sensor, a plurality of partial image frames including at least a first partial image frame and a second partial image frame. The first partial image frame is captured at a first timepoint using a first subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second partial image frame is captured at a second timepoint using a second subset of image sensing pixels of the plurality of image sensing pixels of the image sensor. The second subset of image sensing pixels includes different image sensing pixels than the first subset of image sensing pixels, and the second timepoint is temporally subsequent to the first timepoint. The system is configurable to generate a composite image frame based on the plurality of partial image frames.

Systems and methods for power efficient image acquisition using single photon avalanche diodes (SPADs)

An HMD includes a single photon avalanche diode (SPAD) array comprising a plurality of SPAD pixels. The HMD also includes a display positioned to display images for viewing by an eye of a user. The HMD also includes one or more processors and one or more hardware storage devices storing instructions that are executable by the one or more processors to configure the HMD to perform various acts associated with using the SPAD array to capture an image frame of an environment for display to the user.

SPAD array for intensity image sensing on head-mounted displays

An image sensor configured to capture imagery with mitigated noise includes a plurality of image sensing pixels arranged to form a sensor array. Each image sensing pixel of the plurality of image sensing pixels comprises an active area configured to receive photons to facilitate image capture. Each active area comprises a length and a width. For at least one image sensing pixel of the plurality of image sensing pixels, the length or the width of the active area is smaller than about 80% of a pixel pitch measurement between the at least one image sensing pixel and an adjacent image sensing pixel. A size of the active area relative to the pixel pitch measurement contributes to mitigating sensor noise for the image sensor.

Image sensing pixel configurations for reduced sensor noise

Techniques to temporally filter images via a filtering weight computation are disclosed. A first image having a first timestamp and a second image having a second timestamp are acquired. These images are generated by a camera, and the first timestamp is before the second timestamp. A motion compensation (MC) operation is performed on the first image to produce an MC image. A difference image is generated using the MC image and the second image. The difference image reflects differences in intensities that exist between the two images. A local weight map is generated based on those differences. A global weight map is generated based on certain IMU data. A final weight map is generated based on a combination of the local weight map and the global weight map. The final weight map is used to generate a filtered image.

Temporal filtering weight computation

A system for dark current compensation in SPAD imagery is configurable to capture an image frame with the SPAD array and generate a temporally filtered image by performing a temporal filtering operation using the image frame and at least one preceding image frame. The at least one preceding image frame is captured by the SPAD array at a timepoint that temporally precedes a timepoint associated with the image frame. The system is also configurable to obtain a dark current image frame. The dark current image frame includes data indicating one or more SPAD pixels of the plurality of SPAD pixels that detect an avalanche event without detecting a corresponding photon. The system is also configurable to generate a dark current compensated image by performing a subtraction operation on the temporally filtered image or the image frame based on the dark current image frame.

Systems and methods for dark current compensation in single photon avalanche diode imagery

Techniques for generating a fused enhanced image. A first image is generated using a first camera of a first modality, and a second image is generated using a second camera of a second modality. Pixels that are common between the two images are identified. Textures for the common pixels are determined. A camera characteristic, which is linked to noise, is identified. A scaling factor is applied to the textures in the first image. A first saliency is determined using the scaled textures. A second saliency is determined using the textures from the second image. An alpha map is generated and reflects edge detection weights that have been computed for each one of the common pixels based on the two saliencies. Based on the alpha map, textures are merged from the common pixels to generate the fused enhanced image.

Low light and thermal image normalization for advanced fusion

Techniques for generating a temporally filtered image designed to compensate for global motions of a camera and to compensate for local motions of an object are disclosed. A history frame and a current frame are acquired. A global motion compensation operation is performed on the history frame to reproject a pose of the history frame to match the pose of the current frame. The history frame is compared against the current frame to identify pixels that represent moving objects. For each of those pixels, an optical flow vector is computed. The optical flow vectors are then applied to those pixels to shift those pixels to new locations. These new positions, which are in the history frame, correspond to positions that were identified in the current frame. Afterwards, the current frame is temporally filtered with the history frame.

Advanced temporal low light filtering with global camera motion compensation and local object motion compensation

A system for adding persistence to SPAD imagery is configurable to capture, using a SPAD array, a plurality of image frames. The system is configurable to capture, using an IMU, pose data associated with the plurality of image frames. The pose data includes at least respective pose data associated with each of the plurality of image frames. The system is configurable to determine a persistence term based on the pose data. The system is also configurable to generate a composite image based on the plurality of image frames, the respective pose data associated with each of the plurality of image frames, and the persistence term. The persistence term defines a contribution of each of the plurality of image frames to the composite image.

Systems and methods for adding persistence to single photon avalanche diode imagery

Growing community of inventors

Carnation, WA, United States of America

Christopher Douglas Edmonds