Microsoft Technology Licensing, LLC

Google Inc.

Johnny Chung Lee

Craig C Peeper

Tommer Leyvand

Szymon Piotr Stachniak

Shao Liu

Simon Stachniak

Evan Hardesty Parker

Tommy Leyvand

Ryan Michael Hickman

Desney S Tan

Ryan M Geiss

Mark James Finocchio

Alex A Kipman

Adrian Wong

Jaron Zepel Lanier

Zsolt Mathe

Charles Claudius Marais

Joel Hesch

Dominik Philemon Kaeser

Ashish A Shah

James Fung

Matthew Milton Pharr

Steven Benjamin Goldberg

Daniel William Erickson

Patrick Mihelich

Renzo De Nardi

Kurt Akeley

Craig Donner

Tommer Leyand

A method includes sequentially outputting from an imaging sensor each pixel row of a set of pixel rows of an image captured by the imaging sensor. The method further includes displaying, at a display device, a pixel row representative of a first pixel row of the captured image prior to a second pixel row of the captured image being output by the imaging sensor. An apparatus includes an imaging sensor having a first lens that imparts a first type of spatial distortion, a display device coupled to the imaging sensor, the display to display imagery captured by the imaging sensor with the first spatial distortion, and an eyepiece lens aligned with the display, the eyepiece lens imparting a second type of spatial distortion that compensates for the first type of spatial distortion.

Pass-through display of captured imagery

In a system having a user-portable display device, a method includes maintaining a lightfield data structure representing at least a portion of a four-dimensional (4D) lightfield for a three-dimensional (3D) world in association with a first pose of the user-portable display device relative to the 3D world. The method further includes determining a second pose of the user-portable display device relative to the 3D world, the second pose comprising an updated pose of the user-portable display device. The method additionally includes generating a display frame from the lightfield data structure based on the second pose, the display frame representing a field of view of the 3D world from the second pose.

System and methods for performing electronic display stabilization via retained lightfield rendering

An electronic device includes at least one sensor, a display, and a processor. The processor is configured to determine a dimension of a physical object along an axis based on a change in position of the electronic device when the electronic device is moved from a first end of the physical object along the axis to a second end of the physical object along the axis. A method includes capturing and displaying imagery of a physical object at an electronic device, and receiving user input identifying at least two points of the physical object in the displayed imagery. The method further includes determining, at the electronic device, at least one dimensional aspect of the physical object based on the at least two points of the physical object using a three-dimensional mapping of the physical object.

Measuring dimension of object through visual odometry

An electronic device () includes a modulated light projector () and an electronic rolling shutter (ERS) imaging camera () having a sensor array () with a set of pixel rows. The electronic device includes a controller () to control the modulated light projector () to project a modulated light flash () during capture of a first image frame () by the sensor array. The controller is to initiate the modulated light flash while each pixel row of the set is exposed for gathering light and to terminate the modulated light flash before a pixel row of the set is exposed for capture of a next image frame by the sensor array. The electronic device further includes a processor () to determine modulated light image data () based on the first image frame. The controller also may control the modulated light projector to refrain from projecting the modulated light flash during capture of a second image frame (), adjacent to the first image frame.

Electronic device with modulated light flash operation for rolling shutter image sensor

A method includes determining, at a first time, a representation of a first head rotation of a head mounted display (HMD) using a first inertial sensor sample stream and rendering, at an application processor, a texture based on the first head rotation. The method further includes determining, at a second time subsequent to the first time, a representation of a second head rotation of the HMD using a second inertial sensor sample stream having a higher sampling rate than the first inertial sensor sample stream, and generating, at a compositor, a rotated representation of the texture based on a difference between the first head rotation and the second head rotation.

Electronic display stabilization for head mounted display

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may then be determined.

Methods and systems for determining and tracking extremities of a target

A head mounted display (HMD) device includes first and second laterally-curved displays disposed about a medial plane, wherein each of the first and second curved displays includes a first lateral section distal from the medial plane and having a curvature with a first radius and a second lateral section adjacent to the medial plane and having a curvature with a second radius less than the first radius.

Head mounted display device with dual curved displays

A method includes determining a first two-dimensional (2D) feature descriptor from a first image captured by an imaging camera in a first pose at a time of capture of the first image, the first pose including a first observation direction of the imaging camera. The method further includes storing, at an electronic device, a 3D feature descriptor including the first 2D feature descriptor and a representation of the first pose of the imaging camera. The method additionally includes determining a second 2D feature descriptor from a second image captured by the imaging camera in a second pose at a time of capture of the second image, the second pose including a second observation direction of the imaging camera. The method also includes storing the 3D feature descriptor with the second 2D feature descriptor and a representation of the second pose of the imaging camera.

3D feature descriptors with camera pose information

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A model may be adjusted based on a location or position of one or more extremities estimated or determined for a human target in the grid of voxels. The model may also be adjusted based on a default location or position of the model in a default pose such as a T-pose, a DaVinci pose, and/or a natural pose.

Systems and methods for tracking a model

An image such as a depth image of a scene may be received, observed, or captured by a device. A grid of voxels may then be generated based on the depth image such that the depth image may be downsampled. A background included in the grid of voxels may also be removed to isolate one or more voxels associated with a foreground object such as a human target. A location or position of one or more extremities of the isolated human target may be determined and a model may be adjusted based on the location or position of the one or more extremities.

Human tracking system

An electronic device includes a modulated light projector to project an infrared light pattern and an image sensor. The image sensor includes an array of photosensors and a filter array overlying the array of photosensors. The filter array comprises a mosaic of color filter elements and infrared filter elements, each color filter element overlying a corresponding photosensor of a first subset of photosensors and each infrared filter element overlying a corresponding photosensor of a second subset of photosensors. The electronic device further includes a controller to, in a first mode, control the image sensor to capture a color image using the first subset of photosensors and, in a second mode, control the image sensor to capture a depth image representing a reflection of the infrared light pattern using the second subset of photosensors.

Growing community of inventors

Mountain View, CA, United States of America

Johnny Chung Lee