Microsoft Technology Licensing, LLC

Other

Robert Matthew Craig

Craig C Peeper

Joseph C Bertolami

Johnny Hsienchow Lee

Ryan M Geiss

Jamie Daniel Joseph Shotton

Tommer Leyvand

Mark James Finocchio

Momin Mahmoud Al-Ghosien

Michael Alan Dougherty

Matthew L Bronder

Oliver M Williams

Matt Bronder

Samuel A Mann

John Allen Tardif

William D Devlin

Casey Leon Meekhof

Ketan K Dalal

Vladimir Tankovich

William James Staples

Bhaven P Dedhia

Varsha Jayasimha

Stephen L Rakonza

Mohit Srivastava

Richard A Ersek

Adam Green

Sing Bing Kang

Jonathan E Lange

Matthew William Lee

Dax H Hawkins

Radu Rares Palanca

Joe Bertolami

Patrick Orville Cook

Anton Rakovchuk

A candidate human head is found in depth video using a head detector. A head region of light intensity video is spatially resolved with a three-dimensional location of the candidate human head in the depth video. Facial recognition is performed on the head region of the light intensity video using a face recognizer.

Identification using depth-based head-detection data

Example embodiments of the present disclosure provide techniques for receiving measurements from one or more inertial sensors (i.e. accelerometer and angular rate gyros) attached to a device with a camera or other environment capture capability. In one embodiment, the inertial measurements may be combined with pose estimates obtained from computer vision algorithms executing with real time camera images. Using such inertial measurements, a system may more quickly and efficiently obtain higher accuracy orientation estimates of the device with respect to an object known to be stationary in the environment.

Augmented reality computing with inertial sensors

Systems and methods are disclosed for generating stereoscopic images for a user based on one or more images captured by one or more scene-facing cameras or detectors and the position of the user's eyes or other parts relative to a component of the system as determined from one or more images captured by one or more user-facing detectors. The image captured by the scene-facing detector is modified based on the user's eye or other position. The resulting image represents the scene as seen from the perspective of the eye of the user. The resulting image may be further modified by augmenting the image with additional images, graphics, or other data. Stereoscopic mechanisms may also be adjusted or configured based on the location or the user's eyes or other parts.

Systems and methods for generating stereoscopic images

A computer system comprises a hardware interface, a computer-memory device, an update engine, and a posture-recognition engine. The hardware interface is configured to receive depth video of an environment from a camera. The computer-memory device stores a background model of the environment preservable over a reboot of the computer system, the background model including a plurality of trusted coordinates derived from the depth video. The update engine is configured to update the background model, including moving a trusted coordinate to greater depth if an observed pixel is behind the trusted coordinate over a first duration, but retaining the trusted coordinate if the observed pixel depth is in front of the trusted coordinate over the first duration. The posture-recognition engine is configured to recognize posture of a user in front of a background portion of the video, which is bounded by the trusted coordinates of the background model.

Background model for user recognition

A human subject is tracked within a scene of an observed depth image supplied to a general-purpose body-part tracker. The general-purpose body-part tracker is retrained for a specific scenario. The general-purpose body-part tracker was previously trained using supervised machine learning to identify one or more general-purpose parameters to be used by the general-purpose body-part tracker to track a human subject. During a retraining phase, scenario data is received that represents a human training-subject performing an action specific to a particular scenario. One or more special-purpose parameters are identified from the processed scenario data. The special-purpose parameters are selectively used to augment or replace one or more general-purpose parameters if the general-purpose body-part tracker is used to track a human subject performing the action specific to the particular scenario.

Scenario-specific body-part tracking

A method of tracking a subject includes receiving from a source a depth image of a scene including the subject. The depth image includes a depth for each of a plurality of pixels. The method further includes identifying pixels of the depth image that image the subject and deriving from the identified pixels of the depth image one or more machine readable data structures representing the subject as a model including a plurality of shapes.

Pose tracking pipeline

Systems and methods for unifying coordinate systems in an augmented reality application or system are disclosed. User devices capture an image of a scene, and determine a location based on the scene image. The scene image may be compared to cartography data or images to determine the location. User devices may propose an origin and orientation or transformation data for a common coordinate system and exchange proposed coordinate system data to agree on a common coordinate system. User devices may also transmit location information to an augmented reality system that then determines an a common coordinate system and transmits coordinate system data such as transformation matrices to the user devices. Images presented to users may be adjusted based on user device locations relative to the coordinate system.

Systems and methods for unifying coordinate systems in augmented reality applications

A method and apparatus for rendering the lighting of virtual objects in an augmented reality display. The method includes determining local and ambient light sources based on data provided by one or more light sensors. The light in the physical lighting environment is accounted for by attributing the light to local light sources and/or ambient light sources. A synthesized physical lighting environment is constructed based on the light characteristics of the local and/or ambient light sources, and is used in properly rendering virtual objects in the augmented reality display.

Augmented lighting environments

A human subject is tracked within a scene of an observed depth image supplied to a pose tracking pipeline. An indication of a problem state is received from the pose tracking pipeline, and an identification of the problem state is supplied to the pose tracking pipeline. A virtual skeleton is received from the pose tracking pipeline that includes a plurality of skeletal points defined in three-dimensions. The pose tracking pipeline selects a three-dimensional position of at least one of the plurality of skeletal points in accordance with the identification of the problem state supplied to the pose-tracking pipeline.

Problem states for pose tracking pipeline

A method of tracking a target includes receiving from a source a depth image of a scene including the human subject. The depth image includes a depth for each of a plurality of pixels. The method further includes identifying pixels of the depth image that belong to the human subject and deriving from the identified pixels of the depth image one or more machine readable data structures representing the human subject as a body model including a plurality of shapes.

Growing community of inventors

Bellevue, WA, United States of America

Robert Matthew Craig