Microsoft Technology Licensing, LLC

Other

Zsolt Mathe

Charles Claudius Marais

Ryan M Geiss

Zhengyou Zhang

Todd R Manion

Craig C Peeper

Ashraf A Michail

Tommer Leyvand

Mark James Finocchio

Alex A Kipman

Mark Stefan Grossman

J Andrew Goossen

Szymon Piotr Stachniak

Johnny Chung Lee

Casey Leon Meekhof

Prafulla Jinendra Masalkar

Jason Matthew Gould

Ning Xu

Leonardo G Del Castillo

Michael E Pietraszak

Georg Klein

Andrew Martin Pearson

Marcelo Lopez Ruiz

Joe Bertolami

Eric Langlois

Ed Pinto

Leonardo Del Castillo

Examples are disclosed that relate to depth-aware late-stage reprojection. One example provides a computing system configured to receive and store image data, receive a depth map for the image data, processing the depth map to obtain a blurred depth map, and based upon motion data, determine a translation to be made to the image data. Further, for each pixel, the computing system is configured to translate an original ray extending from an original virtual camera location to an original frame buffer location to a reprojected ray extending from a translated camera location to a reprojected frame buffer location, determine a location at which the reprojected ray intersects the blurred depth map, and sample a color of a pixel for display based upon a color corresponding to the location at which the reprojected ray intersects the blurred depth map.

Depth-aware reprojection

A depth image of a scene may be received, observed, or captured by a device. The depth image may then be analyzed to determine whether the depth image includes a human target. For example, the depth image may include one or more targets including a human target and non-human targets. Each of the targets may be flood filled and compared to a pattern to determine whether the target may be a human target. If one or more of the targets in the depth image includes a human target, the human target may be scanned. A skeletal model of the human target may then be generated based on the scan.

Body scan

A depth image of a scene may be received, observed, or captured by a device. A human target in the depth image may then be scanned for one or more body parts such as shoulders, hips, knees, or the like. A tilt angle may then be calculated based on the body parts. For example, a first portion of pixels associated with an upper body part such as the shoulders and a second portion of pixels associated with a lower body part such as a midpoint between the hips and knees may be selected. The tilt angle may then be calculated using the first and second portions of pixels.

Systems and methods for detecting a tilt angle from a depth image

Example apparatus and methods concern an improved immersive experience for a video gamer that is provided by controlling a game based on the three dimensional location and orientation of a control and display device held by or otherwise associated with the gamer. The location is determined from data comprising a three dimensional position and an orientation of a portion of a player in a three dimensional space associated with a computerized game. The facing and rotation of the device is determined as a function of both the location of the device and the orientation of the device. The orientation may be determined by data from motion sensors in or on the device. Example apparatus and methods control the computerized game based, at least in part, on the position of the device, the facing of the device, and the rotation of the device.

Locating and orienting device in space

An image such as a depth image of a scene may be received, observed, or captured by a device. The image may then be processed. For example, the image may be downsampled, a shadow, noise, and/or a missing potion in the image may be determined, pixels in the image that may be outside a range defined by a capture device associated with the image may be determined, a portion of the image associated with a floor may be detected. Additionally, a target in the image may be determined and scanned. A refined image may then be rendered based on the processed image. The refined image may then be processed to, for example, track a user.

Systems and methods for processing an image for target tracking

A depth image of a scene may be observed or captured by a capture device. The depth image may include a human target and an environment. One or more pixels of the depth image may be analyzed to determine whether the pixels in the depth image are associated with the environment of the depth image. The one or more pixels associated with the environment may then be discarded to isolate the human target and the depth image with the isolated human target may be processed.

Environment and/or target segmentation

A method and system are disclosed for automatic instrumentation that modifies a video game's shaders at run-time to collect detailed statistics about texture fetches such as MIP usage. The tracking may be transparent to the game application and therefore not require modifications to the application. In an embodiment, the method may be implemented in a software development kit used to record and provide texture usage data and optionally generate a report.

Automatic real-time shader modification for texture fetch instrumentation

A system and method are disclosed for calibrating a depth camera in a natural user interface. The system in general obtains an objective measurement of true distance between a capture device and one or more objects in a scene. The system then compares the true depth measurement to the depth measurement provided by the depth camera at one or more points and determines an error function describing an error in the depth camera measurement. The depth camera may then be recalibrated to correct for the error. The objective measurement of distance to one or more objects in a scene may be accomplished by a variety of systems and methods.

In-home depth camera calibration

A method of compressing a digital image defined by a plurality of pixel values in each of one or more channels includes adjusting each pixel value in each of the one or more channels by an average pixel value for that channel. The method further includes splitting each adjusted channel into one or more image blocks, and converting each image block into a frequency block that is a frequency-domain representation of that image block.

Digital image compression and decompression

An image such as a depth image of a scene may be received, observed, or captured by a device and a model of a user in the depth image may be generated. The background of a received depth image may be removed to isolate a human target in the received depth image. A model may then be adjusted to fit within the isolated human target in the received depth image. To adjust the model, a joint or a bone may be magnetized to the closest pixel of the isolated human target. The joint or the bone may then be refined such that the joint or the bone may be further adjusted to a pixel equidistant between two edges the body part of the isolated human target where the joint or bone may have been magnetized.

Growing community of inventors

Issaquah, WA, United States of America

Zsolt Mathe