Microsoft Technology Licensing, LLC

Other

Shahram Izadi

Stephen Edward Hodges

David Alexander Butler

David Kim

Otmar Hilliges

Jamie Daniel Joseph Shotton

Andrew William Fitzgibbon

Pushmeet Kohli

David G Molyneaux

Nicolas Villar

Hrvoje Benko

Christoph Rhemann

James William Scott

Daniel Allen Rosenfeld

Steven Nabil Bathiche

Stuart Taylor

Toby Leonard Sharp

Cem Keskin

Sing Bing Kang

John Helmes

Kenneth Paul Hinckley

Antonio Criminisi

William Ben Kunz

Duncan Paul Robertson

Sean Ryan Francesco Fanello

Richard Newcombe

Patrick Luke Olivier

Andrew David Wilson

Willem Den Boer

Richard Malcolm Banks

Carsten Curt Eckard Rother

Adam G Kirk

Ian Sands

Oliver Arthur Whyte

Christopher Zach

Malcolm Hall

Malte Hanno Weiss

Vivek Pradeep

Xiang Cao

Behrooz Chitsaz

Shawn R LeProwse

Alban Rrustemi

Jonathan James Taylor

Jiawen Chen

Benjamin Michael Glocker

Harper LaFave

Mike Hazas

David Kirk

Eric J Horvitz

Karlton David Powell

Timothy Andrew Large

Jonathan Westhues

Kurt Jenkins

Andreas Georgiou

Ravin Balakrishnan

Meredith June Morris

F David Jones

Abigail Jane Sellen

Vladimir Tankovich

Simon Stachniak

V Kevin Russ

Emad Barsoum

Michael G Molloy

Armando Garcia-Mendoza

Alex Butler

Chih-Chieh Han

Michael Bleyer

Eyal Ofek

Richard S Szeliski

Billy P Chen

Giora Yahav

Shenlong Wang

David Cohen

Andrew Clark Goris

Charles Lawrence Zitnick, Iii

Michael Isard

Alexandru Octavian Balan

Vishwa Vinay

Jason S Flaks

John Michael Winn

William A S Buxton

Avronil Bhattacharjee

Nicholas Yen-Cherng Chen

Brian L Hastings

Reinhard Sebastian Bernhard Nowozin

Yichen Wei

David Keesu Kim

Asaf Pellman

Oliver M Williams

Paul Ronald Barham

Jamie Shotton

Sebastian Nowozin

Amir Nevet

Peter John Ansell

David Sweeney

Youssef Hamadi

William Buxton

Lucas Bordeaux

Matthew Darius Cook

Adarsh Prakash Murthy Kowdle

Detecting material properties such reflectivity, true color and other properties of surfaces in a real world environment is described in various examples using a single hand-held device. For example, the detected material properties are calculated using a photometric stereo system which exploits known relationships between lighting conditions, surface normals, true color and image intensity. In examples, a user moves around in an environment capturing color images of surfaces in the scene from different orientations under known lighting conditions. In various examples, surfaces normals of patches of surfaces are calculated using the captured data to enable fine detail such as human hair, netting, textured surfaces to be modeled. In examples, the modeled data is used to render images depicting the scene with realism or to superimpose virtual graphics on the real world in a realistic manner.

Using photometric stereo for 3D environment modeling

Camera or object pose calculation is described, for example, to relocalize a mobile camera (such as on a smart phone) in a known environment or to compute the pose of an object moving relative to a fixed camera. The pose information is useful for robotics, augmented reality, navigation and other applications. In various embodiments where camera pose is calculated, a trained machine learning system associates image elements from an image of a scene, with points in the scene's 3D world coordinate frame. In examples where the camera is fixed and the pose of an object is to be calculated, the trained machine learning system associates image elements from an image of the object with points in an object coordinate frame. In examples, the image elements may be noisy and incomplete and a pose inference engine calculates an accurate estimate of the pose.

Camera/object pose from predicted coordinates

The subject disclosure is directed towards controlling the intensity of illumination of a scene or part of a scene, including to conserve illumination power. Quality of depth data in stereo images may be measured with different illumination states; environmental conditions, such as ambient light, natural texture may affect the quality. The illumination intensity may be controllably varied to obtain sufficient quality while conserving power. The control may be directed to one or more regions of interest corresponding to an entire scene or part of a scene.

Environment-dependent active illumination for stereo matching

Scalable volumetric reconstruction is described whereby data from a mobile environment capture device is used to form a 3D model of a real-world environment. In various examples, a hierarchical structure is used to store the 3D model where the structure comprises a root level node, a plurality of interior level nodes and a plurality of leaf nodes, each of the nodes having an associated voxel grid representing a portion of the real world environment, the voxel grids being of finer resolution at the leaf nodes than at the root node. In various examples, parallel processing is used to enable captured data to be integrated into the 3D model and/or to enable images to be rendered from the 3D model. In an example, metadata is computed and stored in the hierarchical structure and used to enable space skipping and/or pruning of the hierarchical structure.

Scalable volumetric 3D reconstruction

Systems and methods for stereo matching based upon active illumination using a patch in a non-actively illuminated image to obtain weights that are used in patch similarity determinations in actively illuminated stereo images is provided. To correlate pixels in actively illuminated stereo images, adaptive support weights computations are used to determine similarity of patches corresponding to the pixels. In order to obtain adaptive support weights for the adaptive support weights computations, weights are obtained by processing a non-actively illuminated ('clean') image.

Active stereo with adaptive support weights from a separate image

The subject disclosure is directed towards active depth sensing based upon moving a projector or projector component to project a moving light pattern into a scene. Via the moving light pattern captured over a set of frames, e.g., by a stereo camera system, and estimating light intensity at sub-pixel locations in each stereo frame, higher resolution depth information at a sub-pixel level may be computed than is captured by the native camera resolution.

Super-resolving depth map by moving pattern projector

In one or more implementations, a static geometry model is generated, from one or more images of a physical environment captured using a camera, using one or more static objects to model corresponding one or more objects in the physical environment. Interaction of a dynamic object with at least one of the static objects is identified by analyzing at least one image and a gesture is recognized from the identified interaction of the dynamic object with the at least one of the static objects to initiate an operation of the computing device.

Gesture recognition techniques

Assisting input from a keyboard is described. In an embodiment, a processor receives a plurality of key-presses from the keyboard comprising alphanumeric data for input to application software executed at the processor. The processor analyzes the plurality of key-presses to detect at least one predefined typing pattern, and, in response, controls a display device to display a representation of at least a portion of the keyboard in association with a user interface of the application software. In another embodiment, a computer device has a keyboard and at least one sensor arranged to monitor at least a subset of keys on the keyboard, and detect an object within a predefined distance of a selected key prior to activation of the selected key. The processor then controls the display device to display a representation of a portion of the keyboard comprising the selected key.

Assisting input from a keyboard

Aspects relate to detecting gestures that relate to a desired action, wherein the detected gestures are common across users and/or devices within a surface computing environment. Inferred intentions and goals based on context, history, affordances, and objects are employed to interpret gestures. Where there is uncertainty in intention of the gestures for a single device or across multiple devices, independent or coordinated communication of uncertainty or engagement of users through signaling and/or information gathering can occur.

Gestures, interactions, and common ground in a surface computing environment

Region of interest detection in raw time of flight images is described. For example, a computing device receives at least one raw image captured for a single frame by a time of flight camera. The raw image depicts one or more objects in an environment of the time of flight camera (such as human hands, bodies or any other objects). The raw image is input to a trained region detector and in response one or more regions of interest in the raw image are received. A received region of interest comprises image elements of the raw image which are predicted to depict at least part of one of the objects. A depth computation logic computes depth from the one or more regions of interest of the raw image.

Depth from time of flight camera

A wearable sensor for tracking articulated body parts is described such as a wrist-worn device which enables 3D tracking of fingers and optionally also the arm and hand without the need to wear a glove or markers on the hand. In an embodiment a camera captures images of an articulated part of a body of a wearer of the device and an articulated model of the body part is tracked in real time to enable gesture-based control of a separate computing device such as a smart phone, laptop computer or other computing device. In examples the device has a structured illumination source and a diffuse illumination source for illuminating the articulated body part. In some examples an inertial measurement unit is also included in the sensor to enable tracking of the arm and hand

Wearable sensor for tracking articulated body-parts

A dual-mode, dual-display shared resource computing (SRC) device is usable to stream SRC content from a host SRC device while in an on-line mode and maintain functionality with the content during an off-line mode. Such remote SRC devices can be used to maintain multiple user-specific caches and to back-up cached content for multi-device systems.

Information transmission based on modal change

Model fitting from raw time of flight image data is described, for example, to track position and orientation of a human hand or other entity. In various examples, raw image data depicting the entity is received from a time of flight camera. A 3D model of the entity is accessed and used to render, from the 3D model, simulations of raw time of flight image data depicting the entity in a specified pose/shape. The simulated raw image data and at least part of the received raw image data are compared and on the basis of the comparison, parameters of the entity are computed.

Model fitting from raw time-of-flight images

Growing community of inventors

Cambridge, United Kingdom

Shahram Izadi