Microsoft Technology Licensing, LLC

Microsoft Corporation

Eyal Ofek

Billy P Chen

Michel Pahud

Gur Kimchi

Michael Jack Sinclair

Christian Holz

Andrew David Wilson

Nathalie Henry Riche

Christophe Hurter

Hrvoje Benko

Blaise Aguera Y Arcas

Avi Bar-Zeev

Stephen L Lawler

Pragyana K Mishra

Michael Kroepfl

Steven Mark Drucker

Gonzalo Alberto Ramos

Brian C Beckman

Kenneth Paul Hinckley

Michael F Cohen

Pasquale DeMaio

Sasa Junuzovic

Feng-Hsiung Hsu

Jeffrey Darren Couckuyt

Bo Zhang

Pushmeet Kohli

Jiahe Helen Wang

Ying-Qing Xu

Jaron Zepel Lanier

Benjamin Livshits

David Alexander Molnar

Shai Herzog

Eric J Horvitz

Douglas Christopher Burger

Jin Li

David Nister

Baining Guo

Shahram Izadi

Daniel Buchmueller

Ricky Dean Welsh

Yonatan Wexler

Mar Gonzalez Franco

Boris Epshtein

Ido Omer

Leonard Smith, Jr

Blaise H Aguera Y Arcas

Ran Gal

David Maxwell Gedye

Edward Sean Lloyd Rintel

Lior Shapira

Scott Vernon Fynn

Jonathan Robert Dughi

Elad Gerson

Kenn Cartier

Pourang Polad Irani

Julian R Walker

Alexander Nikolavich Moshchuk

Roberto R Molinari

Dan Hou

Molly Jane Nicholas

Payod Panda

Mark Ruane Dawson

Eric Jordan Gonzalez

Mahdi Azmandian

Mark Hancock

Nicolas Villar

Hugues Herve Hoppe

Thomas Charles Oliver

Blaise Hilary Aguera-Arcas

Xin Wu

Nathan Silberman

John Clavin

Doug Burger

Daniel Allen Rosenfeld

Megan Lesley Tedesco

John Helmes

Margus Veanes

Yasuyuki Matsushita

Johannes P Kopf

Tadayoshi Kohno

William Arthur Stewart Buxton

Eric Michael Whitmire

Kori M Inkpen

Franziska Roesner

Joseph Schwartz

William Buxton

Steve Lombardi

Inrak Choi

Anne H Loomis

Alan Paulin

Tanya L Swartz

Yoichi Ochiai

Zhenyu Qian

Neil Dalchau

Limor Lahiani

Karan Ashok Ahuja

John-Michael Wiley

Gil Shklarski

Bernard L Johnston

Timothy Scott Saponas

A computer-implemented method of generating a mixed reality workflow is described. The method comprises identifying a series of tasks and generating an input task-to-object-mapping by analyzing data that defines a process performed by a first user interacting with objects in a first location. The input task-to-object-mapping that maps each task from the series of tasks to an object used in the respective task. A task-specific non-spatial characteristic of each object in the input task-to-object-mapping is determined and used to map each object in the input task-to-object-mapping to a candidate object identified at a second location to generate an output task-to-object-mapping. The series of tasks, location data defining a position of each candidate object in the second location and output task-to-object-mapping are used to generate a mapped workflow which is then output to a device in the second location.

Mixed reality workflow generation

Aspects relate to observing various activities, interactions, behaviors, and other factors associated with a data exchange and creating one or more markers based on significant details associated with the observance. The one or more markers are retained and selectively rendered as a function of one or more conditions that should be satisfied before the marker is presented to the user. Some markers can contain parameters that should be satisfied in order for the marker to be considered complete. If a parameter is not satisfied, subsequent markers can be created as a function of the rendered marker. The subsequent markers can be rendered when a condition associated with the subsequent marker is satisfied.

Automatic generation of markers based on social interaction

A sensor device is described herein. The sensor device includes a multi-dimensional optical sensor and processing circuitry, wherein the multi-dimensional optical sensor generates images and the processing circuitry is configured to output data that is indicative of hemodynamics of a user based upon the images. The sensor device is non-invasive, and is able to be incorporated into wearable devices, thereby allowing for continuous output of the data that is indicative of the hemodynamics of the user.

Sensor device

The present concepts relate to haptic controllers. In one example the haptic controller can include first and second capstans rotationally secured to a base and an energy storage mechanism connected between the first and second capstans. The example haptic controller can also include a user engagement assembly secured to the first capstan and a controller configured to control rotational forces imparted on the user engagement assembly by controlling rotational friction experienced by the first and second capstans.

Haptic controller

Systems and methods for representing two-dimensional representations as three-dimensional avatars are provided herein. In some examples, one or more input video streams are received. A first subject, within the one or more input video streams, is identified. Based on the one or more input video streams, a first view of the first subject is identified. Based on the one or more input video streams, a second view of the first subject is identified. The first subject is segmented into a plurality of planar object. The plurality of planar objects are transformed with respect to each other. The plurality of planar objects are based on the first and second views of the first subject. The plurality of planar objects are output in an output video stream. The plurality of planar objects provide perspective of the first subject to one or more viewers.

Representing two dimensional representations as three-dimensional avatars

The present concepts relate to devices that can approximate virtual objects when touched by a user. One example device can include a location assembly configured to sense a location of the device on a surface and configured to move the robotic device on the surface and a shape assembly secured to the location assembly and configured to adjust a height and a pitch of an upper surface of the robotic device.

Mobile haptic robots

Methods and systems for rendering augmented reality display data to locations of a physical presentation environment based on a presentation configuration are provided. A physical presentation environment configuration may be accessed that includes locations of a physical presentation environment for mapping augmented reality display data. The augmented reality display data may include a plurality of augmented reality objects that are rendered for display. Presentation attributes of the augmented reality display data may be used in conjunction with the presentation configuration for mapping and rendering the augmented reality display data. The rendered augmented reality display data may be dynamically interactive, and may be generated based on previous presentation configurations, mapping preferences, mapping limitations, and/or other factors.

Presenting augmented reality display data in physical presentation environments

In some examples, a surface, such as a desktop, in front or around a portable electronic device may be used as a relatively large surface for interacting with the portable electronic device, which typically has a small display screen. A user may write or draw on the surface using any object such as a finger, pen, or stylus. The surface may also be used to simulate a partial or full size keyboard. The use of a camera to sense the three-dimensional (3D) location or motion of the object may enable use of above-the-surface gestures, entry of directionality, and capture of real objects into a document being processed or stored by the electronic device. One or more objects may be used to manipulate elements displayed by the portable electronic device.

Three-dimensional object tracking to augment display area

The present concepts relate to devices that can employ graspable controllers that can be employed in various scenarios, such as virtual reality scenarios and augmented reality scenarios. One example device can include multiple expansion assemblies having independently adjustable girths. The multiple expansion assemblies can be stacked adjacent to one another along an axis. A controller can be configured to expand or contract the girths of the expansion assemblies to collectively approximate girths of an object.

Multilayer controller

Aspects of the present disclosure relate to headset virtual presence techniques. For example, a participant of a communication session may not have an associated video feed, for example as a result of a user preference to disable video communication or a lack of appropriate hardware. Accordingly, a virtual presence may be generated for such a non-video participant, such that the non-video participant may be represented within the communication session similar to video participants. The virtual presence may be controllable using a headset device, for example such that movements identified by the headset device cause the virtual presence to move. In some instances, user input may be received to control emotions conveyed by the virtual presence, for example specifying an emotion type and/or intensity.

Headset virtual presence

Various systems and techniques for intuitively and non-obstructively obtaining awareness of fellow collaborator views and/or statuses in an augmented reality environment are disclosed. An interaction is detected between a first and second HMD, both collaborative participants of a shared dataset represented in an augmented reality environment. The detected interaction can cause the first HMD to request state information associated with the second HMD. The second HMD can process the request in accordance with a privacy policy to generate a set of visual data as a response to the request. The second HMD can communicate the generated set of visual data to the first HMD, such that the first HMD can render the received set of visual data as additional computer-generated object(s) to be displayed concurrently with its own augmented view. The additional computer-generated object(s) can be positioned in accordance with a real-world or virtualized position of the second HMD, such that the now-visible state information associated with the second HMD does not obstruct a view of the second HMD's user.

Intuitive augmented reality collaboration on visual data

Aspects of the present disclosure relate to computing device headset input. In examples, sensor data from one or more sensors of a headset device are processed to identify implicit and/or explicit user input. A context may be determined for the user input, which may be used to process the identified input and generate an action that affects the behavior of a computing device accordingly. As a result, the headset device is usable to control one or more computing devices. As compared to other wearable devices, headset devices may be more prevalent and may therefore enable more convenient and more intuitive user input beyond merely providing audio output.

Computing device headset input

Various embodiments are provided herein for tracking a user's physical environment, to facilitate on-the-fly blending of a virtual environment with detected aspects of the physical environment. Embodiments can be employed to facilitate virtual roaming by compositing virtual representations of detected physical objects into virtual environments. A computing device coupled to a HMD can select portions of a depth map generated based on the user's physical environment, to generate virtual objects that correspond to the selected portions. The computing device can composite the generated virtual objects into an existing virtual environment, such that the user can traverse the virtual environment while remaining aware of their physical environment. Among other things, the computing device can employ various blending techniques for compositing, and further provide image pass-through techniques for selective viewing of the physical environment while remaining fully-immersed in virtual reality.

Blending virtual environments with situated physical reality

Growing community of inventors

Redmond, WA, United States of America

Eyal Ofek