Microsoft Technology Licensing, LLC

Ran Gal

Eyal Ofek

Lior Shapira

Douglas Christopher Burger

Pushmeet Kohli

Brian Kevin Guenter

Jiahe Helen Wang

Jaron Zepel Lanier

Nathan Silberman

Benjamin Livshits

David Alexander Molnar

Alexander Nikolavich Moshchuk

Alexander Moshchuk

Blair MacIntyre

Joseph Menke

Viktor A Mateevitsi

John Vilk

Wayne Chang

Javier Arturo Porras Luraschi

Gheric Speiginer

Blair Macintyre

Video noise reduction for a video augmented reality system is provided. A head mounted display includes a display unit; a camera for generating frames of display data. A frame store is provided for storing previous frames of displayed information that was sent to the display unit; and a motion processor is provided in communication with the camera, display unit, and the frame store. The motion processor is operable to: identify an area of interest in a current frame of display data; match the area of interest to similar areas in previous frames stored in the frame store; rotate and translate the matched areas of interest from the one or more previous frames stored in the frame store to match the area of interest in the current frame; and average the prior matched areas of interest with the current area of interest to generate a displayed area of interest.

Video noise reduction for video augmented reality system

A 'Layout Optimizer' provides various real-time iterative constraint-satisfaction methodologies that use constraint-based frameworks to generate optimized layouts that map or embed virtual objects into environments. The term environment refers to combinations of environmental characteristics, including, but not limited to, 2D or 3D scene geometry or layout, scene colors, patterns, and/or textures, scene illumination, scene heat sources, fixed or moving people, objects or fluids, etc., any of which may evolve or change over time. A set of parameters are specified or selected for each object. Further, the environmental characteristics are determined automatically or specified by users. Relationships between objects and/or the environment derived from constraints associated with objects and the environment are then used to iteratively determine optimized self-consistent and scene-consistent object layouts. This enables the Layout Optimizer to augment environments with arbitrary content in a structured constraint-based process that adapts to changing scenes or environments.

Layout design using locally satisfiable proposals

A 'Shared Tactile Immersive Virtual Environment Generator' (STIVE Generator) constructs fully immersive shared virtual reality (VR) environments wherein multiple users share tactile interactions via virtual elements that are mapped and rendered to real objects that can be touched and manipulated by multiple users. Generation of real-time environmental models of shared real-world spaces enables mapping of virtual interactive elements to real objects combined with multi-viewpoint presentation of the immersive VR environment to multiple users. Real-time environmental models classify geometry, positions, and motions of real-world surfaces and objects. Further, a unified real-time tracking model comprising position, orientation, skeleton models and hand models is generated for each user. The STIVE Generator then renders frames of the shared immersive virtual reality corresponding to a real-time field of view of each particular user. Each of these frames is jointly constrained by both the real-time environmental model and the unified real-time tracking model.

Shared tactile interaction and user safety in shared space multi-person immersive virtual reality

Mixed reality social interactions are described. Techniques described herein include determining authentication information associated with a mixed reality display device and determining that a content item is visible in a mixed reality environment associated with the mixed reality display device. In an example, a content item may be determined to be visible based at least in part on content data indicating that the content item is owned by the mixed reality display device and/or has been shared with the mixed reality display device. The content data may also indicate an identification of a content item of the plurality of content items, an owner of the content item, and permissions associated with the content item. The techniques further describe causing a graphical representation of the content item to be presented via a display on the mixed reality display device.

Mixed reality social interaction

An 'AR Privacy API' provides an API that allows applications and web browsers to use various content rendering abstractions to protect user privacy in a wide range of web-based immersive augmented reality (AR) scenarios. The AR Privacy API extends the traditional concept of 'web pages' to immersive &#x201c;web rooms&#x201d; wherein any desired combination of existing or new 2D and 3D content is rendered within a user's room or other space. Advantageously, the AR Privacy API and associated rendering abstractions are useable by a wide variety of applications and web content for enhancing the user's room or other space with web-based immersive AR content. Further, the AR Privacy API is implemented using any existing or new web page coding platform, including, but not limited to HTML, XML, CSS, JavaScript, etc., thereby enabling existing web content and coding techniques to be smoothly integrated into a wide range of web room AR scenarios.

Protecting privacy in web-based immersive augmented reality

Surface reconstruction contour completion embodiments are described which provide dense reconstruction of a scene from images captured from one or more viewpoints. Both a room layout and the full extent of partially occluded objects in a room can be inferred using a Contour Completion Random Field model to augment a reconstruction volume. The augmented reconstruction volume can then be used by any surface reconstruction pipeline to show previously occluded objects and surfaces.

Contour completion for augmenting surface reconstructions

Amalgamated maps, comprising interior maps overlaid on venues indicated in general maps can be automatically generated. Initially, interior maps can be obtained through targeted network searches, whose search results can be filtered to retain those that are most likely useable interior maps. A bounding polygon is generated for both interior map and venue exterior from general map. Subsequently, directional histograms representing orientations of lines in bounding polygons are generated and compared to automatically identify a rotation to align the interior map with the venue exterior from the general map. Anchor points are identified to locally deform the interior map, preserving internal structures, to better align with the venue exterior. Once aligned, the interior map can be combined with the general map, forming an amalgamated map. Updated geocoding can be performed based on locations of establishments in the venue as indicated by the interior map.

Automated fitting of interior maps to general maps

Growing community of inventors

Redmond, WA, United States of America

Ran Gal