Microsoft Technology Licensing, LLC

Gavin Dean Lazarow

Tushar Cyril Bhatnagar

Jeroen Vanturennout

Ethan Eade

David Fields

Jonathan Lyons

Gabriel Takacs

Ali Reza Emami

Jonathan David Hildebrandt

Skyler Mark Goodell

Frederik Schaffalitzky

Gregg Robert Wygonik

David Maxwell Gedye

Jonathan Paul Rosenberg

Jared Adam Brown

Michael Grabner

Jason Randall Wadsworth

Dan Hou

Evan Lincoln Tice

Alex Christopher Turner

Lejie Xu

Richard Renchi Ma

Neena Kamath

Sunil Sudhakar Pandith

Yanshu Fan

Kerri Irene Higgins Sheldon

Erik Manfred Schulz

A computing device and method are provided for transmitting a relevant subset of map data, called a neighborhood, to enable mutual spatial understanding by multiple display devices around a target virtual location to display a shared hologram in the same exact location in the physical environment at the same moment in time. The computing device may comprise a processor, a memory operatively coupled to the processor, and an anchor transfer program stored in the memory and executed by the processor.

Sharing neighboring map data across devices

One example provides a computing device configured to capture, via the camera, first image data imaging a first physical world location, create a first spatial representation of the first physical world location based on the first image data, receive a user input defining a pose of a first virtual spatial anchor point relative to a feature imaged in the first image data, track user movement to a second physical world location, capture second image data imaging the second physical world location, receive a user input defining a pose of a second virtual spatial anchor point relative to a feature imaged in the second image data, and send, to a remote computing device, data representing the first spatial representation, the pose of first spatial anchor point, the second spatial representation, the pose of second spatial anchor point, and a positional relationship between first and second spatial anchor points.

Connecting spatial anchors for augmented reality

Features of the present disclosure solve the above-identified problem by implementing remote localization techniques that enable coordination between multi-user display devices. Specifically, the remote localization techniques identify user location (or 'position') in the virtual reality (VR)/augmented reality (AR) scene using a key-frame that is subset of available information. Thus, when a first display device is uncertain regarding its position within a VR/AR scene, the first display may generate, for example, a single key-frame that is shared between one or more second display devices such that the receiving display device(s) may locate the key-frame with respect to a spatial anchor within the client's map, identify the user's location within the VR/AR scene, and transmit the location back to the first HMD device that created the key-frame. Based on the location information, the first HMD device may synchronize with the second HMD device within a shared scene or map.

Peer to peer remote localization for devices

A first display device and method are provided for accelerating the coarse relocalization process of the first display device by generating a session-specific identifier and sending it to a second display device, which transmits the identifier to a cloud service, which returns at least a portion of a map data set corresponding to the identifier. The returned map data set is then stitched into the local map data of the first display device to create an integrated map, which is used to render one or a plurality of holograms. The first display device may comprise a processor, a memory operatively coupled to the processor, and a fingerprint program stored in the memory and executed by the processor.

Coarse relocalization using signal fingerprints

A computing device and method are provided for accelerating the coarse relocalization process of the computing device by generating a current fingerprint using signal data detected by sensors, statistically analyzing the current fingerprint for proximity to candidate fingerprints in fingerprint data to generate a ranked list, and subsequently retrieving map data corresponding to the closest matching fingerprints in the ranked list. The computing device may comprise a processor, a memory operatively coupled to the processor, and a fingerprint program stored in the memory and executed by the processor.

A computing device is provided, including a display configured to display a plurality of holograms superimposed upon a physical environment. The computing device may further comprise a processor configured to store in non-volatile memory a representation of the physical environment, including a plurality of hologram anchors indicating locations at which the holograms are displayed. The processor may store a priority level of each hologram anchor, wherein each priority level is selected from a plurality of priority levels including a high priority level and a low priority level, and wherein at least one hologram anchor has the low priority level. The processor may determine that a total size of the plurality of hologram anchors exceeds a predetermined size threshold. The processor may, for at least one hologram anchor assigned the low priority level, delete that hologram anchor from the representation of the physical environment.

Hologram anchor prioritization

Embodiments that relate to sharing mixed reality experiences among multiple display devices are disclosed. In one embodiment, a method includes receiving current versions of a plurality of data subtypes geo-located at a keyframe location. A world map data structure is updated to include the current versions, and a neighborhood request including the keyframe location is received from a display device. Based on the keyframe location, an identifier and current version indicator for each data subtype is provided to the device. A data request from the device for two or more of the data subtypes is received, and the two or more data subtypes are prioritized based on a priority hierarchy. Based on the prioritization, current versions of the data subtypes are sequentially provided to the device for augmenting an appearance of a mixed reality environment.

Mixed reality data collaboration

Embodiments extend functionality of a first application with actions from one or more second applications specifically applicable to a content item selected by a user. The selected content item belongs to a category. A description of the selected content item distinguishes the selected content item from other content items in the same category. At least one of a plurality of second applications is selected based on a comparison between the description and application data associated with the second applications. The computing device identifies, to the user, the actions from the second applications applicable to the selected content item. Upon selection of an action, the computing device executes the corresponding second application and provides output data within the context of the first application, or otherwise integrated with the first application experience. In some embodiments, the computing device offers the user execution of a third party application specifically applicable to the selected content item.

Providing item specific functionality via service-assisted applications

Growing community of inventors

Redmond, WA, United States of America

Gavin Dean Lazarow