Magic Leap, Inc.

Daniel Olshansky

Rafael Domingos Torres

Joel David Holder

Ali Shahrokni

Xuan Zhao

Ashwin Swaminathan

Anush Mohan

Christian Ivan Robert Moore

Samuel A Miller

Robert Blake Taylor

David Geoffrey Molyneaux

Eran Guendelman

Prateek Singhal

Jehangir Tajik

Frank Thomas Steinbrücker

Lomesh Agarwal

Hélder Toshiro Suzuki

Siddharth Choudhary

Mohan Babu Kandra

Yilun Cao

David Paul Pena

Jeremy Dwayne Miranda

Hiral Honar Barot

Michael Harold Liebenow

Nicholas Atkinson Kramer

Ben Weisbih

Gilboa Shveki

Keng-Sheng Lin

Kenneth William Tossell

Ofer Kapota

Michael Loyal Kerr

Hirai Honar Barot

John Campbell

A cross reality system enables any of multiple types of devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. Devices of any type, with native hardware and software configured for augmented reality operations may be configured to work with the cross reality system by incorporating components that interface between the native AR framework of the device and the cloud-based localization service. These components may present position information about the device in a format recognized by the localization service. Additionally, these components may filter or otherwise process perception data provided by the native AR framework to increase the accuracy of localization.

Cross reality system supporting multiple device types

Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object, upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.

Cross reality system for large scale environment reconstruction

A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. Both stored maps and tracking maps used by portable devices may have wireless fingerprints associated with them. The portable devices may maintain wireless fingerprints based on wireless scans performed repetitively, based on one or more trigger conditions, as the devices move around the physical world. The wireless information obtained from these scans may be used to create or update wireless fingerprints associated with locations in a tracking map on the devices. One or more of these wireless fingerprints may be used when a previously stored map is to be selected based on its coverage of an area in which the portable device is operating. Maintaining wireless fingerprints in this way provides a reliable and low latency mechanism for performing map-related operations.

Cross reality system with wireless fingerprints

Disclosed herein are systems and methods for setting, accessing, and modifying user privacy settings using a distributed ledger. In an aspect, a system can search previously stored software contracts to locate an up-to-date version of a software contract associated with a user based on a request for access to user data for the particular user. Then, the system determines that the user data is permitted to be shared. The system transmits, to a data virtualization platform, instructions to extract encrypted user data from a data platform. The system can then make available, to a data verification system, a private encryption key and details associated with the software contract to verify that the private encryption key and the user data match. Then the system transmits, to the data virtualization platform, the private encryption key so that the data virtualization platform can decrypt the encrypted user data.

User data management for augmented reality using a distributed ledger

A cross reality system enables any of multiple devices to efficiently and accurately access previously stored maps and render virtual content specified in relation to those maps. The cross reality system may include a cloud-based localization service that responds to requests from devices to localize with respect to a stored map. The request may include one or more sets of feature descriptors extracted from an image of the physical world around the device. Those features may be posed relative to a coordinate frame used by the local device. The localization service may identify one or more stored maps with a matching set of features. Based on a transformation required to align the features from the device with the matching set of features, the localization service may compute and return to the device a transformation to relate its local coordinate frame to a coordinate frame of the stored map.

Cross reality system with localization service

A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

Cross reality system

Various techniques pertaining to methods, systems, and computer program products a spatial persistence process that places a virtual object relative to a physical object for an extended-reality display device based at least in part upon a persistent coordinate frame (PCF). A determination is made to decide whether a drift is detected for the virtual object relative to the physical object. upon or after detection of the drift or deviation, the drift or deviation is corrected at least by updating a tracking map into an updated tracking map and further at least by updating the persistent coordinate frame (PCF) based at least in part upon the updated tracking map, wherein the persistent coordinate frame (PCF) comprises six degrees of freedom relative to the map coordinate system.

Various techniques pertaining to methods, systems, and computer program products identify, for a first portable device at a first location in a three-dimensional (3D) physical environment, a first sparse map, the first portable device connected to a remote computing system for accessing sharable world data of the 3D physical environment that comprises a physical object, wherein the plurality electronic devices include the plurality of portable devices having respective sparse maps installed thereupon. A second sparse map is identified for a second portable device connected to the remote computing system for accessing the sharable world data. The second portable device performs content persistence for a virtual content relative to the physical object at least by using a merged map determined by merging a version of the first sparse map and a version of the second sparse map based at least in part upon one or more persistence coordinate frames.

An extended or cross reality system includes a computing device communicably connected to a plurality of portable electronic devices via a network component, a repository accessible by the computing device and the plurality of portable electronic devices, and a dense map merge component. The extended or cross reality system determines a representation for multiple portions of a 3D environment based at least in part upon on a set of dense maps received from the plurality of portable devices, wherein the set of dense maps is grouped into multiple subgroups based at least in part upon pose data pertaining to the set of dense maps or surface information in the set of dense maps. The extended or cross reality system storing the representation as at least a portion of a shared persistent dense map.

An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.

Growing community of inventors

Mountain View, CA, United States of America

Daniel Olshansky