Magic Leap, Inc.

Danying Hu

Ashwin Swaminathan

Ali Shahrokni

Eran Guendelman

Miguel Andres Granados Velasquez

Javier Victorio Gomez Gonzalez

Mukta Prasad

Tomasz Jan Malisiewicz

Daniel DeTone

A cross reality system enables any of multiple devices to efficiently and accurately access previously persisted maps, even maps of very large environments, and render virtual content specified in relation to those maps. The cross reality system may quickly process a batch of images acquired with a portable device to determine whether there is sufficient consistency across the batch in the computed localization. Processing on at least one image from the batch may determine a rough localization of the device to the map. This rough localization result may be used in a refined localization process for the image for which it was generated. The rough localization result may also be selectively propagated to a refined localization process for other images in the batch, enabling rough localization processing to be skipped for the other images.

Cross reality system with fast localization

A deep neural network provides real-time pose estimation by combining two custom deep neural networks, a location classifier and an ID classifier, with a pose estimation algorithm to achieve a 6D0F location of a fiducial marker. The locations may be further refined into subpixel coordinates using another deep neural network. The networks may be trained using a combination of auto-labeled videos of the target marker, synthetic subpixel corner data, and/or extreme data augmentation. The deep neural network provides improved pose estimations particularly in challenging low-light, high-motion, and/or high-blur scenarios.

Deep neural network pose estimation system

Growing community of inventors

Sunnyvale, CA, United States of America

Danying Hu