Magic Leap, Inc.

Amazon Technologies, Inc.

Siddharth Choudhary

Prateek Singhal

Samuel A Miller

Robert Blake Taylor

Ashwin Swaminathan

Ali Shahrokni

Anush Mohan

Eran Guendelman

Xuan Zhao

Rafael Domingos Torres

Daniel Olshansky

Jehangir Tajik

Christian Ivan Robert Moore

Lomesh Agarwal

Joel David Holder

Hélder Toshiro Suzuki

Jeremy Dwayne Miranda

Siddharth Mahendran

Khushi Gupta

Shiyu Dong

Divya Ramnath

Arumugam Kalai Kannan

Hiral Honar Barot

Michael Harold Liebenow

Nicholas Atkinson Kramer

Nitesh Sekhar

Manushree Gangwar

Kenneth William Tossell

Amit Kumar Agrawal

Antonio Criminisi

Prakash Ramu

Brandon Michael Smith

Jinjin Li

Durga Venkata Kiran Yakkala

Ganesh Subramanian Iyer

Hirai Honar Barot

Siddarth Mahendran

This document describes scene understanding for cross reality systems using occupancy grids. In one aspect, a method includes recognizing one or more objects in a model of a physical environment generated using images of the physical environment. For each object, a bounding box is fit around the object. An occupancy grid that includes a multiple cells is generated within the bounding box around the object. A value is assigned to each cell of the occupancy grid based on whether the cell includes a portion of the object. An object representation that includes information describing the occupancy grid for the object is generated. The object representations are sent to one or more devices.

Scene understanding using occupancy grids

A cross reality system that provides an immersive user experience by storing persistent spatial information about the physical world that one or multiple user devices can access to determine position within the physical world and that applications can access to specify the position of virtual objects within the physical world. Persistent spatial information enables users to have a shared virtual, as well as physical, experience when interacting with the cross reality system. Further, persistent spatial information may be used in maps of the physical world, enabling one or multiple devices to access and localize into previously stored maps, reducing the need to map a physical space before using the cross reality system in it. Persistent spatial information may be stored as persistent coordinate frames, which may include a transformation relative to a reference orientation and information derived from images in a location corresponding to the persistent coordinate frame.

Cross reality system

Described are systems and methods to determine one or more body dimensions of a body based on a processing of one or more two-dimensional images that include a representation of the body. Body dimensions include any length, circumference, etc., of any part of a body, such as shoulder circumference, chest circumference, waist circumference, hip circumference, inseam length, bicep circumference, leg circumference, etc.

Body dimensions from two-dimensional body images

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scalable three-dimensional (3-D) object recognition in a cross reality system. One of the methods includes maintaining object data specifying objects that have been recognized in a scene. A stream of input images of the scene is received, including a stream of color images and a stream of depth images. A color image is provided as input to an object recognition system. A recognition output that identifies a respective object mask for each object in the color image is received. A synchronization system determines a corresponding depth image for the color image. A 3-D bounding box generation system determines a respective 3-D bounding box for each object that has been recognized in the color image. Data specifying one or more 3-D bounding boxes is received as output from the 3-D bounding box generation system.

Scalable three-dimensional object recognition in a cross reality system

An augmented reality viewing system is described. A local coordinate frame of local content is transformed to a world coordinate frame. A further transformation is made to a head coordinate frame and a further transformation is made to a camera coordinate frame that includes all pupil positions of an eye. One or more users may interact in separate sessions with a viewing system. If a canonical map is available, the earlier map is downloaded onto a viewing device of a user. The viewing device then generates another map and localizes the subsequent map to the canonical map.

Growing community of inventors

San Jose, CA, United States of America

Siddharth Choudhary