Magic Leap, Inc.

Arumugam Kalai Kannan

Prateek Singhal

Siddharth Choudhary

Siddharth Mahendran

Khushi Gupta

Shiyu Dong

Divya Ramnath

Manushree Gangwar

Nitesh Sekhar

Siddarth Mahendran

This document describes scene understanding for cross reality systems using occupancy grids. In one aspect, a method includes recognizing one or more objects in a model of a physical environment generated using images of the physical environment. For each object, a bounding box is fit around the object. An occupancy grid that includes a multiple cells is generated within the bounding box around the object. A value is assigned to each cell of the occupancy grid based on whether the cell includes a portion of the object. An object representation that includes information describing the occupancy grid for the object is generated. The object representations are sent to one or more devices.

Scene understanding using occupancy grids

Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for scalable three-dimensional (3-D) object recognition in a cross reality system. One of the methods includes maintaining object data specifying objects that have been recognized in a scene. A stream of input images of the scene is received, including a stream of color images and a stream of depth images. A color image is provided as input to an object recognition system. A recognition output that identifies a respective object mask for each object in the color image is received. A synchronization system determines a corresponding depth image for the color image. A 3-D bounding box generation system determines a respective 3-D bounding box for each object that has been recognized in the color image. Data specifying one or more 3-D bounding boxes is received as output from the 3-D bounding box generation system.

Scalable three-dimensional object recognition in a cross reality system

Growing community of inventors

Sunnyvale, CA, United States of America

Arumugam Kalai Kannan