Google Inc.

Mahesh Ramachandran

Chao X Guo

Qiyue Zhang

Luca Ballan

Hongsheng Yu

Fengtao Fan

Konstantine Nicholas John Tsotsos

Jie Zhang

A tracking system and associated algorithms are disclosed that can provide a user with a continuous, reset-free augmented reality (AR) experience. When the user wears an AR headset equipped with a camera and an inertial measurement unit (IMU), motion sensor data from the IMU can be combined with image data from the camera to create a device pose, representing a position and an orientation of the headset relative to its environment. In some implementations, when a reset occurs, a six-degrees-of-freedom (6DoF) algorithm can be configured to support the pose until a re-initialization is completed. In some implementations, a neural network can be used to correct for IMU integration drifts in the 6DoF algorithm. In some implementations, the IMU-based 6DoF uses a neural network that exploits the device's past motion to infer its future motion.

Pose algorithm for continuous AR experiences

Techniques of providing continual augmented reality navigation include combining first and second 6DoF poses from, respectively, a localization from a map in a vicinity of a location to an image and from predicted 6DoF poses based on IMU data. The first 6DoF pose is based on a visual positioning system that uses map data from a vicinity of a location combined with an image from the world-facing camera of the AR smartglasses that provides 6DoF pose in world coordinates independent of the IMU. The second 6DoF pose is based on compensated rotational velocity and acceleration measurements derived from IMU intrinsic values.

Map-aided inertial odometry with neural network for augmented reality devices

Techniques of providing continual augmented reality navigation include combining first and second 6DoF poses from, respectively, a localization from a map in a vicinity of a location to an image and from predicted 6DoF poses based on IMU data.

Growing community of inventors

Dublin, CA, United States of America

Mahesh Ramachandran