Magic Leap, Inc.

Yu-Hsiang Huang

Igor Napolskikh

Ashwin Swaminathan

Martin Georg Zahnert

Dominik Michael Kasper

Joao Antonio Pereira Faro

Omid Ghasemalizadeh

Benjamin Langmann

Sergiu Sima

Evan Gregory Levine

Christoph Tobler

Manel Quim Sanchez Nicuesa

Blazej Marek Czuprynski

Ankur Gupta

Mohamed Souiai

Etienne Gregoire Grossman

João Antonio Pereira Faro

Manuel Quim Sanchez Nicuesa

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for calibrating an augmented reality device using camera and inertial measurement unit data. In some implementations, a bundle adjustment process jointly optimizes or estimates states of the augmented reality device. The process can use, as input, visual and inertial measurements as well as factory-calibrated sensor extrinsic parameters. The process performs bundle adjustment and uses non-linear optimization of estimated states constrained by the measurements and the factory calibrated extrinsic parameters. The process can jointly optimize inertial constraints, IMU calibration, and camera calibrations. Output of the process can include most likely estimated states, such as data for a 3D map of an environment, a trajectory of the device, and/or updated extrinsic parameters of the visual and inertial sensors (e.g., cameras and IMUs).

Joint camera and inertial measurement unit calibration

Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the preintegration term, and the wearable head device presents the virtual content based on the position of the device.

Dual IMU SLAM

Examples of the disclosure describe systems and methods for presenting virtual content on a wearable head device. In some embodiments, a state of a wearable head device is determined by minimizing a total error based on a reduced weight associated with a reprojection error. A view reflecting the determined state of the wearable head device is presented via a display of the wearable head device. In some embodiments, a wearable head device calculates a first preintegration term and second preintegration term based on the image data received via a sensor of the wearable head device and the inertial data received via a first IMU and a second IMU of the wearable head device. The wearable head device estimates a position of the device based on the first and second preintegration terms, and the wearable head device presents the virtual content based on the position of the device.

Gravity estimation and bundle adjustment for visual-inertial odometry

Growing community of inventors

Mountain View, CA, United States of America

Yu-Hsiang Huang