Snap Inc.

Xsens Holding B.v.

Jeroen Diederik Hol

Matthias Kalkgruber

Georg Halmetschlager-Funek

Daniel Wagner

Clemens Birklbauer

Per Johan Slycke

Gerald Nilles

Kai Zhou

Mathieu Emmanuel Vignau

Erick Mendez Mendez

Giovanni Bellusci

Qi Qi

Niall Murphy

Daniel Wolf

Hendrik Johannes Luinge

Nikolaj Kuntner

Freerk Dijkstra

Hendrick J Luinge

A method for dynamically initializing a 3 degrees of freedom (3DOF) tracking device is described. In one aspect, the method includes accessing a gyroscope signal from a gyroscope of the 3DOF tracking device, accessing an accelerometer signal from an accelerometer of the 3DOF tracking device, determining an initial state includes a combination of an initial orientation, an initial position, and an initial velocity of the 3DOF tracking device, the initial state indicating a starting condition of the 3DOF tracking device, integrating the gyroscope signal and the accelerometer signal to obtain orientation and position signals using the initial state, and refining an inclination signal of the orientation signal using the position signal.

Dynamic initialization of 3DOF AR tracking system

An eyewear device with flexible frame for Augmented Reality (AR) is disclosed. At least two sensors and a display are mounted on the flexible frame. When in use, the real time geometry of the eyewear device may change from factory calibrated geometry, resulting in low quality AR rendering. A modeling module is provided to model the real time geometry of the eyewear device on the fly using sensor information of the at least two sensors. The modeled real time geometry is then provided to a rendering module to accurately display the AR to the user.

Geometry modeling of eyewear devices with flexible frames

A method for improving the startup time of a six-degrees of freedom tracking system is described. An augmented reality system receives a device initialization request and activates a first set of sensors in response to the device initialization request. The augmented reality system receives first tracking data from the first set of sensors. The augmented reality system receives an augmented reality experience request and in response to the augmented reality request, causes display of a set of augmented reality content items based on the first tracking data and simultaneously activates a second set of sensors. The augmented reality system receives second tracking data from the activated second set of sensors. The augmented reality system updates the display of the set of augmented reality content items based on the second tracking data.

Reducing startup time of augmented reality experience

Systems and methods of generating ground truth datasets for producing virtual reality (VR) experiences, for testing simulated sensor configurations, and for training machine-learning algorithms. In one example, a recording device with one or more cameras and one or more inertial measurement units captures images and motion data along a real path through a physical environment. A SLAM application uses the captured data to calculate the trajectory of the recording device. A polynomial interpolation module uses Chebyshev polynomials to generate a continuous time trajectory (CTT) function. The method includes identifying a virtual environment and assembling a simulated sensor configuration, such as a VR headset. Using the CTT function, the method includes generating a ground truth output dataset that represents the simulated sensor configuration in motion along a virtual path through the virtual environment. The virtual path is closely correlated with the motion along the real path as captured by the recording device. Accordingly, the output dataset produces a realistic and life-like VR experience. In addition, the methods described can be used to generate multiple output datasets, at various sample rates, which are useful for training the machine-learning algorithms which are part of many VR systems.

Generating ground truth datasets for virtual reality experiences

A method and apparatus of tracking poses of a rolling-shutter camera in an augmented reality (AR) system is provided. The method and apparatus use camera information and inertial sensor readings from Inertial Measurement Unit (IMU) to estimate the pose of the camera at a reference line. Thereafter, relative pose changes at scanlines may be calculated using the inertial sensor data. The estimated reference pose of the camera is then further refined based on the visual information from the camera, the relative pose changes and the optimized reference line pose of a previous image. Thereafter, the estimate of the scanline poses may be updated using the relative pose changes obtained in the earlier steps.

Pose tracking for rolling shutter camera

In an embodiment, a method and system are provided for determining a pedestrian position via dead reckoning on a portable device carried by the pedestrian (or 'user'). The device has at least a processor, a 3D accelerometer, and a 3D gyroscope. The 3D accelerometer, and a 3D gyroscope are sampled to calculate device orientation, and then 3D accelerometer samples are gathered during a sensor time frame and an average step frequency is calculated. The user's speed and direction of movement are estimated and a device velocity is calculated relative to a prior device velocity via dead-reckoning. The estimated device velocity is corrected for drift and the pedestrian position is calculated based on the corrected estimated current device velocity.

Pedestrian dead reckoning position tracker

Growing community of inventors

Hengelo, Netherlands

Jeroen Diederik Hol