Niantic, Inc.

Niantic Spatial, Inc.

Ben Benfold

Victor Adrian Prisacariu

Daniel Knoblauch

An online system uses a pose prior model and a pose objective function to estimate the pose of a client device. A pose prior model is a model for prior information known about client devices and their poses without reference to a particular client device and its pose data. The online system receives pose data from a client device and computes an estimated pose for the client device based on the received pose data, the pose prior model, and a generated initial candidate pose for the client device. The online system uses these as inputs to a pose objective function and optimizes the pose objective function to estimate a pose for the client device. The online system transmits this estimated pose to the client device, and may use the estimated pose as the pose for the client device for the purposes of delivering content to the user.

Estimating pose for a client device using a pose prior model

The present disclosure describes a method for estimating a pose of a client device using a magnetic field vector map. The method includes receiving a plurality of magnetic field measurements from a plurality of client devices, each magnetic field measurement describing a magnetic field vector at a geographic location. The method further includes grouping the magnetic field measurements into one or more region groups, aggregating the magnetic field measurements in each region group to generate a probability distribution of magnetic field vectors associated with the geographic region, determining a magnetic field vector within each geographic region, and generating a magnetic field vector map. Based on the magnetic field vector map, the method may include estimating a pose of a client device based on a user location of the client device and received magnetic field vector from the client device.

Magnetic field vector map for orientation determination

The present disclosure describes approaches to camera re-localization that improve the speed and accuracy with which pose estimates are generated by fusing output of a computer vision algorithm with data from a prior model of a geographic area in which a user is located. For each candidate pose estimate output by the algorithm, a game server maps the estimate to a position on the prior model (e.g., a specific cell on a heatmap-style histogram) and retrieves a probability corresponding to the mapped position. A data fusion module fuses, for each candidate pose estimate, a confidence score generated by the computer vision algorithm with the location probability from the prior model to generate an updated confidence score. If an updated confidence score meets or exceeds a score threshold, a re-localization module initiates a location-based application (e.g., a parallel reality game) based on the associated candidate pose estimate.

Refining camera re-localization determination using prior pose model

The disclosure describes a method for calibrating a magnetic sensor of a client device. The method may include receiving a set of magnetic field measurements, each of which includes a device location, an orientation of the client device, and an observed magnetic field vector measured by the magnetic sensor. The method may include computing a device correction vector for the client device based on the set of magnetic field measurements. For each magnetic field measurement, the method includes determining a world magnetic field vector at the device location of the magnetic field measurement, computing an expected measured magnetic field vector at the device location, accessing an estimated device correction vector for the client device, computing an expected adjusted vector for the client device, comparing the observed magnetic field vector associated with the magnetic field measurement and the expected adjusted vector, and computing the device correction vector based on the comparison.

Magnetic sensor calibration using VIO system

A method of determining a position for a virtual object is described. A location of a client device is determined, and, based on the determined location a set of map segments is retrieved. A virtual object is determined to be displayed on the client device. Relation vectors between the virtual object and each map segment of the retrieved set of map segments are obtained. Each relation vector is weighted based on object parameters of the virtual object. A position to display the virtual object is determined based on the weighted relation vectors. The virtual object is provided for display on the client device as the determined position.

Maintaining object alignment in 3D map segments

An augmented reality ('AR') device applies smooth correction methods to correct the location of the virtual objects presented to a user. The AR device may apply an angular threshold to determine whether a virtual object can be moved from an original location to a target location. An angular threshold is a maximum angle by which a line from the AR device to the virtual object can change within a timestep. Similarly, the AR device may apply a motion threshold, which is a maximum on the distance that a virtual object's location can be corrected based on the motion of the virtual object. Furthermore, the AR device may apply a pixel threshold to the correction of the virtual object's location. A pixel threshold is a maximum on the distance that a pixel projection of the virtual object can change based on the virtual object's change in location.

Smooth object correction for augmented reality devices

The present disclosure describes approaches to camera re-localization that improve the accuracy of re-localization determinations by performing simulated consistency checks for three-dimensional maps. Client devices associated with users of a location-based application transmit image scans to a game server, which divides the received scan data into mapping sets used to generate 3D maps of environments and validation sets used to test the accuracy of the maps. To perform the testing, the game server identifies query scans in the validation set having GPS coordinates within a threshold distance of the mapped location and uses the 3D map of the environment to generate a pose estimate for each frame. The results of the localization queries are analyzed by comparing differences between the localization pose estimates and differences between the poses of independent pairs of frames in the query scan to evaluate the accuracy of the 3D map.

Growing community of inventors

Oxford, United Kingdom

Ben Benfold