Qualcomm Incorporated

Alessandro Mulloni

Daniel Wagner

Gerhard Reitmayr

Dieter Schmalstieg

Tobias Langlotz

Daniel Knoblauch

Mahesh Ramachandran

Clemens Arth

Prince Gupta

Gerhard Schall

A method of determining a reference coordinate system includes: obtaining information indicative of a direction of gravity relative to a device; and converting an orientation of a device coordinate system using the direction of gravity relative to the device to produce the reference coordinate system. The method may also include setting an origin of the reference coordinate system and/or determining a scale value of the reference coordinate system. The method may also include refining the reference coordinate system.

Reference coordinate system determination

Disclosed are a system, apparatus, and method for depth camera image re-mapping. A depth camera image from a depth camera may be received and a depth camera's physical position may be determined. The depth camera's physical position may be determined relative to an other physical position, such as the physical position of a color camera. The depth camera image may be transformed according to a processing order associated with the other physical position.

Method and apparatus for efficient depth image transformation

One disclosed example method for view independent color equalized 3D scene texturing includes capturing a plurality of keyframes of an object; accessing a 3D representation of the object comprising a surface mesh model for the object, the surface mesh model comprising a plurality of polygons; for each polygon, assigning one of the plurality of keyframes to the polygon based on one or more image quality characteristics associated with a portion of the keyframe corresponding to the polygon; reducing a number of assigned keyframes by changing associations between assigned keyframes; and for each polygon of the surface mesh model having an assigned keyframe: equalizing a texture color of at least a portion of the polygon based at least in part on one or more image quality characteristics of the plurality of keyframes associated with the polygon; and assigning the equalized texture color to the 3D representation of the object.

View independent color equalized 3D scene texturing

Embodiments disclosed pertain to systems, method s and apparatus for the initialization of Computer Vision (CV) applications on user devices (UDs) comprising a camera and a display. In some embodiments, an optimal camera trajectory for initialization of a Computer Vision (CV) application may be determined based on an initial camera pose and an estimated pivot distance. For example, the initial camera pose may be estimated based on a first image captured by the camera. Further, the display may be updated in real-time with an indication of a desired movement direction for the camera. In some embodiments, the indication of desired movement direction may be based, in part, on a current camera pose and the optimal trajectory, where the current camera pose may be estimated based on a current image captured by the camera.

Methods for facilitating computer vision application initialization

A mobile platform maps and tracks the orientation of the mobile platform in real-time as the mobile platform rotates. The mobile platform captures a plurality of camera images as the camera rotates and projects the plurality of camera images consecutively to a panoramic cylindrical map. The map is extended by projecting areas of any camera image that correspond to unmapped portions of the panoramic cylindrical map. Keypoints are extracted from mapped portions of the map and using the keypoints the orientation of the camera is determined. Accumulated error is removed when closing the panoramic cylindrical map by extending the map horizontally beyond 360 degrees to produce overlapping regions and extracting and matching keypoints from the overlapping regions. Additionally, an incorrect estimate of the initial orientation of the map may be corrected by determining the correct orientation and re-projecting the map onto a map with the correct orientation.

Visual tracking using panoramas on mobile devices

Apparatuses and methods for fast visual simultaneous localization and mapping are described. In one embodiment, a three-dimensional (3D) target is initialized immediately from a first reference image and prior to processing a subsequent image. In one embodiment, one or more subsequent reference images are processed, and the 3D target is tracked in six degrees of freedom. In one embodiment, the 3D target is refined based on the processed the one or more subsequent images.

Fast initialization for monocular visual SLAM

Embodiments disclosed facilitate resource utilization efficiencies in Mobile Stations (MS) during 3D reconstruction. In some embodiments, camera pose information for a first color image captured by a camera on an MS may be obtained and a determination may be made whether to extend or update a first 3-Dimensional (3D) model of an environment being modeled by the MS based, in part, on the first color image and associated camera pose information. The depth sensor, which provides depth information for images captured by the camera, may be disabled, when the first 3D model is not extended or updated.

Real-time 3D reconstruction with power efficient depth sensor usage

A mobile platform generates a panoramic map by rotating a camera and stores the panoramic map. The mobile platform performs a computer vision operation on the panoramic map while continuing to generate the panoramic map. The mobile platform may determine the location of the camera when the panoramic map is generated, produce an annotation on the panoramic map and transmit the determined location, annotation, and portion to a storage device. In another example, the mobile platform may receive an annotation, a portion of a source panoramic map associated with the annotation, and a location where the source panoramic map was generated. The mobile platform may compare the portion of the source panoramic map to the panoramic map to identify a position on the panoramic map that corresponds to the portion of the source panoramic map, and display the annotation based on the determined position on the panoramic map.

Online creation of panoramic augmented reality annotations on mobile platforms

A mobile device uses vision and orientation sensor data jointly for six degree of freedom localization, e.g., in wide-area environments. An image or video stream is captured while receiving geographic orientation data and may be used to generate a panoramic cylindrical map of an environment. A bin of model features stored in a database is accessed based on the geographic orientation data. The model features are from a pre-generated reconstruction of the environment produced from extracted features from a plurality of images of the environment. The reconstruction is registered to a global orientation and the model features are stored in bins based on similar geographic orientations. Features from the panoramic cylindrical map are matched to model features in the bin to produce a set of corresponding features, which are used to determine a position and an orientation of the camera.

Sensor-aided wide-area localization on mobile devices

A mobile platform uses orientation sensors and vision-based tracking to track with absolute orientation. The mobile platform generates a panoramic map by rotating a camera, which is compared to an image frame produced by the camera, to determine the orientation of the camera with respect to the panoramic map. The mobile platform also estimates an orientation of the panoramic map with respect to a world reference frame, e.g., magnetic north, using orientation sensors, including at least one accelerometer and a magnetic sensor. The orientation of the camera with respect to the world reference frame is then determined using the orientation of the camera with respect to the panoramic map and the orientation of the panoramic map with respect to the world reference frame. A filter, such as a Kalman filter, provides an accurate and stable estimate of the orientation of the panoramic map with respect to the world reference frame.

Growing community of inventors

Vienna, Austria

Alessandro Mulloni