Qualcomm Incorporated

Tobias Langlotz

Dieter Schmalstieg

Gerhard Reitmayr

Daniel Wagner

Alessandro Mulloni

Hartmut Seichter

Raphael Grasset

Thanh Quoc Nguyen

Denis Kalkofen

Raphael David Andre Grasset

Mathaus Zingerle

Markus Tatzgern

Techniques are presented for constructing a digital representation of a physical environment. In some embodiments, a method includes obtaining image data indicative of the physical environment; receiving gesture input data from a user corresponding to at least one location in the physical environment, based on the obtained image data; detecting at least one discontinuity in the physical environment near the at least one location corresponding to the received gesture input data; and generating a digital surface corresponding to a surface in the physical environment, based on the received gesture input data and the at least one discontinuity.

Structural modeling using depth sensors

A mobile platform maps and tracks the orientation of the mobile platform in real-time as the mobile platform rotates. The mobile platform captures a plurality of camera images as the camera rotates and projects the plurality of camera images consecutively to a panoramic cylindrical map. The map is extended by projecting areas of any camera image that correspond to unmapped portions of the panoramic cylindrical map. Keypoints are extracted from mapped portions of the map and using the keypoints the orientation of the camera is determined. Accumulated error is removed when closing the panoramic cylindrical map by extending the map horizontally beyond 360 degrees to produce overlapping regions and extracting and matching keypoints from the overlapping regions. Additionally, an incorrect estimate of the initial orientation of the map may be corrected by determining the correct orientation and re-projecting the map onto a map with the correct orientation.

Visual tracking using panoramas on mobile devices

A mobile platform generates a panoramic map by rotating a camera and stores the panoramic map. The mobile platform performs a computer vision operation on the panoramic map while continuing to generate the panoramic map. The mobile platform may determine the location of the camera when the panoramic map is generated, produce an annotation on the panoramic map and transmit the determined location, annotation, and portion to a storage device. In another example, the mobile platform may receive an annotation, a portion of a source panoramic map associated with the annotation, and a location where the source panoramic map was generated. The mobile platform may compare the portion of the source panoramic map to the panoramic map to identify a position on the panoramic map that corresponds to the portion of the source panoramic map, and display the annotation based on the determined position on the panoramic map.

Online creation of panoramic augmented reality annotations on mobile platforms

A source video stream is processed to extract a desired object from the remainder of video stream to produce a segmented video of the object. Additional relevant information, such as the orientation of the source camera for each frame in the resulting segmented video of the object, is also determined and stored. During replay, the segmented video of the object, as well as the source camera orientation are obtained. Using the source camera orientation for each frame of the segmented video of the object, as well as target camera orientation for each frame of a target video stream, a transformation for the segmented video of the object may be produced. The segmented video of the object may be displayed over the target video stream, which may be a live video stream of a scene, using the transformation to spatially register the segmented video to the target video stream.

Spatially registered augmented video

A mobile device uses an image-driven view management approach for annotating images in real-time. An image-based layout process used by the mobile device computes a saliency map and generates an edge map from a frame of a video stream. The saliency map may be further processed by applying thresholds to reduce the number of saliency levels. The saliency map and edge map are used together to determine a layout position of labels to be rendered over the video stream. The labels are displayed in the layout position until a change of orientation of the camera that exceeds a threshold is detected. Additionally, the representation of the label may be adjusted, e.g., based on a plurality of pixels bounded by an area that is coincident with a layout position for a label in the video frame.

Growing community of inventors

Graz, Austria

Tobias Langlotz