Qualcomm Incorporated

Giuliano Maciocci

Paul Mabbutt

Andrew J Everitt

James Joseph Mulholland

David T Berry

Arpit Mittal

Michael L Tunmer

Louise M Auger

Methods, systems, computer-readable media, and apparatuses for selecting an Augmented Reality (AR) object on a head mounted device (HMD) are presented. In some embodiments, an HMD may define a Region-of-Interest (ROI) based on a gesture formed by at least one hand of a user. Subsequently the HMD may display to the user a shape on the HMD. In at least one arrangement, the shape outlines the ROI. Additionally, the HMD may display to the user a plurality of AR objects, each of the plurality of AR objects being associated with a target inside the ROI. Furthermore, the HMD may reduce the size of the ROI based on a first movement of the at least one hand of the user to form a reduced-sized ROI. In at least one arrangement, the reduced-sized ROI is used to select a specific AR object from the plurality of AR objects.

Human-body-gesture-based region and volume selection for HMD

Embodiments of the present invention are directed toward enabling a user to quickly interact with a graphical user interface (GUI) displayed by the HMD. Utilizing techniques provided herein, a hand or other object can be used to select visual elements displayed by the HMD. The visual elements can be located within larger active regions, allowing the user to more easily select a visual element by selecting the active region in which the visual element is disposed.

User interface interaction for transparent head-mounted displays

Methods, systems, computer-readable media, and apparatuses for providing intuitive, functional, and convenient ways of enabling a user of a head-mounted display unit or another augmented reality enabled device to interact with various user interfaces and other features provided by such a unit or device are presented. In some embodiments, a computing device, such as a head-mounted display unit, may receive camera input of a scene. Subsequently, the computing device may identify at least one reference object in the scene, for example, based on detecting one or more rectangles in the received camera input. The computing device then may receive input that defines a surface segment relative to the at least one reference object. Thereafter, the computing device may render the surface segment.

Augmented reality surface segmentation

Systems and methods are disclosed for target-based AR devices to perform low-power front-end passive scanning of targets to alert users to AR content linked to any image targets the users may be viewing. Passive scanning by AR devices relieves users of the need to manually activate a camera for AR image target identification, and helps to identify image targets the users may be unknowingly viewing. To conserver power, AR devices may autonomously activate a camera to perform an exploratory scan when the AR devices detect from users' movement patterns that users may be interested in certain targets or is in a state of attentiveness. AR devices may identify one or more image targets from the exploratory scans. If users elect to interact with the AR content, AR devices may activate the camera to perform a full capture or real-time tracking of the image targets to augment the AR content.

Methods and apparatus for augmented reality target detection

A head mounted device provides an immersive virtual or augmented reality experience for viewing data and enabling collaboration among multiple users. Rendering images in a virtual or augmented reality system may include capturing an image and spatial data with a body mounted camera and sensor array, receiving an input indicating a first anchor surface, calculating parameters with respect to the body mounted camera and displaying a virtual object such that the virtual object appears anchored to the selected first anchor surface. Further operations may include receiving a second input indicating a second anchor surface within the captured image that is different from the first anchor surface, calculating parameters with respect to the second anchor surface and displaying the virtual object such that the virtual object appears anchored to the selected second anchor surface and moved from the first anchor surface.

Anchoring virtual images to real world surfaces in augmented reality systems

System and methods are disclosed to selectively project individual UI elements of display images from stereoscopic HMDs as monocular or stereoscopic images. An UI element may be tagged for rendering as a monocular image or as stereoscopic images with a stereo separation to create a perception of depth. Tagging of UI elements as monocular images or stereoscopic images may be based on whether the UI elements are persistent elements. Persistent elements present information to the user but do not require the user's focused attention. Persistent elements may be rendered as monocular images to allow the user to focus on other UI elements or on the real world while maintaining awareness of information presented by the persistent elements. In contrast, non-persistent UI elements may require the user's focused attention. Non-persistent UI elements may be rendered as stereoscopic images for projection at a specific depth plane to invite the user's focus.

Methods and apparatus for displaying images on a head mounted display

Embodiments of the present invention are directed toward allowing a see-through HMD to intelligently react to sensor and other input to adjust how visual elements showing video are displayed. The HMD can react to different sensor input in different ways, enabling the HMD to provide graduated responses based on the degree and type of sensor input received. These responses may further be based on a determination of the context of the user to help ensure an intelligent, appropriate response.

Reactive user interface for head-mounted display

A head mounted device provides an immersive virtual or augmented reality experience for viewing data and enabling collaboration among multiple users. Rendering images in a virtual or augmented reality system may include performing operations for capturing an image of a scene in which a virtual object is to be displayed, recognizing a body part present in the captured image, and adjusting a display of the virtual object based upon the recognized body part. The rendering operations may also include capturing an image with a body mounted camera, capturing spatial data with a body mounted sensor array, recognizing objects within the captured image, determining distances to the recognized objects within the captured image, and displaying the virtual object on a head mounted display.

Selective hand occlusion over virtual projections onto physical surfaces using skeletal tracking

Described herein is a system and method for displaying a stereoscopic plenoptic image or video to a user, with eye-tracking capabilities. A display device displays a plenoptic image or video on a screen, and focuses the plenoptic image or video at the depth plane corresponding to the eye-coordinates of the user's gaze. The focused depth plane in the displayed plenoptic frame updates as the frames progress in the video stream, thereby constantly re-focusing the depth plane of the currently displayed plenoptic frame depending on the location of the user's gaze.

System and method for generating 3-D plenoptic video images

A head mounted device provides an immersive virtual or augmented reality experience for viewing data and enabling collaboration among multiple users. Rendering images in a virtual or augmented reality system may include capturing an image and spatial data with a body mounted camera and sensor array, receiving input indicating a first anchor surface, calculating parameters with respect to the body mounted camera and displaying a virtual object such that the virtual object appears anchored to the selected first anchor surface. Further rendering operations may include receiving a second input indicating a second anchor surface within the captured image that is different from the first anchor surface, calculating parameters with respect to the second anchor surface and displaying the virtual object such that the virtual object appears anchored to the selected second anchor surface and moved from the first anchor surface.

System for the rendering of shared digital interfaces relative to each user's point of view

A user interface (UI) of a head mounted display (HMD) is provided that allows a user to access one or more persistent data elements that are otherwise outside the user's initial field of view by using a head movement, such as a head tilt (i.e., movement about a horizontal axis) and/or rotation (i.e., movement about a vertical axis). Embodiments also can provide for further movement and/or other manipulation of data of persistent data elements with further detected movement of the user's head.

User interface for a head mounted display

Apparatus and methods for extending services of a user device include identifying a first application resident on a user device having a first service that utilizes a first data type. Further, the apparatus and methods include identifying a second application of a plurality of applications having a second service that utilizes a second data type. Additionally, the apparatus and methods include matching a compatibility of a selected one of the first service or the first data type of the first application with a respective one of the second service or the second data type of the second application.

Apparatus and methods of extending application services

A method and apparatus are presented for the presentation and activation of contextual actions for interpreted content. In one aspect, keywords are recognized from an existing webpage, re-processed into a second webpage, and presented via a browser. The indicators are selectable and may invoke functionality resident on the wireless device within one or more native applications.

Growing community of inventors

Duxford, United Kingdom

Giuliano Maciocci