Microsoft Technology Licensing, LLC

Ecole Nationale De L'aviation Civile

Ecole National De L'aviation Civile

Christophe Hurter

Eyal Ofek

Michel Pahud

Nathalie Henry Riche

Steven Mark Drucker

Michael Traoré Sompagnimdi

Sasa Junuzovic

Methods and systems for rendering augmented reality display data to locations of a physical presentation environment based on a presentation configuration are provided. A physical presentation environment configuration may be accessed that includes locations of a physical presentation environment for mapping augmented reality display data. The augmented reality display data may include a plurality of augmented reality objects that are rendered for display. Presentation attributes of the augmented reality display data may be used in conjunction with the presentation configuration for mapping and rendering the augmented reality display data. The rendered augmented reality display data may be dynamically interactive, and may be generated based on previous presentation configurations, mapping preferences, mapping limitations, and/or other factors.

Presenting augmented reality display data in physical presentation environments

Various systems and techniques for intuitively and non-obstructively obtaining awareness of fellow collaborator views and/or statuses in an augmented reality environment are disclosed. An interaction is detected between a first and second HMD, both collaborative participants of a shared dataset represented in an augmented reality environment. The detected interaction can cause the first HMD to request state information associated with the second HMD. The second HMD can process the request in accordance with a privacy policy to generate a set of visual data as a response to the request. The second HMD can communicate the generated set of visual data to the first HMD, such that the first HMD can render the received set of visual data as additional computer-generated object(s) to be displayed concurrently with its own augmented view. The additional computer-generated object(s) can be positioned in accordance with a real-world or virtualized position of the second HMD, such that the now-visible state information associated with the second HMD does not obstruct a view of the second HMD's user.

Intuitive augmented reality collaboration on visual data

In various embodiments, methods and systems for implementing a multi-device mixed interactivity system are provided. The interactivity system includes paired mixed-input devices for interacting and controlling virtual objects. In operation, a selection profile associated with a virtual object is accessed. The selection profile is generated based on a selection input determined using real input associated with a selection device and virtual input associated with a mixed-reality device. The selection device has a first display and the mixed-reality device has a second display that both display the virtual object. An annotation input for the virtual object based on a selected portion corresponding to the selection profile is received. An annotation profile based on the annotation input is generated. The annotation profile includes annotation profile attributes for annotating a portion of the virtual object. An annotation of the selected portion of the virtual reality object is caused to be displayed.

Annotation using a multi-device mixed interactivity system

In various embodiments, computerized systems and methods for displacement oriented interaction with objects in a computer-mediated environment are provided. In one embodiment, the system detects a wearable device moved with a displacement transversal to a longitudinal axis of the wearable device. If the system determines that the displacement is within a displacement range associated with an actionable item, the system may select the actionable item or activate an operation associated with the actionable item, such as modifying an object in the computer-mediated reality environment.

Displacement oriented interaction in computer-mediated reality

Systems and methods for enabling user-interactions with virtual objects (VOs) included in immersive environments (IEs) are provided. A head-mounted display (HMD) device is communicatively coupled with a hover-sensing (HS) device, via a communication session. The HMD device provides an IE to a wearer by displaying a field-of-view (FOV) that includes a VO. The user executes user-interactions, such as 2D and/or 3D hand gestures, fingertip gestures, multi-fingertip gestures, stylus gestures, hover gestures, and the like. The HS device detects the user-interactions and generates interaction data. The interaction data is provided to the HMD device via the communication session. The HMD device updates the FOV and/or the VO based on the interaction data. A physical overlay that includes a 3D protrusion is coupled with the HS device. The overlay is transparent to the hover-sensing capabilities of the HS device. The protrusion provides tactile feedback to the user for the user-interactions.

Hover-based user-interactions with virtual objects within immersive environments

In various embodiments, methods and systems for implementing a multi-device mixed interactivity system are provided. The interactivity system includes paired mixed-input devices for interacting and controlling virtual objects. In operation, a selection input associated with a virtual object is accessed. The selection input is based on real input associated with a selection device and virtual input associated with a mixed-reality device. The selection device has a first display that displays the virtual object and the mixed-reality device has a second display that displays the virtual object. A selection profile is generated based on the selection input. The selection profile comprises one or more selection profile attributes for isolating a portion of the virtual object. A selected portion of the virtual object is determined based on the selection profile. The selected portion of the virtual reality object is caused to be displayed on the first display of the selection device.

Selection using a multi-device mixed interactivity system

A mechanism is provided for exploring three dimensional environments such as those generated from X-ray and tomography scans, in such a way as to 'see round' obstacles to an article of interest, without degrading the overall context of an article of interest in terms of it position in relationship to other articles, and to the viewpoint of the user. This is achieved by defining a light guide curve leading to the user's viewpoint, with respect to which ray tracing is performed to define the image displayed to the user whereby rays passing within a predetermined distance of the light guide curve at a relative angle within a predetermined range thereto are deviated so as to conform therewith to a degree proportional to its distance therefrom.

Selective display in an environment defined by a data set

Systems and techniques from displaying virtual representations of real-world spaces and objects in various environments are disclosed. A source environment at a first location can be scanned by a head-mounted display (HMD) device to generate three-dimensional datasets corresponding to the physical environment at the first location. The three-dimensional datasets can include detected physical properties associated with the physical environment. At a second location, the HMD can re-create the source environment, and render for display a virtual representation of the physical environment based on the three-dimensional datasets, where the virtual representation of the source environment is rendered to maintain any one of the detected physical properties associated with the physical environment. Further, at the second location, the HMD can enable a user to view one or more physical objects within the virtual representation of the physical environment such that the physical object is perceived to be within the source environment.

Virtually representing spaces and objects while maintaining physical properties

In a crowded representation of a virtual three dimensional space defined in terms of voxels, an object of interest will often be occluded by one or more objects of varying densities between the virtual camera defining the user's point of view, and the object of interest. To automatically identify an optimal camera position, an number of candidate positions are considered, for example situated at the vertices of a regular polyhedron centred on the object of interest. For each of these candidate positions, a ray is cast towards the object of interest, and the occlusion for each intervening voxel is determined as the product of that voxel's density, and a density transfer function. The virtual camera position corresponding to the least occluded path is then selected as the new point of view.

Point of view selection in virtual 3D environment

Objects in a voxel based computer generated three dimensional environment are identified by traversing adjacent voxels meeting a predetermined criterion with respect to a scalar metadata value associated with each voxel, such as opacity or density. These adjacent voxels may be explored in accordance with a tree-crawling algorithm such as a breadth first or depth first algorithm. Once all adjacent cells meeting the criterion are identified, these are determined to represent a discrete object, and displayed as such. The starting point for the traversing process may be the voxel closest to a virtual camera position along the line of sight of that virtual camera meeting the criterion.

Object definition in virtual 3D environment

In various embodiments, computerized methods and systems for syndicating direct and indirect interactions with objects in a computer-mediated environment to facilitate precise interactions with the objects in the computer-mediated environment are provided. The system detects a direct interaction with an object in the computer-mediated reality environment. The direct interaction may be a natural or hypernatural interaction. Subsequently, the system may determine various options of indirect interaction with the object related to the direct interaction. The indirect interaction may be generated by a controller. Upon receiving an indirect interaction, the system may modify the object based on the syndication of the direct interaction and the indirect interaction.

Syndication of direct and indirect interactions in a computer-mediated reality environment

A zoom operation in an eye tracking based graphical user interface, where the zoom operation, as initiated by a scroll wheel etc. takes place with respect to the focus of the user's attention as determined on the basis of eye tracking data. Mechanisms for determining the point of attention based on a sliding window recording point of regard and subject to a variety of weighting functions are proposed.

Growing community of inventors

Toulouse, France

Christophe Hurter