Apple Inc.

Metaio Gmbh

Stefan Misslinger

Peter Meier

Lejing Wang

Anton Fedosov

Selim Ben Himane

Ben Blachnitzky

Michael Kuhn

Justin D Stoyles

Thibaut Weise

Reinhard Klapfer

A method for representing virtual information in a view of a real environment is provided that includes: providing a system setup including at least one display device, wherein the system setup is adapted for blending in virtual information on the display device in at least part of the view, determining a position and orientation of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and a position of the at least one item of virtual information, determining whether the position of the item of virtual information is inside a 2D or 3D geometrical shape, determining a criterion which is indicative of whether the built-in real object is at least partially visible or non-visible in the view of the real environment, and blending in the at least one item of virtual information on the display device in at least part of the view of the real environment.

Method for representing virtual information in a real environment

In one implementation, a method includes: determining an interpupillary distance (IPD) measurement for a user based on a function of depth data obtained by the depth sensor and image data obtained by the image sensor; and calibrating a head-mounted device (HMD) provided to deliver augmented reality/virtual reality (AR/VR) content by setting one or more presentation parameters of the HMD based on the IPD measurement in order to tailor one or more AR/VR displays of the HMD to a field-of-view of the user.

Method and device for improved interpupillary distance measurement

The disclosure relates to a method and system for presenting at least part of an image of a real object in a view of a real environment, comprising providing a first image of at least part of a real object captured by a first camera, determining at least part of the real object in the first image as an object image area, determining a first 3D plane relative to the first camera, the first camera being at a position where the first image is captured, providing at least one image feature related to the real object in the first image, providing at least one first ray passing an optical center of the first camera being at a position where the first image is captured and the at least one image feature, determining, according to a first plane normal direction of the first 3D plane, at least one first angle between the first 3D plane and the at least one first ray, providing a second image of a real environment captured by a second camera, determining a second 3D plane relative to the second camera, the second camera being at a position where the second image is captured, mapping at least part of the object image area to a target space in a view of the real environment captured at a viewpoint, wherein the at least one image feature is mapped to at least one target position in a coordinate system of the view, determining at least one second angle between the second 3D plane and at least one second ray passing the viewpoint of capturing the view and the at least one target position according to a second plane normal direction of the second 3D plane, wherein the at least one image feature is mapped to the at least one target position such that a difference between the at least one first angle and the at least one second angle is minimized.

Method and system for presenting at least part of an image of a real object in a view of a real environment, and method and system for selecting a subset of a plurality of images

There is disclosed a method and mobile device for displaying points of interest in a view of a real environment displayed on a screen of the mobile device with a functionality for interaction with a user, which comprises the steps of: capturing an image of the real environment or a part of the real environment using a camera, determining at least one point of interest related to the real environment, determining an image position of the at least one point of interest in the image, displaying at least part of the image on at least part of the screen, overlaying a computer-generated indicator with the at least part of the image on the screen at a screen position according to the image position of the at least one point of interest, displaying a computer-generated virtual object related to the at least one point of interest on the screen at a screen position determined according to the screen position of the computer-generated indicator and which is adjacent to a bottom edge of the screen, displaying a visually perceivable relation indication indicative of a relation between the computer-generated virtual object and the computer-generated indicator. The mobile device may perform an action related to the at least one point of interest if at least part of the computer-generated virtual object displayed on the screen is touched.

Method for representing points of interest in a view of a real environment on a mobile device and mobile device therefor

Method for determining the pose of a camera with respect to an object of a real environment for use in authoring/augmented reality application that includes generating a first image by the camera capturing a real object of a real environment, generating first orientation data from at least one orientation sensor associated with the camera or from an algorithm which analyzes the first image for finding and determining features which are indicative of an orientation of the camera, allocating a distance of the camera to the real object, generating distance data indicative of the allocated distance, determining the pose of the camera with respect to a coordinate system related to the real object of the real environment using the distance data and the first orientation data. May be performed with reduced processing requirements and/or higher processing speed, in mobile device such as mobile phones having display, camera and orientation sensor.

Methods and systems for determining the pose of a camera with respect to at least one object of a real environment

A method for representing virtual information in a view of a real environment is provided that includes the following steps: providing a system setup comprising at least one display device, determining a position of a viewing point relative to at least one component of the real environment, providing a geometry model of the real environment, providing at least one item of virtual information and its position, determining a visualization mode of blending in the at least one item of virtual information on the display device according to the position of the viewing point and the geometry model, calculating a ray between the viewing point and the item of virtual information, and determining a number of boundary intersections by the ray, wherein if the number of boundary intersections is less than 2, the item of virtual information is blended in a non-occlusion mode, otherwise in an occlusion mode.

Method for determining the pose of a camera with respect to at least one object of a real environment for use in authoring/augmented reality application that includes generating a first image by the camera capturing a real object of a real environment, generating first orientation data from at least one orientation sensor associated with the camera or from an algorithm which analysis the first image for finding and determining features which are indicative of an orientation of the camera, allocating a distance of the camera to the real object, generating distance data indicative of the allocated distance, determining the pose of the camera with respect to a coordinate system related to the real object of the real environment using the distance data and the first orientation data. May be performed with reduced processing requirements and/or higher processing speed, in mobile device such as mobile phones having display, camera and orientation sensor.

Growing community of inventors

San Jose, CA, United States of America

Stefan Misslinger