Varjo Technologies Oy

Vago Technologies Oy

Ari Antti Peuhkurinen

Yuri Popov

Mikko Strandborg

Klaus Melakari

Urho Konttori

Akseli Anttila

Ville Miettinen

Thomas Carlsson

Ville Timonen

Evgeny Zuev

Tomi Lehto

Aleksei Romanov

Oiva Sahlsten

Jaakko Viitanen

Niki Dobrev

Kaj Harri Wikberg

Iurii Popov

Disclosed is a system with server(s) communicably coupled to client device(s). The server(s) is configured to obtain a 3D model of a real-world environment; receive, from client device(s), viewpoint information indicative of a viewpoint from a perspective of which a mixed-reality (MR) image is to be generated; for virtual object(s) to be embedded in MR image, determine portion(s) of the virtual object(s) being occluded by real object(s) present in the real-world environment, based on optical depths determined from 3D model corresponding to a viewpoint, a position at which the virtual object(s) is to be embedded with respect to viewpoint, and at least one of: size of the virtual object(s), shape of the virtual object(s), orientation of the virtual object(s) with respect to viewpoint; and send a remaining portion of the virtual object(s) that is not being occluded to client device(s).

Mixed-reality occlusion using 3D reconstruction

A display system including: display apparatus; display-apparatus-tracking means; input device; processor. The processor is configured to: detect input event and identify actionable area of input device; process display-apparatus-tracking data to determine pose of display apparatus in global coordinate space; process first image to identify input device and determine relative pose thereof with respect to display apparatus; determine pose of input device and actionable area in global coordinate space; process second image to identify user's hand and determine relative pose thereof with respect to display apparatus; determine pose of hand in global coordinate space; adjust poses of input device and actionable area and pose of hand such that adjusted poses align with each other; process first image, to generate extended-reality image in which virtual representation of hand is superimposed over virtual representation of actionable area; and render extended-reality image.

Display systems and methods for aligning different tracking means

A display apparatus including means for tracking pose of user's head, light source(s) and processor configured to: process pose-tracking data to determine position, orientation, velocity and acceleration of head; predict viewpoint and view direction of user in extended-reality environment; determine region of extended-reality environment to be presented, based on viewpoint and view direction; determine sub-region(s) of region whose rendering information is to be derived from previous rendering information of corresponding sub-region(s) of previously-presented region of extended-reality environment; generate rendering information of sub-region(s) based on previous rendering information; send, to rendering server, information indicating remaining sub-regions required to be re-rendered and pose information indicating viewpoint and view direction; receive, from rendering server, rendering information of remaining sub-regions; merge rendering information of sub-region(s) and rendering information of remaining sub-regions to generate image(s); and display image(s) via light source(s).

Display apparatuses and rendering servers incorporating prioritized re-rendering

An imaging system and a method for producing extended-reality images for a display apparatus. The imaging system includes camera and processor. The processor is configured to: control camera to capture image of real-world environment; analyse captured image to identify first image segment representing input device and to determine location of at least one actionable area of input device in first image segment; determine at least one functional element to be presented for the actionable area, the functional element being indicative of at least one of: functionality of the at least one actionable area, status of the at least one actionable area; and process captured image to generate extended-reality image in which the functional element is virtually superimposed over the actionable area of input device or a virtual representation of the actionable area of input device.

Imaging system and method for producing images with virtually-superimposed functional elements

A display apparatus including first and second displays or projectors that display first and second images for first and second eyes, respectively; first and second portions facing first and second eyes, respectively, first and second portions having first (F-F') and second (S-S&#x2032;) optical axes, respectively; means for tracking positions and orientations of first and second eyes relative to corresponding optical axes, respectively; and processor. The processor or external processor obtains current relative positions and orientations of both eyes; determines first and second transformations for first and second input image frames, respectively, given transformation being applied to correct apparent per-pixel distortions produced when given input image frame is displayed; and applies first and second transformations to generate first and second distortion-corrected image frames, respectively, wherein processor renders first and second distortion-corrected image frames.

Display apparatus and method of correcting image distortion therefor

A display apparatus including means for detecting user's gaze direction; first light source; second light source; configuration () of reflective elements to split light to create copies of second image; optical waveguides whose outlets are arranged in tiled layout; opacity masks arranged at inlets of waveguides; and processor. Input image is processed to generate first and second images. Processor is configured to: display first and second images; and for optical waveguides, control respective opacity masks to block respective copies of second image or selectively allow portions of respective copies of second image to pass through towards inlets of waveguides. Light from first light source is optically combined with light exiting outlets of waveguides, to produce output image.

Display apparatuses and methods employing waveguides and opacity masks

A display apparatus including first display or projector for displaying first images for first eye; second display or projector for displaying second images for second eye; first portion and second portion arranged to face first and second eyes; means for tracking poses of first and second eyes relative to first and second optical axes, respectively; and processor. Processor or external processor is configured to: obtain given pose of given eye relative to given optical axis; generate information pertaining to given visual artifact that is formed over given image at image plane when given image is displayed; determine artifact-superposing portion of given image; and process given image based on generated information and artifact-superposing portion, to generate given artifact-compensated image. Processor displays given artifact-compensated image via given display or projector.

Display apparatus and method of compensating for visual artifacts

A display apparatus including means for detecting gaze direction of user; first and second image renderers; processor, processor or external processor processes input image to generate first image and second image, processor renders first and second images simultaneously; configuration of reflective elements arranged to split light emanating from second image renderer into plurality of directions to produce plurality of projections of second image in tiled layout; optical combiner; and opacity mask that is controlled based upon gaze direction to selectively allow given portion of the projections to pass through towards optical combiner, whilst blocking remaining portion of projections. Optical combiner is arranged to optically combine projection of first image with given portion of the projections, to produce on image plane an output image.

Display apparatus and method using reflective elements and opacity mask

A system for facilitating shared rendering between display devices including first display device and second display device that are communicably coupled with first computing device and second computing device, respectively. The system includes means for tracking pose of first display device, means for tracking pose of second display device, image server, first client, second client. First client is configured to: send, to image server, first information indicative of pose of first display device; receive first image frame; render first image frame at first display device; receive second information indicative of pose of second display device; send, to image server, second information; receive second image frame; and send second image frame, wherein second client renders second image frame at second display device.

Systems and methods for facilitating shared rendering

A display apparatus including image renderer, camera and processor. The processor or external processor communicably coupled to said processor is configured to: render at least one extended-reality image during first mode of operation of display apparatus; determine second mode of operation to which display apparatus is to be switched; control camera to capture at least one real-world image of real-world environment; generate at least one composite image from at least one next extended-reality image and at least one real-world image, wherein first portion of at least one composite image is derived from at least one next extended-reality image, and second portion of at least one composite image is derived from at least one real-world image; and render at least one composite image during second mode of operation of display apparatus.

Display apparatus and method for generating and rendering composite images

A display apparatus including display or projector, means for tracking position and orientation of user's head, and processor coupled to display or projector and said means. The processor or external processor communicably coupled to processor is configured to obtain head-tracking data indicative of position and orientation; process head-tracking data to determine current position and orientation of user's head and velocity and/or acceleration with which position and orientation is changing; predict first and second position and orientation of user's head at time tand t, respectively; determine first and second projection matrices to be applied to three-dimensional image data of given frame, respectively; and apply first and second projection matrices to said image data to generate first image frame and second image frame, respectively. Processor is configured to render first and second image frame at time tand t, respectively.

Growing community of inventors

Helsinki, Finland

Ari Antti Peuhkurinen