Varjo Technologies Oy

Ari Antti Erik Peuhkurinen

Klaus Melakari

Oiva Arvo Oskari Sahlsten

Mikko Strandborg

Roope Rainisto

Ville Miettinen

Ville Timonen

Mikko Ollila

Urho Konttori

Kai Inha

Petteri Timonen

Antti Hirvonen

Evgeny Zuev

Joni-Matti Määttä

Jarno Wuolijoki

Timo Suoranta

Niki Dobrev

Perttu Keinänen

Disclosed is an encoder for encoding images, the encoder having a processor configured to obtain colour data of colour channels of a given image and other data of at least one other channel of the given image, wherein the at least one other channel comprises at least one of: an alpha channel, a depth channel, a velocity channel of the given image; divide the given image into blocks, wherein the colour data and the other data are divided into their own respective blocks; and encode the blocks into encoded data, wherein a given block of the colour data and a corresponding block of the other data are encoded together into the encoded data.

Image encoding based on cross-analysis of multiple channels

Calibration data pertaining to chromatic aberration in optics of client device(s) is obtained by server(s). First pose information indicative of at least a pose of the client device(s) over a first time period is received at server(s) from the client device(s). A first predicted pose of the client device(s) corresponding to a future time instant is estimated, at the server(s), based on first pose information. An image frame is reprojected from an initial pose to the first predicted pose, at the server(s), to generate a first reprojected image frame. A chromatic aberration correction is applied on the first reprojected image frame, at the server(s), based on the calibration data, to generate a first output image frame. The first output image frame is sent from the server(s) to client device(s).

Multi-stage chromatic aberration correction

A system includes devices, each including an illuminator, sensor and processor, where the processor of a given device is configured to: project a pattern of light onto surroundings, while detecting reflections of the pattern of light onto the surroundings, determine shapes of surfaces present in the surroundings and distances of the surfaces from a pose of the given device; obtain pattern information indicative of pattern(s) of light projected by other active illuminator(s) of other device(s), detect reflections of pattern(s) from the pose of the given device; determine a relative pose of other device(s) with respect to the pose of the given device; and send, to a server, surface information indicative of the shapes and the distances of the surfaces from the pose of the given device, along with pose information indicative of the relative pose of the other device(s) with respect to the pose of the given device.

Employing patterned light to determine relative poses

Disclosed is an extended-reality (XR) device with pose-tracking means; light source(s) for displaying XR images to a user; and a processor configured to: send, to a server a network address of an XR device; send, to the server, request to provide a network address of a computing device; receive, from the server, the network address of the computing device; establish a direct communication link between the XR device and the computing device, using the network address of the computing device; send, to the computing device, information indicative of a pose of XR device or of a head of the user, via the direct communication link; receive, from computing device, XR image(s) generated according to the pose, via direct communication link; and display XR image(s) via light source(s).

Using user's computing device for rendering and streaming

A display apparatus including: light source(s); gaze-tracking means; and processor(s) configured to: determine gaze directions of user's eyes; send, to rendering server, information indicative of gaze direction determined at first time instant; receive image frame(s) generated according to gaze, and being optionally timestamped with second time instant; display image frame(s) at third time instant; determine time lag between any one of: first time instant and third time instant, or second time instant and third time instant; detect whether or not time lag exceeds first predefined threshold; when time lag exceeds first predefined threshold, switch on gaze-lock mode; select forward line of vision as fixed gaze direction; send, to rendering server, information indicative of fixed gaze; and receive image frames generated according to fixed gaze; and display image frames.

Display apparatus and method incorporating adaptive gaze locking

A system for producing image frames for display at display device. The system includes graphics processing units including first graphics processing unit and second graphics processing unit that are communicably coupled to each other and pose-tracking means. Second graphics processing unit is configured to: process pose-tracking data, to determine device pose and velocity and/or acceleration with which device pose is changing; execute rendering application(s) to generate framebuffer data corresponding to image frame; and send, to first graphics processing unit, framebuffer data and information indicative of device pose and velocity and/or acceleration. First graphics processing unit is configured to: execute first compositing application to post-process framebuffer data, based at least on said information; and drive light source(s) using post-processed framebuffer data to display image frame.

Systems and methods employing multiple graphics processing units for producing images

A system for facilitating scalable shared rendering, including plurality of servers communicably coupled to each other, each server executing executable instance of rendering software, being communicably coupled to display apparatus(/es), wherein when executed, rendering software causes each server to receive information indicative of poses of users of display apparatus(/es), utilise three-dimensional model(/s) of extended-reality environment to generate images from poses, send images to respective display apparatus(/es) for display, wherein at least one of plurality of servers is configured to detect when total number of display apparatuses to be served exceeds predefined threshold number, and employ new server and execute new executable instance of rendering software when predefined threshold number is exceeded, wherein new display apparatuses are served by new server, thereby facilitating scalable shared rendering.

Systems and methods for facilitating scalable shared rendering

A display apparatus including: light source(s) per eye, first tracking means, and processor(s) configured to: process first tracking data, collected by first tracking means, to determine location of display apparatus in real-world environment; obtain software application(s) that is available for location of display apparatus along with metainformation indicative of location in real-world environment with which software application(s) is/are associated; determine relative location of display apparatus with respect to location with which software application(s) is/are associated; execute software application(s) to create and overlay virtual content on image(s) representing real-world environment, based on relative location of display apparatus with respect to location with which software application(s) is/are associated; and display image(s) via light source(s).

Display apparatuses and methods for facilitating location-based virtual content

A display apparatus including first and second light sources, gaze-tracking means, and processor(s) configured to: process gaze-tracking data to determine gaze direction; identify gaze region; determine first and second portions of gaze region; send, to rendering server, resolution information indicative of at least one of: gaze direction, gaze region, first and second portions of gaze region, different required resolutions of at least two input images; receive said input images comprising first input image(s) and second input image(s), from rendering server; process first input image(s) to generate first region of first output image and second region of second output image; process second input image(s) to generate remaining regions of first and second output images; and display first and second output images via first and second light sources.

Display apparatuses and methods incorporating foveated rendering

An encoding method and a decoding method. The encoding method includes generating curved image by creating projection of visual scene onto inner surface of imaginary 3D geometric shape that is curved in at least one dimension; dividing curved image into input portion and plurality of input rings; encoding input portion and input rings into first planar image and second planar image, respectively, such that input portion is stored into first planar image, and input rings are packed into corresponding rows of second planar image; and communicating, to display apparatus, first and second planar images and information indicative of sizes of input portion and input rings.

Foveation-based image encoding and decoding

An imaging system including visible-light camera(s), pose-tracking means, and processor(s). The processor(s) is/are configured to: control visible-light camera(s) to capture visible-light image, whilst processing pose-tracking data to determine pose of camera(s); obtain three-dimensional model of real-world environment; create occlusion mask, using three-dimensional model; cull part of virtual object(s) to generate culled virtual object(s), wherein virtual object(s) is to be embedded at given position in visible-light image; detect whether width of culled part or remaining part of virtual object(s) is less than predefined percentage of total width of virtual object(s); if width of culled part is less than predefined percentage, determine new position and embed entirety of virtual object(s) at new position to generate extended-reality image; and if width of remaining part is less than predefined percentage, cull entirety of virtual object(s).

Imaging systems and methods incorporating improved culling of virtual objects

An imaging system including visible-light camera(s), depth sensor(s), pose-tracking means, and server(s) configured to: control visible-light camera(s) and depth sensor(s) to capture visible-light images and depth images of real-world environment, respectively, whilst processing pose-tracking data to determine poses of visible-light camera(s) and depth sensor(s); reconstruct three-dimensional lighting model of real-world environment representative of lighting in different regions of real-world environment; receive, from client application, request message comprising information indicative of location in real-world environment where virtual object(s) is to be placed; utilise three-dimensional lighting model to create sample lighting data for said location, wherein sample lighting data is representative of lighting at given location in real-world environment; and provide client application with sample lighting data.

Imaging systems and methods for facilitating local lighting

A display apparatus comprising: first light source(s) per eye, scanning mirror(s) per eye, pattern converting element per eye, and processor(s) configured to control first light source(s) to emit a light beam, whilst controlling scanning mirror(s) to draw subframe(s) of first image frame over pattern converting element, wherein subframe(s), when drawn, comprises plurality of light spots arranged in first pattern, wherein pattern converting element is employed to direct light beam incident thereon towards target surface, whilst converting first pattern of plurality of light spots into second pattern, thereby producing on target surface output image having spatially-variable resolution.

Display apparatuses and methods incorporating pattern conversion

A display apparatus including light source(s), camera(s), head-tracking means, and processor configured to: obtain three-dimensional model of real-world environment; control camera(s) to capture given image of real-world environment, whilst processing head-tracking data obtained from head-tracking means to determine pose of users head with respect to which given image is captured; determine region of three-dimensional model that corresponds to said pose of users head; compare plurality of features extracted from region of three-dimensional model with plurality of features extracted from given image, to detect object(s) present in real-world environment; employ environment map of extended-reality environment to generate intermediate extended-reality image based on pose of users head; embed object(s) in intermediate extended-reality image to generate extended-reality image; and display extended-reality image via light source(s).

Display apparatuses and methods incorporating image masking

A system for producing extended-reality images for a display apparatus. The system includes camera(s) and processor communicably coupled to camera(s), wherein processor is configured to: control camera(s) to capture image(s) representing test object present in real-world environment, wherein test object is physically covered three-dimensionally with coded pattern; obtain information pertaining to three-dimensional geometry of coded pattern; analyze image(s) to identify first image segment representing part of coded pattern visible in image(s); determine virtual content to be presented for test object, based on said part of coded pattern; process image(s) to generate extended-reality image(s) in which virtual content is virtually superimposed over said part of the coded pattern, based on information pertaining to three-dimensional geometry of coded pattern.

Growing community of inventors

Helsinki, Finland

Ari Antti Erik Peuhkurinen