Varjo Technologies Oy

Antti Hirvonen

Mikko Strandborg

Ari Antti Erik Peuhkurinen

Jarno Wuolijoki

Tarmo Räntilä

A display apparatus has a display area of an output image divided into a plurality of regions. For a given region of the output image, a corresponding first region in a video-see-through (VST) image and a corresponding second region in at least one virtual-reality (VR) image are determined. From a depth map corresponding to the VST image, a minimum optical depth and a maximum optical depth in the first region of the VST image are determined. From a depth map corresponding to the at least one VR image, a minimum optical depth and a maximum optical depth in the second region of the at least one VR image are determined. When the minimum optical depth in the first region of the VST image is greater than the maximum optical depth in the second region of the at least one VR image, pixel data for pixels in the given region of the output image from the second region of the at least one VR image is fetched. Otherwise, when the minimum optical depth in the second region of the at least one VR image is greater than the maximum optical depth in the first region of the VST image, the pixel data for the pixels in the given region of the output image from the first region of the VST image is fetched.

Reducing texture lookups in extended-reality applications

Disclosed is a system comprising data repository(ies) and server(s) configured to: access, from data repository(ies), a first three-dimensional (3D) model of a virtual environment and a second 3D model of a real-world environment; combine the first 3D model and the second 3D model to generate a combined 3D model of an extended-reality (XR) environment; perform occlusion culling using the combined 3D model, to identify a set of virtual objects that occlude real object(s) in the XR environment from a perspective of a viewpoint; and send, to display apparatus(es), a virtual-reality (VR) image representing the set of virtual objects and information indicative of portion(s) of a field of view that corresponds to the real object(s) that is being occluded by the set of virtual objects from the perspective of the viewpoint.

Image signal processing based on occlusion culling

A system includes at least one server that is communicably coupled to at least one display apparatus. The at least one server is configured to obtain a real-world depth map corresponding to a target pose, and obtain a plurality of virtual-reality (VR) images and a plurality of VR depth maps for the target pose, wherein the plurality of VR images are generated by respective ones of a plurality of rendering applications that are executing at the at least one server. Further, the at least one server is configured to composite the plurality of VR images to generate a single VR image, based on the real-world depth map and the plurality of VR depth maps; and send the single VR image to the at least one display apparatus.

Server-side extended-reality pre-composition

A real-world depth map is obtained corresponding to a viewpoint from a perspective of which a virtual-reality (VR) depth map has been generated. For a given pixel in the VR depth map, an optical depth (D) of a corresponding pixel in the real-world depth map is found. A lower bound (D−D1) for the given pixel is determined by subtracting a first predefined value (D1) from the optical depth (D) of the corresponding pixel in the real-world depth map. An upper bound (D+D2) for the given pixel is determined by adding a second predefined value (D2) to the optical depth (D) of the corresponding pixel in the real-world depth map. An optical depth of the given pixel fetched from the VR depth image is re-mapped, from a scale of the lower bound to the upper bound determined for the given pixel to another scale of A to B, wherein A and B are scalars. Re-mapped optical depths of pixels of the VR depth map are then encoded into an encoded depth map. This encoded depth map is sent to at least one display apparatus.

Re-mapping virtual-reality depth based on real-world depth

Disclosed is a display apparatus with gaze-tracking means, light source(s), eyepiece lens, and processor(s) configured to: process gaze-tracking data to determine a gaze direction; determine relative position of a pupil with respect to eyepiece lens and relative orientation (A) of the pupil with respect to optical axis (OO') of eyepiece lens; estimate distortions per pixel(s) in image, based on the relative position, the relative orientation and distortion information; identify an image segment whose pixels' estimated distortions are higher than predefined threshold distortion; modify image by replacing pixels of identified image segment with black pixels or near-black pixels; and utilise modified image for display to given eye via light source(s).

Hiding lens distortions using black pixels

A display apparatus including first and second light sources, gaze-tracking means, and processor(s) configured to: process gaze-tracking data to determine gaze direction; identify gaze region; determine first and second portions of gaze region; send, to rendering server, resolution information indicative of at least one of: gaze direction, gaze region, first and second portions of gaze region, different required resolutions of at least two input images; receive said input images comprising first input image(s) and second input image(s), from rendering server; process first input image(s) to generate first region of first output image and second region of second output image; process second input image(s) to generate remaining regions of first and second output images; and display first and second output images via first and second light sources.

Growing community of inventors

Espoo, Finland

Antti Hirvonen