Varjo Technologies Oy

Pixieray Oy

Ville Miettinen

Mikko Strandborg

Klaus Melakari

Urho Konttori

Ari Antti Erik Peuhkurinen

Mikko Ollila

Oiva Arvo Oskari Sahlsten

Niko Eiden

Ari Antti Peuhkurinen

Thomas Carlsson

Rebecca Qing Melakari

Tomi Lehto

Thiyagarajan Manihatty Bojan

Qing Xu

Oleksii Shtankevych

A left part (LL) of a left image and a right part (RR) of a right image are rendered, wherein a horizontal FOV of the left part of the left image extends towards a right side of a gaze point (X) of the left image till only a first predefined angle (N) from the gaze point of the left image, while a horizontal FOV of the right part of the right image extends towards a left side of a gaze point (X) of the right image till only a second predefined angle (N) from the gaze point of the right image. A right part (RL) of the left image is reconstructed by reprojecting a corresponding part of the right image, while a left part (LR) of the right image is reconstructed by reprojecting a corresponding part of the left image. The left image and the right image are generated by combining respective rendered parts and respective reconstructed parts.

Gaze-assisted stereo reprojection

An eye-tracking apparatus. At least one light source and a plurality of sensors are arranged at a periphery of a first surface of at least one lens. A frame is employed to hold the at least one lens. A processor coupled to the at least one light source and the plurality of sensors is configured to control the at least one light source to emit light towards the user's eye, control the plurality of sensors to sense reflections of the light off a surface of the user's eye, and process sensor data pertaining to the sensed reflections to determine a gaze direction of the user's eye.

Eyeglass lens with eye-tracking components

An encoder for encoding images includes at least one processor configured transform a given input pixel of an input image having (x, y) coordinates in a Cartesian coordinate system into a given transformed pixel of a transformed image having (ρ, θ) coordinates in a log-polar coordinate system, using a log-polar transformation in which a radial distance (ρ) of the given transformed pixel is a logarithm of a distance of the given input pixel from an origin in the Cartesian coordinate system, and an angular distance (θ) of the given transformed pixel is a sum of an arctangent of a slope of a line connecting the given input pixel to the origin and a function of the radial distance, encode the transformed image, by employing a compression algorithm, into an encoded image, and send the encoded image to a display apparatus for subsequent decoding thereat.

Encoders and decoders employing log-polar coordinate system

A method of transmitting image data in an image display system, includes dividing the image data into framebuffers, and for each framebuffer: dividing the framebuffer into a number of vertical stripes, each stripe including one or more scanlines, dividing each vertical stripe into at least a first and a second block, each of the first and the second block comprising pixel data to be displayed in an area of the image, and storing first pixel data in the first block with a first resolution and second pixel data in the second block having a second resolution which is lower than the first resolution, transmitting the framebuffer over the digital display interface to a decoder unit, and unpacking the framebuffer, including upscaling the pixel data in the second block to compensate for the lower second resolution and optionally upscaling the pixel data in the first block.

Method of transporting a framebuffer

An encoding method and a decoding method. The encoding method includes generating curved image by creating projection of visual scene onto inner surface of imaginary 3D geometric shape that is curved in at least one dimension; dividing curved image into input portion and plurality of input rings; encoding input portion and input rings into first planar image and second planar image, respectively, such that input portion is stored into first planar image, and input rings are packed into corresponding rows of second planar image; and communicating, to display apparatus, first and second planar images and information indicative of sizes of input portion and input rings.

Foveation-based image encoding and decoding

A display apparatus including: light source(s); camera(s); and processor(s) configured to: display extended-reality image for presentation to user, whilst capturing eye image(s) of user's eyes; analyse eye image(s) to detect eye features; employ existing calibration model to determine gaze directions of user's eyes; determine gaze location of user; identify three-dimensional bounding box at gaze location within extended-reality environment, based on position and optical depth of gaze location; identify inlying pixels of extended-reality image lying within three-dimensional bounding box, based on optical depths of pixels in extended-reality image; compute probability of user focussing on given inlying pixel and generate probability distribution of probabilities computed for inlying pixels; identify at least one inlying pixel calibration target, based on probability distribution; and map position of calibration target to eye features, to update existing calibration model to generate new calibration model.

Display apparatuses and methods for calibration of gaze-tracking

A display apparatus including means for tracking pose of user's head, light source(s) and processor configured to: process pose-tracking data to determine position, orientation, velocity and acceleration of head; predict viewpoint and view direction of user in extended-reality environment; determine region of extended-reality environment to be presented, based on viewpoint and view direction; determine sub-region(s) of region whose rendering information is to be derived from previous rendering information of corresponding sub-region(s) of previously-presented region of extended-reality environment; generate rendering information of sub-region(s) based on previous rendering information; send, to rendering server, information indicating remaining sub-regions required to be re-rendered and pose information indicating viewpoint and view direction; receive, from rendering server, rendering information of remaining sub-regions; merge rendering information of sub-region(s) and rendering information of remaining sub-regions to generate image(s); and display image(s) via light source(s).

Display apparatuses and rendering servers incorporating prioritized re-rendering

A gaze-tracking system for use in head-mounted display apparatus. The gaze-tracking system includes: illuminators; camera; and processor configured to: illuminate illuminators in sequential manner; control camera to capture eye images of user's eye during illumination of illuminators; identify reflection(s) of illuminator in eye image; determine extent of deformation in shape of reflection(s) with respect to shape of illuminator; determine extent of displacement in position of reflection(s) with respect to position of illuminator; compute user-specific score for illuminator based on extents of deformation and displacement; select illuminator(s) based on user-specific scores; illuminate illuminator(s); control camera to capture eye image of user's eye during illumination of illuminator(s); and detect gaze direction of user based upon relative position of pupil of user's eye with respect to reflections of illuminator(s) in eye image.

Gaze-tracking system and method employing selective glints

A display apparatus including pose-tracking means; image renderer per eye; liquid-crystal device including liquid-crystal structure and control circuit; and processor. Processor is configured to: process pose-tracking data to determine user's head pose; detect if rate at which head pose is changing is below predefined threshold rate; if yes, switch on lock mode, select head pose for session of lock mode, and generate output image frames according to head pose during session; if no, generate output image frames according to corresponding head poses of user using pose-tracking data; and display output image frames, whilst shifting light emanating to from pixels of image renderer to multiple positions (P-P) in sequential and repeated manner, said shifting causes resolution of output image frames to appear higher than display resolution of image renderer.

Display apparatus and method incorporating adaptive pose locking

In a display apparatus, a liquid-crystal structure, arranged in front of an image renderer, is controlled to shift light of a given sub-pixel to target positions according to a shifting sequence in a repeated manner, while output image frames are displayed. To generate a given output image frame, a given target position to which the light is to be shifted is determined based on the shifting sequence. An input colour value of the given sub-pixel provided in a given input image frame is then adjusted to generate an output colour value of the given sub-pixel for the given output image frame, based on an output colour value of at least one other sub-pixel whose light overlaps with the given target position during display of a previous output image frame, and a retention coefficient between a colour of the at least one other sub-pixel and a colour of the given sub-pixel.

Display apparatus and method incorporating sub-pixel aware shifting and overlap compensation

A display apparatus including gaze-tracking means, image renderers, liquid-crystal devices including liquid-crystal structure and control circuit, to shift light emanating from given pixel of image renderer to multiple positions, given pixel including colour component; and processor configured to: process gaze-tracking data to determine gaze direction of user's eye; determine gaze point; display first output image frame; detect if magnitude of difference between first output value and initial second output value of colour component of given pixel in first and second output image frames exceeds first threshold difference; when detected that magnitude of difference exceeds first threshold difference, update initial second output value to sum of first output value and product of distance factor and difference between initial second output and first output values; and display second output image frame.

Display apparatus and method incorporating gaze-based modulation of pixel values

A display apparatus communicably coupled with server arrangement. Display apparatus includes means for tracking user's gaze; means for tracking pose of display apparatus; light source; optical element; and processor. Processor is configured to: process gaze-tracking data; process pose-tracking data; send, to server arrangement, gaze information and apparatus information, server arrangement predicts gaze information and apparatus information, and processes input image to generate image based on predictions; receive, image, predicted gaze information, and predicted apparatus information; determine adjustment required in configuration of light source and optical element prior to displaying image; determine if portion of previous image is to be displayed during adjustment; if yes, display said portion during adjustment; if no, switch off or dim light source during adjustment; display image via light source after adjustment.

Synchronizing light sources and optics in display apparatuses

A method of encoding images and a method of decoding images. The method of encoding comprises dividing an input image into a first input portion and a plurality of input rings that are concentric with the first input portion, the first input portion and the plurality of input rings being shaped as a simple closed curve; storing the first input portion into a first image; packing the plurality of input rings into a second image, second image having a plurality of rows, a given input ring of input image being packed into a corresponding row of second image; and

Foveation-based encoding and decoding of images

An encoder including data processing arrangement configured to analyze depth information pertaining to input image to identify depth edge(s) and to determine location of depth edge(s); analyze input image to identify edges of objects and to determine locations of said edges; select, amongst edges in input image, matching edge(s) whose location matches with location of depth edge(s); obtain intermediate decoded image by decoding intermediate encoded image obtained from intermediate encoding of input image; analyze intermediate decoded image to identify edges of objects and to determine locations of said edges; determine edge(s) in intermediate decoded image that corresponds to matching edge(s); modify pixel values in input image and/or optical depths in depth information, based on difference in location of determined edge(s) in intermediate decoded image and location of matching edge(s); and encode input image and depth information, upon said modification, to generate encoded data.

Encoder and encoding method for mitigating discrepancies in reconstructed images

Disclosed is gaze-tracking system for head-mounted display apparatus, the gaze-tracking system including a plurality of non-circular light sources that emit light for illuminating a user's eye when the display apparatus is worn by user, at least one camera, and a processor. The processor is configured to control the at least one camera to capture an image of user's eye and reflections of the plurality of non-circular light sources from user's eye; identify at least one of the plurality of non-circular light sources from where at least one of the reflections originated, based upon shapes, rotational orientations and relative positions of the reflections of the plurality of non-circular light sources, to differentiate said reflections from visual artifacts; and detect a gaze direction of user based upon a relative position of a pupil of user's eye with respect to the reflections of the plurality of non-circular light sources.

Growing community of inventors

Helsinki, Finland

Ville Miettinen