Varjo Technologies Oy

Nokia Technologies Oy

Ville Timonen

Petteri Timonen

Ari Antti Erik Peuhkurinen

Joni-Matti Määttä

Tuomas Tölli

Mikko Ollila

Emre Baris Aksu

Mikko Strandborg

Kimmo Tapio Roimela

Roope Rainisto

Thomas Carlsson

Ari Antti Peuhkurinen

Mika Pesonen

Johannes Pystynen

Oleksandr Dovzhenko

Roman Golovanov

Johannes Rajala

Anna Nilsson

Niki Dobrev

Tarek Mohsen

Disclosed is a video transport method including receiving high dynamic range (HDR) video content comprising HDR images; performing HDR mastering for HDR images according to HDR mastering parameters; transforming an input colour space of each HDR image amongst HDR images into a transport colour space, and transporting HDR images having transport colour space. Herein, transforming an input colour space of HDR images into transport colour space is performed by: plotting colour component values (CCVs) of pixels in a first HDR image amongst a given set of HDR images, across a multi-dimensional colour space; identifying a bounding box corresponding to the first HDR image, wherein the bounding box encloses CCVs of two or more colour components represented in multi-dimensional colour space; determining a transformation to be applied to input colour space of the given set of HDR images; applying transformation to each HDR image in given set of HDR images.

Video transport method and system

Disclosed is head-mounted display (HMD) apparatus with an ambient light sensor (ALS), tracking camera (TC), video-see-through (VST) camera, and processor(s) configured to: capture average red-green-blue (RGB) illuminance (A) of region of real-world environment (RE) from HMD pose; capture grayscale image (GI) of RE from HMD pose, wherein pixel values indicate intensities (i); calculate average TC intensity (I) for part of FOV of TC that overlaps with FOV of ALS, based on intensities (i); calculate average illuminance (J) for said part of FOV of TC that overlaps with FOV of ALS, based on intensities (i) and pre-determined response (r) of ALS; calculate corrected average RGB illuminance (AC) for region from HMD pose, based on A and ratio of I and J; and determine RGB illuminance for each pixel of GI, based on AC, ratio of intensity (i) of said pixel in GI to I.

Generating lighting map using ambient light sensor and tracking camera

A method including: receiving visible-light images captured using camera(s) and depth data corresponding to said images; identifying image segments of visible-light image that represent objects or their parts belonging to different material categories; detecting whether at least two adjacent image segments in visible-light image pertain to at least two different material categories related to same object category; and when it is detected that at least two adjacent image segments pertain to at least two different material categories related to same object category, identifying at least two adjacent depth segments of depth data corresponding to at least two adjacent image segments; and correcting errors in optical depths represented in at least one of at least two adjacent depth segments, based on optical depths represented in remaining of at least two adjacent depth segments.

Depth maps and 3D reconstruction with segmentation masks

An imaging system including visible-light camera(s), depth sensor(s), pose-tracking means, and server(s) configured to: control visible-light camera(s) and depth sensor(s) to capture visible-light images and depth images of real-world environment, respectively, whilst processing pose-tracking data to determine poses of visible-light camera(s) and depth sensor(s); reconstruct three-dimensional lighting model of real-world environment representative of lighting in different regions of real-world environment; receive, from client application, request message comprising information indicative of location in real-world environment where virtual object(s) is to be placed; utilise three-dimensional lighting model to create sample lighting data for said location, wherein sample lighting data is representative of lighting at given location in real-world environment; and provide client application with sample lighting data.

Imaging systems and methods for facilitating local lighting

A display apparatus including light source(s), camera(s), head-tracking means, and processor configured to: obtain three-dimensional model of real-world environment; control camera(s) to capture given image of real-world environment, whilst processing head-tracking data obtained from head-tracking means to determine pose of users head with respect to which given image is captured; determine region of three-dimensional model that corresponds to said pose of users head; compare plurality of features extracted from region of three-dimensional model with plurality of features extracted from given image, to detect object(s) present in real-world environment; employ environment map of extended-reality environment to generate intermediate extended-reality image based on pose of users head; embed object(s) in intermediate extended-reality image to generate extended-reality image; and display extended-reality image via light source(s).

Display apparatuses and methods incorporating image masking

The invention relates to a method and technical equipment for implementing the method. The method comprises generating a three-dimensional segment of a scene of a content; generating more than one two-dimensional views of the three-dimensional segment, each two-dimensional view representing a virtual camera view; generating multi-view streams by encoding each of the two-dimensional views; encoding parameters of a virtual camera to the respective stream of the multi-view stream; receiving a selection of one or more streams of the multi-view stream; and streaming only the selected one or more streams.

Method and an apparatus and a computer program product for adaptive streaming

An imaging system for producing images for a display apparatus, the imaging system including: at least one camera; means for tracking an orientation of the at least one camera; and at least one processor communicably coupled to said camera and said means. The at least one processor is configured to: create and store an N-dimensional data structure representative of an environment; for a plurality of orientations of the at least one camera, determine values of camera attributes to be employed to capture a given image from a given orientation and update the N-dimensional data structure with the determined values; access the N-dimensional data structure to find values of camera attributes for a current orientation; and control the at least one camera to employ the found values for capturing an image of real-world scene from the current orientation.

Imaging system and method of producing images

A display system and method for correcting drifts in camera poses. Images are captured via camera, and camera poses are determined in global coordinate system. First features are extracted from first image. Relative pose of first feature with respect to camera is determined. Pose of first feature in global coordinate system is determined, based on its relative pose and first camera pose. Second features are extracted from second image. Relative pose of second feature with respect to camera is determined. Pose of second feature in global coordinate system is determined, based on its relative pose and second camera pose. Matching features are identified between first features and second features. Difference is determined between pose of feature based on first camera pose and pose of feature based on second camera pose. Matching features that satisfy first predefined criterion based on difference are selected. Correction transform that, when applied to second camera pose, yields corrected second camera pose is generated, such that corrected differences between poses of matching features based on corrected second camera pose and corresponding poses of matching features based on first camera pose satisfy second predefined criterion. Correction transform is applied to second camera pose. Second image is processed, based on corrected second camera pose, to generate extended-reality image.

Display systems and methods for correcting drifts in camera poses

A display system including display or projector , camera, means for tracking position and orientation of user's head, and processor. The processor is configured to control camera to capture images of real-world environment using default exposure setting, whilst processing head-tracking data to determine corresponding positions and orientations of user's head with respect to which images are captured; process images to create environment map of real-world environment; generate extended-reality image from images using environment map; render extended-reality image; adjust exposure of camera to capture underexposed image of real-world environment; process images to generate derived image; generate next extended-reality image from derived image using environment map; render next extended-reality image; and identify and modify intensities of oversaturated pixels in environment map, based on underexposed image and position and orientation with respect to which underexposed image is captured.

Display system and method of using environment map to generate extended-reality images

An imaging system for producing images for a display apparatus. The imaging system includes at least one camera, and processor communicably coupled to the at least one camera. The processor is configured to: obtain, from display apparatus, information indicative of current gaze direction of a user; determine, based on current gaze direction of the user, an object of interest within at least one display image, wherein the at least one display image is representative of a current view presented to user via display apparatus; adjust, based on a plurality of object attributes of the object of interest, a plurality of camera attributes of the at least one camera for capturing a given image of a given real-world scene; and generate from the given image a view to be presented to user via display apparatus.

Imaging system and method for producing images via gaze-based control

An imaging system for producing mixed-reality images for display apparatus. The imaging system includes a camera and a processor communicably coupled to the camera. The processor is configured to control the camera to capture image of real-world environment; analyze the image to identify surface that displays visual content; compare the visual content displayed in the image with reference image of the visual content to determine size, position and orientation of the surface with respect to the camera; process the reference image of the visual content to generate processed image of the visual content; and replace the visual content displayed in the image with the processed image to generate mixed-reality image, wherein resolution of the processed image is higher than resolution of the visual content displayed in the image.

Growing community of inventors

Espoo, Finland

Ville Timonen