Varjo Technologies Oy

Intel Corporation

Nokia Corporation

Nokia Technologies Oy

Varjo Technologies Dy

Nokis Corporation

Varjo Technologies 0Y

Mikko Ollila

Mikko Strandborg

Klaus Melakari

Oiva Arvo Oskari Sahlsten

Marko Eromäki

Mikko Muukki

Ossi Kalevo

Kai Inha

Urho Konttori

Joonas Nissinen

Juha Alakarhu

Roope Rainisto

Jarno Nikkanen

Ari Antti Erik Peuhkurinen

Ville Miettinen

Ville Timonen

Petteri Timonen

Marko Bonden

Jani Penttila

David Isherwood

Tuomo Mörsky

Oiva Sahlsten

Olli-Pekka Haila

Timo Toivanen

Disclosed is an apparatus including an image sensor to capture an input image with one or more textual features. The one or more textual features are present in an unreadable or distorted form. Further, the apparatus includes a processor configured to detect the one or more textual features of the input image and execute a neural network to concurrently deduce a plurality of glyphs that form one or more relevant words or abbreviations based on the detected one or more textual features of the input image. Further, the processor is configured to generate an output image with enhanced one or more textual features in a legible form based on the deduced plurality of glyphs.

Apparatus and method of image processing to enhance textual features in output image

A method for depth image enhancement implemented in at least one apparatus includes reading and processing Phase Detection Autofocus (PDAF) pixels of a region-of-interest (ROI) area of a gaze region in an image obtained from a color camera sensor; utilizing one or more depth camera sensors to provide one or more depth maps of the ROI area of the gaze region; and combining the processed PDAF pixels of the ROI area of the gaze region and the one or more depth maps of the ROI area of the gaze region to obtain an updated ROI area with complementary depth information of the ROI area of the gaze region.

Method and apparatus for depth image enhancement

Disclosed is a system with an image sensor comprising a plurality of photo-sensitive cells; and processor(s) configured to: receive, from the image sensor, a plurality of image signals captured by corresponding photo-sensitive cells of the image sensor; obtain information indicative of a current pupil size of a user of a display apparatus; determine a given luminosity range corresponding to the current pupil size; and selectively perform a sequence of image signal processes on the plurality of image signals and control a plurality of parameters employed for performing the sequence of image signal processes, based on the given luminosity range, to generate an image.

Eye-directed image signal processing

First image data including first subsampled image data of at least a first part of a first field of view a first image sensor is obtained. Second image data including second subsampled image data of at least a second part of a second field of view of a second image sensor is obtained. The first part and the second part have at least a part of an overlapping field of view. The first image data and the second image data are processed to generate a first image and a second image, respectively. During processing, interpolation and demosaicking on the first subsampled image data is performed by utilising the second subsampled image data, while interpolation and demosaicking on the second subsampled image data is performed by utilising the first subsampled image data.

Complementing subsampling in stereo cameras

An imaging system includes a controllable light source; an image sensor; metalens for focusing incoming light onto the image sensor; and processor(s). The processor(s) is configured to control a light source using a first illumination intensity and/or first illumination wavelength, while controlling the image sensor to capture first image; control the light source using a second illumination intensity and/or second illumination wavelength, while controlling the image sensor to capture a second image; calculate measured differences between pixel values of pixels in the first image and pixel values of corresponding pixels in the second image; estimate expected pixel value differences based on difference between the first and second illumination intensities and/or differences between the first and second illumination wavelengths; and correct pixel values of pixels in the first image and/or second image based on deviation in measured differences from expected differences.

Correcting images captured with metalenses using different illumination characteristics

Each smallest repeating unit (SRU) in a colour filter array has a first sub-unit (FSU) and a second sub-unit (SSU). For a first sub-image, first image data is read out from photo-sensitive cells that correspond to FSUs of SRUs lying in a row (R-R) or column, and by skipping reading out from photo-sensitive cells that correspond to SSUs of a first predefined percent (FPP) of the SRUs, while reading out the first image data from photo-sensitive cells that correspond to SSUs of a remainder of the SRUs. For a second sub-image, second image data is read out from photo-sensitive cells that correspond to FSUs of the FPP, whilst skipping reading out from photo-sensitive cells that correspond to FSUs of the remainder, and by skipping reading out from photo-sensitive cells that correspond to the SSUs of the SRUs. Sub-pixel shift is performed between the sub-images. Image(s) are then generated from the sub-images.

Subsampling and wobulation in colour filter arrays having smallest repeating units with different sub-units

Disclosed is an imaging system with an image sensor; a wobulator that is to be employed to perform sub-pixel shifts when capturing images with the image sensor; and processor(s). The processor(s) is configured to, in each cycle, obtain two or three sub-images (-) from the image sensor, wherein each sub-image in a same cycle has subsampled image data of at least a part of a field of view of the image sensor, the subsampled image data being subsampled according to a same subsampling pattern in the two or three sub-images of a given cycle; control the wobulator to perform one or two sub-pixel shifts; and process the two or three sub-images, to generate image(s).

Imaging with subsampling and wobulation

Disclosed is an apparatus and a method for depth sensing that includes a scanning light source configured to sequentially illuminate an environment utilizing a scan pattern. The apparatus also includes a birefringent layer adapted to receive and split reflected scan pattern from the environment into an ordinary ray (O-ray) image and an extraordinary ray (E-ray) image. The apparatus further includes an imaging sensor positioned along an optical path of the birefringent layer to capture the O-ray image and the E-ray image as transmitted therefrom. The apparatus further includes a processor configured to compute differences between positions of the O-ray image and the E-ray image and derive a depth map of the environment based on the computed differences.

Depth sensing for extended reality (XR) apparatus

Disclosed is imaging system with an image sensor having photo-sensitive cells arranged on photo-sensitive surface of image sensor; colour filter array (CFA) arranged on optical path of photo-sensitive surface, CFA with: first, second, third and fourth types of colour filters that allow wavelengths lying in first, second, third wavelength ranges, respectively, to pass through; and light valve device having grid of light valves, wherein light valves of grid are arranged on optical paths of at least fourth type of colour filters; and processor(s) configured to: for given colour filter of fourth type, control corresponding light valve during given time period to allow wavelengths lying in first, second, or third wavelength range, to pass through towards given colour filter; read out image data from photo-sensitive cells; and process image data to generate given image.

Adjustable colour filter array using liquid crystal device

Image data is read out from a left part (LL) of a left field of view (FOV) of a left image sensor (L) and a right part (RR) of a right FOV of a right image sensor (R). The left part of the left FOV extends horizontally towards a right side of a gaze point (X) till only a first predefined angle (N1) from the gaze point. The right part of the right FOV extends horizontally towards a left side of a gaze point (X) till only a second predefined angle (N2) from the gaze point. The image data is processed to construct a left part of a left image and a right part of a right image. A right part of the left image and a left part of the right image are reconstructed using image reprojection. The left image and the right image are generated by combining respective parts.

Gaze-assisted image capture and stereo reprojection

An imaging system including: a first camera and a second camera corresponding to a first eye and a second eye of a user, respectively; and processor(s) configured to: control the first camera and the second camera to capture a sequence of first images and a sequence of second images, respectively; and apply motion blur correction to one of a given first image and a given second image, whilst applying at least one of: defocus blur correction, image sharpening, contrast enhancement, edge enhancement to another of the given first image and the given second image.

Differently correcting images for different eyes

A system includes at least one server configured to receive, from a client device, information indicative of gaze directions of a user's eyes; determine a gaze point and a gaze depth of the user's eyes, using said information; identify, in an image, a gaze-contingent region that includes and surrounds the gaze point; identify a region of interest in the gaze-contingent region, wherein the region of interest comprises pixels whose optical depth lies within a first predefined distance (Δd) from the gaze depth; encode the image by applying a first encoding setting to the pixels belonging to the region of interest; and send the image that is encoded, to the client device.

Gaze depth-assisted encoding of region of interest in images

Disclosed is system with imaging unit(s) and processor(s), imaging unit(s) with image sensor of photo-sensitive cells; and light valve device having array of light valves. The processor(s) is configured to: control light valves to allow light to pass therethrough, while controlling remaining light valves in the array to stop allowing the light to pass therethrough, thereby casting shadows of remaining light valves on first part and second part in first set and second set of photo-sensitive cells, respectively; read out image signals from photo-sensitive cells; select pairs of photo-sensitive cells from first set and second set; for given pair, determine phase difference between image signals read out from photo-sensitive cells of first set and second set; and adjust focus distance of the imaging unit(s), based on phase differences.

Phase detection autofocus using liquid crystal device

An imaging system including a first camera and a second camera corresponding to a first eye and a second eye of a user, respectively; and at least one processor. The at least one processor is configured to control the first camera and the second camera to capture a sequence of first images and a sequence of second images of a real-world environment, respectively; and apply a first extended depth-of-field correction to one of a given first image and a given second image, whilst applying at least one of: defocus blur correction, image sharpening, contrast enhancement, edge enhancement to another of the given first image and the given second image.

Selective extended depth-of-field correction

A system includes a tracking device and a light source. The tracking device has a camera and a first controller. The camera captures a first image and a second image. The first image is captured during a first period of time (t0-t1) and the second image is captured during a second period of time (t2-t3). The first controller is coupled to the camera and configured to obtain first timing information and second timing information; form timing instructions; and communicate timing instructions to light source over a communication interface. The light source is configured to use timing instructions to illuminate first amount of light and a second amount of light. The first controller is further configured to calculate image intensity difference between the first and second image to identify from first image pixel(s) illuminated by the light source.

Growing community of inventors

Tampere, Finland

Mikko Ollila