Koninklijke Philips Corporation N.v.

Philips Lighting Holding B.v.

Dolby Laboratories Licensing Corporation

Koninklijke Philips Electronics N.v.

Bart Kroon

Mark Thomas Johnson

Christiaan Varekamp

Eibert Gerjan Van Putten

Olexandr Valentynovych Vdovin

Philip Steven Newton

Patrick Luc Els Vandewalle

Achim Gerhard Rolf Koerber

Marcellinus Petrus Carolus Michael Krijn

Rick Bezemer

Denny Mathew

Siebe Tjerk De Zwart

Wiebe De Haan

Arnoldus Werner Johannes Oomen

Jeroen Gerardus Henricus Koppens

Age Jochem Van Dalfsen

Nathan Souviraa-Labastie

Bartolomeus Wilhelmus Damianus Van Geest

Nicolaas Lambert

Berent Willem Meerbeek

Bart Andre Salters

Wilhelmus Hendrikus Alfonsus Bruls

Frederik Jan De Bruijn

Adrianus Sempel

Jochen Renaat Van Gheluwe

Kars-Michiel Hubert Lenssen

Ramon Antoine Wiro Clout

Johannes Petrus Wilhelmus Baaijens

Johannes Henricus Maria Korst

Evert Jan Van Loenen

Alexander Franciscus Kolen

Jan Van Der Horst

Gopi Lakshminarayanan

Petrus Johannes Hendrikus Seuntiens

Fons Bruls

Elbert Gerjan Van Putten

Bartholomeus Wilhelmus Damianus Van Geest

Wilhelmus Hendricus Alfonsus Bruls

Nathan SouviraÀ-Labastie

Markthomas Johnson

A method for transmitting multi-view image frame data. The method comprises obtaining multi-view components representative of a scene generated from a plurality of sensors, wherein each multi-view component corresponds to a sensor and wherein at least one of the multi-view components includes a depth component and at least one of the multi-view components does not include a depth component. A virtual sensor pose is obtained for each sensor in a virtual scene, wherein the virtual scene is a virtual representation of the scene and wherein the virtual sensor pose is a virtual representation of the pose of the sensor in the scene when generating the corresponding multi-view component. Sensor parameter metadata is generated for the multi-view components, wherein the sensor parameter metadata contains extrinsic parameters for the multi-view components and the extrinsic parameters contain at least the virtual sensor pose of a sensor for each of the corresponding multi-view components. The extrinsic parameters enable the generation of additional depth components by warping the depth components based on their corresponding virtual sensor pose and a target position in the virtual scene. The multi-view components and the sensor parameter metadata is thus transmitted.

Coding hybrid multi-view sensor configurations

The invention provides a method for generating multilayer images. The method comprises receiving three-dimensional, 3D, source data of a scene and generating a plurality of layers at different depths in a virtual 3D scene, wherein each layer corresponds to a particular layer depth value relative to an origin point. Image data is generated for each layer based on the 3D source data, wherein the image data comprises texture data and transparency data, and displacement maps are generated for the layers based on the 3D source data and the depth values of the layers. The displacement maps define the depth of different points on a corresponding layer relative to the layer depth, wherein the generation of the displacement maps is restricted such that a depth surface, as defined by any displacement map, does not intersect any other depth surface in the virtual 3D scene. A depth surface is defined as the mapping of a displacement map onto the corresponding layer.

Low complexity multilayer images with depth

Generating an image signal comprises a receiver that receives source images representing a scene. A combined image generator generates combined images from the source images. Each combined image is derived from source images. An evaluator determines prediction quality measures for elements of the source images where the prediction quality measure for an element of a source image is indicative of a difference between pixel values in the source image and predicted pixel values for pixels in the element. The predicted pixel values are pixel values resulting from prediction of pixels from the combined images. A determiner determines low-quality segments of the source images comprising elements for which the prediction quality measure is indicative of a difference between the pixels that is above a threshold. An image signal generator generates an image signal comprising image data representing the combined images and the low-quality segments of the source images.

Image signal representing a scene

An apparatus comprises a receiver () receiving an image signal representing a scene. The image signal includes image data comprising a number of images where each image comprises pixels that represent an image property of the scene along a ray having a ray direction from a ray origin. The ray origins are different positions for at least some pixels. The image signal further comprises a plurality of parameters describing a variation of the ray origins and/or the ray directions for pixels as a function of pixel image positions. A renderer () renders images from the number of images based on the plurality of parameters.

Methods of encoding and decoding depth data are disclosed. In an encoding method, depth values and occupancy data are both encoded into a depth map. The method adapts how the depth values and occupancy data are converted to map values in the depth map. For example, it may adaptively select a threshold, above or below which all values represent unoccupied pixels. By adapting how the depth and occupancy are encoded, based on analysis of the depth values, the method can enable more effective encoding and transmission of the depth data and occupancy data. The encoding method outputs metadata defining the adaptive encoding. This metadata can be used by a corresponding decoding method, to decode the map values. Also provided are an encoder and a decoder for depth data, and a corresponding bitstream, comprising a depth map and its associated metadata.

Coding scheme for depth data

A distribution system comprises an audio server () for receiving incoming audio from remote clients () and for transmitting audio derived from the incoming audio to the remote clients (). An audio apparatus comprises an audio a receiver () which receives data comprising: audio data for a plurality of audio components representing audio from a remote client of the plurality of remote clients; and proximity data for at least one of the audio components. The proximity data is indicative of proximity between remote clients. A generator () of the apparatus generates an audio mix from the audio components in response to the proximity data. For example, an audio component indicated to be proximal to a remote client may be excluded from an audio mix for that remote client.

Audio apparatus, audio distribution system and method of operation therefor

A method for calibrating at least one of the six-degrees-of-freedom of all or part of cameras in a formation positioned for scene capturing, the method comprising a step of initial calibration before the scene capturing. The step comprises creating a reference video frame which comprises a reference image of a stationary reference object. During scene capturing the method further comprises a step of further calibration wherein the position of the reference image of the stationary reference object within a captured scene video frame is compared to the position of the reference image of the stationary reference object within the reference video frame, and a step adapting the at least one of the six-degrees-of-freedom of a multiple cameras of the formation if needed in order to get an improved scene capturing after the further calibration.

Method of calibrating cameras

A method for transitioning from a first set of video tracks, VT, to a second set of video tracks, VT, when rendering a multi-track video, wherein each video track has a corresponding rendering priority. The method comprises receiving an instruction to transition from a first set of first video tracks VTto a second set of second video tracks VT, obtaining the video tracks VTand, if the video tracks VTare different to the video tracks VT, applying a lowering function to the rendering priority of one or more of the video tracks in the first set of video tracks VTand/or an increase function to the rendering priority of one or more video tracks in the second set of video tracks VT. The lowering function and the increase function decrease and increase the rendering priority over time respectively. The rendering priority is used in the determination of the weighting of a video track and/or elements of a video track used to render a multi-track video.

Changing video tracks in immersive videos

A method of generating an image signal from a plurality of images of a scene performs iterations that comprise generating () predicted images for a candidate set of images from a set of (previously) included images. A set of selected images is selected () from the set of candidate images in response to a prediction quality. For each selected image a subset of pixels is determined () in response to pixel prediction qualities for the pixels of the image, and a set of partial images corresponding to the selected images is generated () by selecting () a subset of pixels. The selected image is deleted from the candidate set and the partial image is added to the included set. The approach may provide a low complexity and low resource selection of image data representing a scene.

Apparatus and method of generating an image signal

An image source () provides an image divided into segments of different sizes with only a subset of these comprising image data. A metadata generator () generates metadata structured in accordance with a tree data structure where each node is linked to a segment of the image. Each node is a branch node linking the parent node to child nodes linked to segments that are subdivisions of the parent node, or a leaf node which has no children. A leaf node is either an unused leaf node linked to a segment for which the first image comprises no image data or a used leaf node linked to a segment for which the first image comprises image data. The metadata indicates whether each node is a branch node, a used leaf node, or an unused leaf node. An image signal generator () generates an image signal comprising the image data of the first image and the metadata.

Methods of encoding and decoding immersive video are provided. In an encoding method, source video data comprising a plurality of source views is encoded into a video bitstream. At least one of the source views is down-sampled prior to encoding. A metadata bitstream associated with the video stream comprises metadata describing a configuration of the down-sampling, to assist a decoder to decode the video bitstream. It is believed that the use of down-sampled views may help to reduce coding artifacts, compared with a patch-based encoding approach. Also provided are an encoder and a decoder for immersive video, and an immersive video bitstream.

Coding scheme for immersive video with asymmetric down-sampling and machine learning

An apparatus comprises a receiver () for receiving an image representation of a scene. A determiner () determines viewer poses for a viewer with respect to a viewer coordinate system. An aligner () aligns a scene coordinate system with the viewer coordinate system by aligning a scene reference position with a viewer reference position in the viewer coordinate system. A renderer () renders view images for different viewer poses in response to the image representation and the alignment of the scene coordinate system with the viewer coordinate system. An offset processor () determines the viewer reference position in response to an alignment viewer pose where the viewer reference position is dependent on an orientation of the alignment viewer pose and has an offset with respect to a viewer eye position for the alignment viewer pose. The offset includes an offset component in a direction opposite to a view direction of the viewer eye position.

Image representation of a scene

Autostereoscopic display device comprising a backlight (), a display panel () comprising rows and columns of pixels and a lenticular arrangement (), wherein the backlight () provides a striped output comprising stripes in the column direction or offset by an acute angle to the column direction the lenticular arrangement comprises a first lenticular lens array () on the side of the display panel () facing the display output for directing different display panel pixel outputs in different directions and a second lenticular lens array () on the opposite side of the display panel (), facing the backlight (), for providing collimation of the striped back-light output.

Auto-stereoscopic display device with a striped backlight and two lenticular lens arrays

An autostereoscopic display device uses an electroluminescent display. A set of pixels is provided beneath view forming elements (such as lenses), with a plurality of pixels across the view forming element width direction. The pixels are arranged with at least two different angular orientations with respect to the substrate. The out-coupling performance is improved by arranging for the light emission direction to be substantially perpendicular to the desired emitting surface of the view forming elements.

Autostereoscopic display device

Growing community of inventors

Eindhoven, Netherlands

Bart Kroon