Koninklijke Philips Corporation N.v.

Patrick Luc Els Vandewalle

Christiaan Varekamp

Bart Kroon

Mark Jozef Willem Mertens

Wilhelmus Hendrikus Alfonsus Bruls

Jeroen Hubert Christoffel Jacobus Stessen

Johannes Yzebrand Tichelaar

Rutger Nijland

Johannes Gerardus Rijk Van Mourik

Roeland Focco Everhard Goris

Philip Steven Newton

Mathias Hubertus Godefrida Peeters

Bart Koon

An apparatus comprises a store () storing a set of anchor poses for a scene, as well as typically 3D image data for the scene. A receiver () receives viewer poses for a viewer and a render pose processor () determines a render pose in the scene for a current viewer pose of the viewer pose where the render pose is determined relative to a reference anchor pose. A retriever () retrieves 3D image data for the reference anchor pose and a synthesizer () synthesizes images for the render pose in response to the 3D image data. A selector selects the reference anchor pose from the set of anchor poses and is arranged to switch the reference anchor pose from a first anchor pose of the set of anchor poses to a second anchor pose of the set of anchor poses in response to the viewer poses.

Apparatus and method for generating images of a scene

To enable a high quality HDR video communication, which can work by sending corresponding LDR images potentially via established LDR video communication technologies, which works well in practical situations, applicant has invented a HDR video decoder () arranged to calculate a HDR image (Im_RHDR) based on applying to a received 100 nit standard dynamic range image (Im_RLDR) a set of luminance transformation functions, the functions comprising at least a coarse luminance mapping (FC), which is applied by a dynamic range optimizer (), and a mapping of the darkest value (0) of an intermediate luma (Y'HPS), being output of the dynamic range optimizer, to a received black offset value (Bk_off) by a range stretcher (), the video decoder comprising a gain limiter () arranged to apply an alternate luminance transformation function to calculate a subset () of the darkest luminances of the HDR image, from corresponding darkest lumas (Y&#x2032;_in) of the standard dynamic range image.

Encoding and decoding HDR videos

An apparatus comprises a receiver () for receiving an image signal comprising a number of three dimensional images representing a scene from different viewpoints, each three dimensional image comprising a mesh and a texture map, the signal further comprising a plurality of residual data texture maps for a first viewpoint being different from the different viewpoints of the number of three dimensional images, a first residual data texture map of the plurality of residual texture maps providing residual data for a texture map for the first viewpoint relative to a texture map resulting from a viewpoint shift of a first reference texture map being a texture map of a first three dimensional image of the number of three dimensional images and a second residual data texture map of the plurality of residual texture maps providing residual data for a texture map for the first viewpoint relative to a texture map resulting from a viewpoint shift of a second reference texture map being a texture map of a second three dimensional image of the number of three dimensional images. A mesh processor () generates a first mesh for the first viewpoint and a second mesh for the first viewpoint from an additional reference mesh being a mesh of the second three dimensional image of the plurality of three dimensional images. A first texture processor () generates a first intermediate texture map in response to a viewpoint transformation of the first reference texture map from a viewpoint of the first three dimensional image to the first viewpoint and an second intermediate texture map in response to a viewpoint transformation of the second reference texture map from a viewpoint of the second three dimensional image to the first viewpoint. A second texture processor () generates a first texture map in response to a combination of the first intermediate texture map and the first residual data texture map, and a second texture map in response to a combination of the second intermediate texture map and the second additional residual data texture map. An image generator () generates an image for a second viewpoint from the first texture map, the first mesh, the second texture map, and the second mesh.

Processing of 3D image information based on texture maps and meshes

An apparatus comprises a store () storing a set of image parts and associated depth data for images representing a scene from different view poses (position and orientation). A predictability processor () generates predictability measures for image parts of the set of images for view poses of the scene. A predictability measure for a first image part for a first view pose is indicative of an estimate of the prediction quality for a prediction of at least part of an image for a viewport of the first view pose from a subset of image parts of the set of image parts not including the first image part. A selector () selects a subset of image parts of the set of image parts in response to the predictability measures, and a bitstream generator () for generating the image bitstream comprising image data and depth data from the subset of image parts.

Apparatus and method for generating an image data bitstream

An apparatus comprises a store () storing images corresponding to different positions and viewing directions for a scene, and associated position parameter vectors for the images where the vector for an image comprises data indicative of a viewing position and a viewing direction. A receiver () receives a viewing position parameter vector from a remote client (). A selector () selects a set of images in response to a comparison of the viewing position parameter vector and the associated position parameter vectors. An image synthesizer () generates an image from the set of images. A data generator () generates a reference position parameter vector for the synthesized image indicating a viewing position and direction for the synthesized image. An image encoder () encodes the synthesized image and an output generator () generates an output image signal comprising the encoded synthesized image and the reference position parameter vector. The output image signal is transmitted to the remote client.

Image processing using a plurality of images for a three dimension scene, having a different viewing positions and/or directions

An apparatus comprises receiver () receiving a light intensity image, confidence map, and image property map. A filter unit () is arranged to filter the image property map in response to the light intensity image and the confidence map. Specifically, for a first position, the filter unit () determines a combined neighborhood image property value in response to a weighted combination of neighborhood image property values in a neighborhood around the first position, the weight for a first neighborhood image property value at a second position being dependent on a confidence value for the first neighborhood image property value and a difference between light intensity values for the first position and for the second position; and determines a first filtered image property value for the first position as a combination of a first image property value at the first position in the image property map and the combined neighbor image property value.

Method and apparatus for processing an image property map

To enable a high quality HDR video communication, which can work by sending corresponding LDR images potentially via established LDR video communication technologies, which works well in practical situations, applicant has invented a HDR video decoder () arranged to calculate a HDR image (Im_RHDR) based on applying to a received 100 nit standard dynamic range image (Im_RLDR) a set of luminance 5 transformation functions, the functions comprising at least a coarse luminance mapping (FC), which is applied by a dynamic range optimizer (), and a mapping of the darkest value (0) of an intermediate luma (Y'HPS), being output of the dynamic range optimizer, to a received black offset value (Bk_off) by a range stretcher (), the video decoder comprising a gain limiter () arranged to apply an alternate luminance transformation function to 10 calculate a subset () of the darkest luminances of the HDR image, from corresponding darkest lumas (Y&#x2032;_in) of the standard dynamic range image.

An apparatus for determining a depth map for an image of a scene comprises an active depth sensor () and a passive depth sensor () for determining a depth map for the image. The apparatus further comprises a light determiner () which determines a light indication indicative of a light characteristic for the scene. The light indication may specifically reflect a level of visible and/or infrared light for the scene. A depth map processor () determines an output depth map for the image by combining the first depth map and the second depth map. Specifically, a depth value for the output depth map is determined as a combination of depth values of the first and the second depth map where the combining is dependent on the light indication. The light determiner () estimates the first infrared light indication from a visible light indication indicative of a light level in a frequency band of the visible light spectrum.

Method and apparatus for determining a depth map for an image

An apparatus is arranged to generate a triangle mesh for a three dimensional image. The apparatus includes a depth map source () which provides a depth map and a tree generator () generates a k-D tree from the depth map. The k-D tree representing a hierarchical arrangement of regions of the depth map satisfying a requirement that a depth variation measure for undivided regions is below a threshold. A triangle mesh generator () positions an internal vertex within each region of the k-D tree. The triangle mesh is then generated by forming sides of triangles of the triangle mesh as lines between internal vertices of neighboring regions. The approach may generate an improved triangle mesh that is suitable for many 3D video processing algorithms.

Generation of triangle mesh for a three dimensional image

An apparatus comprises a depth map source () providing a depth map for an image and a confidence map source () providing a confidence map with confidence values for pixels of the depth map. The confidence values designate the pixels as confident or non-confident pixels reflecting whether the depth value meets a reliability criterion or not. A depth modifier () performs a depth modification operation which modifies depth values for pixels of the depth map. A modified depth value for a first pixel is determined as a current depth value for the first pixel if this is designated as a confident pixel or if there are no confident pixels within a neighborhood set of pixels, and otherwise the modified value is determined as the maximum of the current depth value and a depth value determined as a function of depth values of pixels within the neighborhood set being designated as confident pixels.

Processing a depth map for an image

An autostereoscopic 3D display comprises a first unit () for generating an intermediate 3D image. The intermediate 3D image comprises a plurality of regions and the first unit () is arranged to generate a first number of image blocks of pixel values corresponding to different view directions for the region regions. The number of image blocks is different for some regions of the plurality of regions. A second unit () generates an output 3D image comprising a number of view images from the intermediate 3D image, where each of the view images correspond to a view direction. The display further comprises a display arrangement () and a driver () for driving the display arrangement () to display the output 3D image. An adaptor () is arranged to adapt the number of image blocks for a first region in response to a property of the intermediate 3D image or a representation of a three dimensional scene from which the first image generating unit () is arranged to generate the intermediate image.

Method and apparatus for generating a three dimensional image

An apparatus receives an image and an associated depth map comprising input depth values. A contour detector () detects contours in the image. A model processor () generates a contour depth model for a contour by fitting a depth model to input depth values for the contour. A depth value determiner () determines a depth model depth value for at least a one pixel of the contour from the contour depth model. The depth model may e.g. correspond to a single depth value which e.g. may be set to a maximum input depth value. A modifier () generates a modified depth map from the associated depth map by modifying depth values of the associated depth map. This includes generating a modified depth value for a pixel in the modified depth map in response to the depth model depth value. The depth model depth value may e.g. replace the input depth value for the pixel.

Method and apparatus for determining a depth map for an angle

An apparatus for reducing the visibility of disparity estimation errors at edges, and in particular at overlays. The apparatus comprises a receiver () for receiving a three dimensional image represented by at least image values (brightness/contrast values) and a disparity value. A subset selector () evaluates an image property criterion for the image value for a group of pixels and determines a subset of pixels of the group of pixels for which the image property criterion is met. The criterion may for example reflect whether the pixel belongs to an image object edge. A distribution evaluator () generates a frequency distribution for disparity values of the subset of pixels and an analyzer () determines a shape property for the frequency distribution (the presence of a peak). An adaptor () determining a disparity remapping in response to the shape property and a remapper () modifies disparity values of the three dimensional image by applying the disparity remapping. The approach may e.g. reduce image depth when overlay graphics is likely to be present.

Processing a disparity of a three dimensional image

A 3D video system transfers video data from a video source device () to a destination device (). The destination device has a destination depth processor () for providing destination depth data. The source device provides depth filtering data including filter location data, the depth filtering data representing a processing condition for processing the destination depth data in a filter area of the video indicated by the filter location data. The destination depth processor () is arranged for processing, in dependence on the depth filtering data, the destination depth data in an area of the video indicated by the filter location data. The depth filtering data enables the rendering process to improve the quality of the depth data.

Metadata for depth filtering

A signal processing device for processing three-dimensional image data, having two-dimensional (2D) image data and 2D depth data, includes an input for receiving the 2D image data and the 2D depth data. Further, a first outlier detector is used for establishing a statistical depth deviation of a depth value from other depth values in a first spatial neighborhood of the 2D depth data. A second outlier detector is used for establishing a statistical image deviation of an image value from other image values in a second spatial neighborhood of the 2D image data. The signal processing device also includes a depth value generator for, in dependence on the statistical depth deviation and the statistical image deviation, providing a replacement depth value when the statistical depth deviation exceeds the statistical image deviation.

Growing community of inventors

Oud-Turnhout, Belgium

Patrick Luc Els Vandewalle