Interdigital Vc Holdings, Inc.

Interdigital Ce Patent Holdings, Sas

Gerard Briand

Julien Fleureau

Renaud Dore

Franck Thudor

Bertrand Chupeau

Thierry Tapie

Remy Gendrot

Franck Galpin

Neus Sabater

Methods, device and data stream are provided to encode unaligned multi-view plus depth images associated with assistance metadata for generating depth maps aligned with the unaligned color views. At the encoding stage, for an unaligned color view, the contribution of each depth map is evaluated. A subset of the most contributing depth maps is selected and metadata representative of this subset is encoded in the data stream of the unaligned multi-view plus depth images in association with the unaligned color view. At the decoding stage, to generate a depth map unaligned with the unaligned color view, only the depth maps referenced in the assistance metadata are warped.

Depth-color alignment with assistance metadata for transcoding of volumetric video

Methods, apparatuses and streams are disclosed for transmitting tiled volumetric video and, at the receiver, for generating an atlas image compatible with a legacy decoder. At the server side, viewport information is obtained and a first list of central tiles and a second list of border tiles are selected. A central tile is a part of an image obtained by projecting the 3D scene onto an image plane according to a central point of view. A border tile is an image comprising dis-occluding patches. Sizes and shapes of border tiles are function of size and shape of central tiles. At the client side, tiles are arranged according to a layout selected in a set of layouts according to the number, sizes and shapes of border tiles.

Method and apparatuses for delivering a volumetric video content

Methods and devices for encoding and decoding 3D scenes of an immersive video are disclosed with the related data stream. Taking advantage of the fact that parts of the 3D scene does not change from one frame of the video to another, even when the cameras move and/or the lighting condition change, dynamic and static parts of the 3D scene are encoded in different atlases. In an embodiment, static patch atlases are repeated between two updates. In another embodiment, only updates are encoded and transmitted. The decoder maintains a memory associating entity of the 3D scene with a set of static patches that is updated with instructions linked to update static patch atlases. At each presentation time the static part of the 3D scene to decode is obtained from the memory.

Volumetric video with long-term patch entity memory

A sequence of three-dimension scenes is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of 3D scenes. Points of a 3D scene visible from a determined point of view are encoded as a color image in a first track of the stream in order to be decodable independently from other tracks of the stream. The color image is compatible with a three degrees of freedom rendering. Depth information and depth and color of residual points of the scene are encoded in separate tracks of the stream and are decoded only in case the decoder is configured to decode the scene for a volumetric rendering.

Methods, devices and stream for encoding and decoding three degrees of freedom and volumetric compatible video stream

At least one embodiment relates to a method and apparatus for encoding a volumetric video representing a scene, said encoding being based on patches representing the color and depth of a 2D projection of subparts of the scene, wherein a first patch is packed in a second patch for a given time interval lower than or equal to a time period along which the second patch is defined when said first patch can be packed in said second patch over said time interval. Decoding method and apparatus are also provided.

Processing volumetric data

Methods and devices for encoding and decoding three degrees of freedom and volumetric compatible video stream

A patch-based atlas format in intra-periods of varying length is used to encode a volumetric video. A first atlas layout is built for a first sequence of 3D scenes. The number of 3D scenes in the sequence is chosen to fit the size of a Group of Pictures of the codec. A second sequence is iteratively set up by appending the next 3D scene of the sequence to encode while the number of patches of the layout built for this iterative second sequence is lower than or equal to the number of patches of the first layout. When iterations end, one of the layouts is selected to generate every atlas of the group. In such a way, size of metadata is decreased, and compression is enhanced.

Method and apparatus for encoding and decoding volumetric video

Generating an image from a source image can involve encoding a projection of a part of a three-dimensional scene. Pixels of a source image comprise a depth and a color attribute. Pixels of a source image are de-projected as colored point cloud. A de-projected point in a 3D space has the color attribute of the pixel that it has been de-projected from. Also, a score is attributed to the generated point according to a local depth gradient and/or a local color gradient of the pixel it comes from, the lower the gradient, the higher the score. The generated point cloud is captured by a virtual camera for rendering on a display device. The point cloud is projected onto the viewport image by blending color of points projected on a same pixel, the blending being weighted by the scores of these points.

Method and apparatus for decoding three-dimensional scenes

Method and device for generating a stream of data representative of a 3D point cloud. The 3D point cloud is partitioned into a plurality of 3D elementary parts. A set of two-dimensional 2D parametrizations is determined, each 2D parametrization representing one 3D part of the point cloud with a set of parameters. Each 3D part is represented as a 2D pixel image. A depth map and a color map are determined as a first patch atlas and a second patch atlas. A data stream is generated by combining and/or coding the parameters of the 2D parametrization, the first patch atlas, the second patch atlas and mapping information that links each 2D parametrization with its associated depth map and color map in respectively the first and second patch atlas.

Growing community of inventors

Ploufragan, France

Gerard Briand