Interdigital Vc Holdings, Inc.

Interdigital Ce Patent Holdings, Sas

Interdigital Ce Patent Holdings

Thomson Licensing

Interdigital Madison Patent Holdings, Sas

Thomson Licensing S.a.

Julien Fleureau

Renaud Dore

Franck Thudor

Bertrand Chupeau

Thierry Tapie

Philippe Guillotel

Fabien Danieau

Gerard Briand

Nicolas Mollet

Didier Doyen

Neus Sabater

Benoit Vandame

Francois Gerard

Anatole Lecuyer

Marc Christie

Franck Galpin

Sylvain Thiebaud

Christel Chamaret

Charles Salmon-Legagneur

Pierre Hellier

Xavier Burgos

Bertrand Leroy

Phi Bang Nguyen

Julia Navarro

Guillaume Boisson

Philippe Robert

Mary-Luc Champel

Remi Houdaille

Francois Le Clerc

Quang Khanh Ngoc Duong

Remy Gendrot

Arnaud Leroy

Kiran Varanasi

Olivier Dumas

Antole Lecuyer

Khanh-Duy Le

Yoan Lefevre

Jeremie Bernon

Gaetan Moisson-Franckhauser

Methods and devices for generated multiplane images from a representation of a 3D scene are disclosed. When an MPI is generated by the system, an image of the accumulation of the alpha values is computed. A convolutional networked is trained with a penalization function that penalizes MPIs with high values of accumulated alpha according to a parameter selected or trained to reduce the redundancy of information between layers of the MPI. When generating a MPI from a subset of the views of a new captured 3D scene, a first MPI is generated with the trained CNN with the associated parameter. An error is calculated between the other views and corresponding synthesized with the first MPI. This error is used to modify the parameter. Then, a second MPI is generated with the trained CNN with the new parameter.

Method and apparatus for generating compact multiplane images

Methods and devices for encoding and decoding 3D scenes of an immersive video are disclosed with the related data stream. Taking advantage of the fact that parts of the 3D scene does not change from one frame of the video to another, even when the cameras move and/or the lighting condition change, dynamic and static parts of the 3D scene are encoded in different atlases. In an embodiment, static patch atlases are repeated between two updates. In another embodiment, only updates are encoded and transmitted. The decoder maintains a memory associating entity of the 3D scene with a set of static patches that is updated with instructions linked to update static patch atlases. At each presentation time the static part of the 3D scene to decode is obtained from the memory.

Volumetric video with long-term patch entity memory

Methods and devices for generating a multiplane image from a representation of a 3D scene are disclosed. Given a representation of the 3D scene, for example as a multiview image, a first MPI is generated by a convolutional neural network trained for generating a MPI according to a provided sampling of the depths of the 3D scene. The presence of objects in different layers, that is the complementary of the accumulated transparency of layers, is analyzed to resample the sampling of layers' depth. The new sampling is used by the CNN to generate a new MPI from a representation of the same 3D scene.

Method and apparatus for generating adaptive multiplane images

Methods and apparatus for encoding and decoding a volumetric scene are disclosed. A set of attribute and geometry patches is obtained by projecting samples of the volumetric scene onto the patches according to projection parameters. If the geometry patch is comparable to a planar layer located at a constant depth according to the projection parameters, only the attribute patch is packed in an attribute atlas image and the depth value is encoded in metadata. Otherwise, both attribute and geometry patches are packed in an atlas. At the decoding, if metadata for an attribute patch indicates that its geometry may be determined from the projection parameters and a constant depth, the attributes are inverse projected on a planar layer. Otherwise, attributes are inverse projected according to the associated geometry patch.

Method and apparatus for encoding and decoding volumetric content in and from a data stream

A method and a device for generating a data stream representative of a volumetric video generated from a sequence of multi-plane images are disclosed. A quantization law depending on an angular resolution used to generate patches from the multi-plane images is set up. For each intra-periods of the frame sequence, patches are generated for a given angular resolution. If generated patches do not fit in atlas images of a target size, the angular resolution is decreased and patches are generated again according to this new angular resolution.

Method and apparatus for encoding volumetric video represented as a multiplane image

To represent a 3D scene, the MPI format uses a set of fronto-parallel planes. Different from MPI, the current MIV standard accepts a 3D scene represented as sequence input pairs of texture and depth pictures as input. To enable transmission of an MPI cube via the MIV-V3C standard, in one embodiment, an MPI cube is divided into empty regions and local MPI partitions that contain 3D objects. Each partition in the MPI cube can be projected to one or more patches. For a patch, the geometry is generated as well as the texture attribute and alpha attributes, and the alpha attributes may be represented as a peak and a width of an impulse. In another embodiment, an MPI RGBA layer of the MPI is cut into sub-images. Each sub-image may correspond to a patch, and the RGB and alpha information of the sub-image are assigned to the patch.

3D scene transmission with alpha layers

Methods and devices are provided to encode points of a sphere onto an image by projecting them as a function of an isotropic region guided mapping operator and/or to decode pixels of an image onto a sphere by de-projected them according to the same parametrized mapping operator. A region of interest is determined on the sphere and the angular resolution of pixels of the image is modulated as a function of their distance to the reference pixel corresponding to the center of the region of interest. Chosen angular resolutions and size of the region of interest are associated with the image and encoded in a stream.

Method and apparatus to encode and decode images of points of a sphere

A sequence of three-dimension scenes is encoded as a video by an encoder and transmitted to a decoder which retrieves the sequence of 3D scenes. Points of a 3D scene visible from a determined point of view are encoded as a color image in a first track of the stream in order to be decodable independently from other tracks of the stream. The color image is compatible with a three degrees of freedom rendering. Depth information and depth and color of residual points of the scene are encoded in separate tracks of the stream and are decoded only in case the decoder is configured to decode the scene for a volumetric rendering.

Methods, devices and stream for encoding and decoding three degrees of freedom and volumetric compatible video stream

Methods and devices are provided to encode and decode a data stream carrying data representative of a three-dimensional scene, the data stream comprising color pictures packed in a color image; depth pictures packed in a depth image; and a set of patch data items comprising de-projection data; data for retrieving a color picture in the color image and geometry data. Two types of geometry data are possible. The first type of data describes how to retrieve a depth picture in the depth image. The second type of data comprises an identifier of a parametric function and a list of parameter values for the identified parametric function.

Method and apparatus for encoding and decoding three-dimensional scenes in and from a data stream

Methods and devices for encoding and decoding a data stream representative of a 3D volumetric scene comprising haptic features associated with objects of the 3D scene are disclosed. At the encoding, haptic features are associated with objects of the scene, for instance as haptic maps. Haptic components are stored in points of the 3D scene as color may be. These components are projected onto patch pictures which are packed in atlas images. At the decoding, haptic components are un-projected onto reconstructed points as color may be according to the depth component of pixels of the decoded atlases.

Haptic atlas coding and decoding format

Methods, device and data stream format are disclosed in the present document for the encoding, the formatting and the decoding of depth information representative of a 3D scene. Compression and decompression of quantized values by a video codec leads to a value error. This error on values is particularly sensitive for depth encoding. The present invention proposes to encode and decode depth with a quantization function that minimize an angle error when a value error on quantized depth creates a location delta between the projected point and the de-projected point. The inverse of such a quantization function has to be encoded in metadata associated with the 3D scene, for example as a LUT, to be retrieved at the decoding, as such functions are not tractable.

Method and apparatus for depth encoding and decoding

At least one embodiment relates to a method and apparatus for encoding a volumetric video representing a scene, said encoding being based on patches representing the color and depth of a 2D projection of subparts of the scene, wherein a first patch is packed in a second patch for a given time interval lower than or equal to a time period along which the second patch is defined when said first patch can be packed in said second patch over said time interval. Decoding method and apparatus are also provided.

Processing volumetric data

Methods and devices for encoding and decoding data representative of a 3D scene are disclosed. A set of first patches is generated from a first MVD content acquired from a first region of the 3D scene. A patch is a part of one of the views of the MVD content. A set of second patches is generated from a second MVD content acquired from a second region of the 3D scene. An atlas packing first and second patches is generated and associated with metadata indicating, for a patch of the atlas, whether the patch is a first or a second patch At the decoding side, first patches are used for rendering the viewport image and second patches are used for pre-processing or post-processing the viewport image.

Volumetric video with auxiliary patches

Methods and devices are provided for encoding, transmitting and decoding 3DoF+ volumetric video. At the encoding stage one input view (among all the input ones) is selected to convey the viewport dependent light effect and its id is transmitted to the decoder as an extra metadata. On the decoder side, when patches coming from this selected view are available for the rendering of the viewport, they are preferentially used regarding the other candidates whatever the view to synthesize position.

Method and apparatus for coding and decoding volumetric video with view-driven specularity

Methods and devices for encoding and decoding three degrees of freedom and volumetric compatible video stream

Growing community of inventors

Rennes, France

Julien Fleureau