Nokia Technologies Oy

Sebastian Schwarz

Lukasz Kondrad

Miska Matias Hannuksela

Payman Aflaki Beni

Lauri Aleksi Ilola

Emre Baris Aksu

Kimmo Tapio Roimela

Lauri Ilola

Vinod Kumar Malamal Vadakital

Mika Pesonen

Alireza Aminlou

Christoph Bachhuber

Patrice Rondao Alface

Vida Fakour Sevom

Ioan Tabus

Aleksei Martemianov

Emre Can Kaya

Payman Aflaki

A method and technical equipment for encoding, where the method comprises at least receiving a video presentation frame, where the video presentation represents a three-dimensional data in the form of mesh data (); separating from the mesh data information on vertices, wherein the information comprises at least connectivity data defining connections between vertices (); determining parameters relating to said connectivity data (); encoding the parameters to a first bitstream as a video component (); and storing the encoded first bitstream for transmission to a rendering apparatus (). In addition to encoding, also decoding is disclosed.

Method, an apparatus and a computer program product for volumetric video encoding and decoding

There are disclosed various methods, apparatuses and computer program products for video encoding and decoding. In some embodiments the method for video encoding comprises obtaining compressed volumetric video data representing a three-dimensional scene or object (); capsulating the compressed volumetric video data into a data structure (); obtaining data of a two-dimensional projection of at least a part of the three-dimensional scene as seen from a certain viewport (); and including the data of the two-dimensional projection into the data structure ().

Apparatus, a method and a computer program for omnidirectional video

Encoding a two depth images of points of a bounded volume (); determining a first section across a second axis between the depth images to generate a feasible occupancy image and a list of pixels with unknown occupancy status (); encoding a binary sequence according to a true occupancy image, wherein the binary sequence transmits the occupancy status for the list of pixels with unknown occupancy status (); detecting anchoring points from the binary sequence, and encoding missing elements between the anchoring points (); determining a first section across a third axis from the bounding volume reconstructed so far and create a list of pixels with unknown status (); encoding a binary sequence according to the true occupancy image the sequence conveying the occupancy status of the list of pixels with unknown status (); and detecting the anchor points from the sequence and corresponding list of pixels and encoding missing elements between the anchoring points ().

Method, an apparatus and a computer program product for video encoding and video decoding

An apparatus comprising circuitry configured to: obtain a three-dimensional model comprising at least one patch, at least one geometry component, at least one occupancy component, and zero or more texture components; subsample the at least one geometry component of the at least one patch of a three-dimensional object at occupied positions using a subsampling criterion; define respective search windows around the respective occupied positions; select respective salient points relative to the respective occupied positions within the respective search windows; triangulate the salient points to approximate a shape of the three-dimensional object; detect zero or more triangles that overlap with at least one unoccupied pixel; split the zero or more triangles that overlap with at least one unoccupied pixel until no triangle overlaps with the unoccupied pixels; and add zero or more additional triangles close to a border of the three-dimensional object to generate a resulting mesh signaled to a decoder.

V3C patch remeshing for dynamic mesh coding

The embodiments relate to a method comprising receiving as an input a volumetric video frame comprising volumetric video data (); decomposing the volumetric video frame into one or more patches, wherein a patch comprises a volumetric video data component (); packing several patches, where at least two patches of the several patches comprise a different volumetric video data component with respect to each other, into one video frame (); generating a bitstream comprising an encoded video frame (); signaling, in or along the bitstream, existence of encoded video frame containing patches of more than one different volumetric video data component (); and transmitting the encoded bitstream to a storage for rendering (). The embodiments also relate to a technical equipment for implementing the method.

Method, an apparatus and a computer program product for volumetric video encoding and video decoding

A method comprising: providing a 3D representation of at least one object as an input for an encoder (); projecting the 3D representation onto at least one 2D patch (); generating at least a geometry image and a texture image from the 2D patch (); generating, based on the geometry image, a mesh comprising a number of vertices (); mapping the number of vertices to two-dimensional (2D) coordinates of the texture image (), and signalling said 2D coordinates of the texture image to be applied to the number of vertices of the mesh in or along a bitstream ().

Apparatus, a method and a computer program for volumetric video

A method for decoding comprises receiving a compressed bitstream relating to a video presentation; decoding from the received bitstream data relating to three-dimensional blocks of a video frame; for each block of the video frame, determining information on surface parameters; generating a bounding box for a three-dimensional block according to the surface parameters; calculating a ray direction for each pixel of the three-dimensional block from a viewpoint to coordinates of the pixel; determining at least two points according to an intersection of the ray to the generated bounding box; ray casting points between the determined at least two points until a condition to finish the ray casting is met; reconstructing three-dimensional data from the geometry image and the texture image according to the information on one or more surface parameters for a block; reconstructing a video presentation according to the reconstructed three-dimensional data for rendering.

A method including projecting a 3D representation of at least one object onto at least one 2D patch; generating a geometry image, a texture image, an occupancy map and auxiliary patch information from 5 the 2D patch, wherein the auxiliary patch information includes metadata indicating whether a filtering process shall be applied on said 2D patch upon reconstruction of the 3D representation of said at least one object; and encoding the auxiliary patch information and zero or more of the following: the geometry image, the texture image and the 10 occupancy map in or along a bitstream.

An apparatus includes at least one processor; and at least one memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to: obtain a scene comprising three-dimensional information in the form of point clouds, three-dimensional meshes, two-dimensional projections of three-dimensional information, light sources, animations or any other form considered as a representation or description of three-dimensional content; extract lighting information from the obtained scene; process the extracted lighting information into at least one explicit lighting parameter and/or at least one pre-processed lighting map; and encode the scene with the at least one pre-processed lighting map and/or the at least one lighting parameter in a file format or as a visual volumetric video-based coding bitstream.

Dynamic re-lighting of volumetric video

There are disclosed a method and an apparatus for video encoding. A method can include projecting a 3D representation of at least one object onto at least one 2D patch (); generating a geometry image, a texture image, an occupancy map and auxiliary patch information from the 2D patch, wherein the auxiliary patch information comprises metadata relating to properties of the patch and one or more indicators for configuring reconstruction of the 3D representation of the at least one object (); and encoding the geometry image, the texture image, the occupancy map and the auxiliary patch information in or along a bitstream ().

An encoder may determine that volumetric video data comprises at least one time period during which a computational requirement for rendering or decoding changes at least a threshold amount; and include, in a bitstream, an indication of the change in the computational requirement for rendering or decoding of the volumetric video data during the at least one time period. A decoder may receive an indication of a change in at least one computational requirement for rendering or decoding of volumetric video data; and reallocate at least one computational resource based on the received indication.

Signaling profile level exemptions for immersive video

Apparatuses, methods, and computer programs are disclosed to implement video-based cloud compression model to world signaling. An example apparatus includes at least one processor; and at least one non-transitory memory including computer program code; wherein the at least one memory and the computer program code are configured to, with the at least one processor, cause the apparatus at least to perform: provide first signaling information comprising information related to a world domain, wherein the world domain is a point cloud frame that is represented by a number of points in a first volumetric coordinate system; and provide second signaling information comprising information related to a conversion of a model domain to the world domain, wherein the model domain represents the point cloud frame by a number of points in a second volumetric coordinate system.

Video-based point cloud compression model to world signaling information

There are disclosed various methods, apparatuses and computer program products for volumetric video encoding and decoding. In some embodiments of a method for encoding, one or more patches comprising information of a three-dimensional scene are obtained. An attribute parameter set is formed on the basis of visual properties of a surface of an object in the three-dimensional scene. Also an indication that the attribute parameter set relates to visual properties of the surface of the object and an indication on a range for which the attribute parameter set is valid.

There are disclosed various methods, apparatuses and computer program products for volumetric video encoding and decoding. In some embodiments, two or more patches formed from a three-dimensional image information are obtained, each patch representing projection data of at least a part of an object to a projection plane. A rectangle totally covering the patch is determining for each of the two or more patches. A sorting criteria is determined on the basis of a width and a height of the rectangle. The patches are sorted on the basis of the sorting criteria of the rectangles determined for the two or more patches. An initial size of a grid is selected on the basis of one or more of the largest rectangles; and the two or more patches are inserted into the grid. The grid is encoded into a bitstream.

Embodiments for volumetric video encoding and decoding relating to one or more three-dimensional objects are disclosed. In encoding, after mapping from 3D space to 2D plane () a point in the 2D plane is examined () to determine which points of the 3D object are mapped to the same point to obtain a set of candidate points. Candidate points belonging to a same surface can be used to determine a center of mass for the surface (). A depth value of the centre of mass is mapped to a 2D projection depth plane (). A colour value for the centre of mass is interpolated from colour values of points of the set of surface points which are nearest neighbours of the center of mass (), and used as the colour of the surface in the texture plane (). Corresponding embodiments for decoding are provided.

Growing community of inventors

Unterhaching, Germany

Sebastian Schwarz