Tencent America LLC

Chao Huang

Shan Liu

Xiaozhong Xu

Xiang Zhang

Jun Tian

Wei Fan

Hui Tang

Lianyi Han

Shih-Yao Lin

Kun Wang

Zhen Qian

Zhimin Huo

Yusheng Xie

Min Tu

Tao Yang

Xiaozhong Chen

Thuong Nguyen Canh

Shihyao Lin

Aspects of the disclosure provide methods and apparatuses for mesh coding. In some examples, an apparatus for mesh coding includes processing circuitry. The processing circuitry receives a bitstream carrying a 3D mesh frame. The 3D mesh frame represents a surface of an object with polygons, the polygons are defined by vertices and edges connecting the vertices, the 3D mesh frame is parameterized into a plurality of 2D charts in a 2D UV atlas. The processing circuitry determines respective chart based coding modes for the plurality of 2D charts according to one or more indices decoded from the bitstream, and decodes, from the bitstream, at least respective geometry information associated with the plurality of 2D charts according to the respective chart based coding modes. The processing circuitry reconstructs the 3D mesh frame according to the respective geometry information associated with the plurality of 2D charts.

Method and apparatus for chart based mesh compression

Method, apparatus, and system for atlas sampling-based dynamic mesh compression are provided. The process may include receiving a mesh frame with polygons representing a surface of an object; determining that the mesh frame is associated with an irregular UV chart in response to one or more singularity components in the mesh frame; performing boundary detection to the UV chart to identify one or more boundary loops in the UV chart; and compressing information of the identified one or more boundary loops into a bitstream.

Atlas sampling based mesh compression with charts of general topology

A method and apparatus that receives a coded video bitstream from an encoder; retrieves, from the coded video bitstream, a mesh including a plurality of polygons that describe one or more surfaces of a volumetric object; predicts a current value of an index of the UV-to-XYZ index array from at least one previously coded value included in another index in the UV-to-XYZ index array, wherein the UV-to-XYZ index array includes a plurality of indices which correspond each two-dimensional (2D) UV vertex associated with the mesh to a three-dimensional (3D) XYZ vertex associated with the mesh; derives a prediction residual associated with the current value of the index of the UV-to-XYZ index array; and reconstructs a boundary UV coordinate based on the predicted current value of the index of the UV-to-XYZ index array and the derived prediction residual.

Predictive coding of boundary UV2XYZ index for mesh compression

Coding information of a mesh is received. The coding information includes a plurality of first coordinates and a plurality of second coordinates corresponding to a plurality of vertices and a texture map that are associated with the mesh. A respective first coordinate and a respective second coordinate associated with each of the plurality of vertices are normalized by adjusting the respective first coordinate based on a first factor and the respective second coordinate based on a second factor. The first factor and the second factor are associated with at least one of (i) a bit depth value indicating a coded range of the first coordinates and the second coordinates and (ii) a size of the texture map. The normalized respective first coordinate and the normalized respective second coordinate are expanded based on the first factor and the second factor respectively.

UV coordinate ranges and texture map size

A method and apparatus comprising computer code configured to cause a processor or processors to obtain volumetric data of at least one three-dimensional (3D) visual content, derive a first layer of a mesh including a plurality of first vertices from at least one frame of the volumetric data, determine displacement vectors from ones of the plurality of vertices of the first layer to respective ones of a plurality of second vertices of a second layer of the mesh, a total quantity of the plurality of second vertices is greater than a total quantity of the plurality of first vertices, and signal the volumetric data based on the displacement vectors.

Vertex prediction based on decoded neighbors

Method, apparatus, and system for generating temporally correlated UV atlases are provided. The process may include generating plurality of consistent UV charts based on partition information associated with one or more previous frames; removing one or more non-manifold vertices from the plurality of consistent UV charts based on assigning one or more incident faces associated with the one or more non-manifold vertices to corresponding UV charts; merging more than one of the plurality of consistent UV charts based on a similarity between the plurality of consistent UV charts and one or more reference charts associated with the one or more previous frames; and generating temporally correlated UV atlases for the current frame based on aligning the plurality of consistent UV charts with the one or more reference charts associated with the one or more previous frames.

Mesh parameterization with temporally correlated UV atlases

Aspects of the disclosure provide methods and apparatuses for mesh coding (e.g., compression and decompression). In some examples, an apparatus for mesh coding includes processing circuitry. The processing circuitry decodes, using a decoder supporting a first bitdepth, a plurality of segmental attribute values having the first bitdepth from a bitstream carrying a mesh that represents a surface of an object. The plurality of segmental attribute values is associated with attribute values of the mesh, the attribute values of the mesh have a second bitdepth that is higher than the first bitdepth. The processing circuitry determines the attribute values of the mesh having the second bitdepth according to the plurality of segmental attribute values having the first bitdepth.

Segment based compression methods for mesh compression

A method and apparatus that receives a coded video bitstream from an encoder; retrieves, from the coded video bitstream, a mesh including a plurality of polygons that describe a surface of a volumetric object; triangulates one or more vertices of the mesh, the one or more vertices are located on a boundary of the mesh or interior of the mesh; infers connectivity information between the one or more vertices using the triangulation of the one or more vertices; and reconstructs one or more boundary UV coordinates based on the connectivity information.

Triangulation methods with boundary information for dynamic mesh compression

There is includes a method and apparatus comprising computer code configured to cause a processor or processors to perform obtaining volumetric data of at least one three-dimensional (3D) visual content, obtaining a patch by dividing a plurality of 3D meshes from the volumetric data, the patch including vertices of at least one of the 3D meshes, forming a prediction group comprising a subset of the vertices of the patch, and signaling a prediction mode of the prediction group collectively for the subset of the plurality of vertices of the patch.

Dynamic mesh compression using inter and intra prediction

A plurality of vertices corresponding to a mesh of a current frame that is positionally tracked is divided into a plurality of vertex groups based on a predetermined constant integer. The plurality of vertex groups includes a first vertex group. A first position of a current vertex in the first vertex group is estimated based on a second position of a reference vertex in a reference frame that is different from the current frame. An estimation error indicating a difference between the reference vertex in the reference frame and the current vertex in the first vertex group is determined. A prediction mode of vertices in the first vertex group is encoded based at least on the estimation error between the reference vertex and the current vertex. Prediction information of the vertices in the first vertex group is generated based on the encoded prediction mode of the vertices of the first vertex group.

Grouping of mesh vertices in position compression

A plurality of candidate predictions of a current vertex is determined. The current vertex is one of a plurality of vertices of a mesh. Each of the plurality of candidate predictions is determined based on a respective one of a plurality of triangles in the mesh and including two-dimensional (2D) coordinates in a 2D domain. An average prediction of the plurality of candidate predictions is computed. A 2D prediction of the current vertex is selected from a prediction list that includes the plurality of candidate predictions and the average prediction. A three-dimensional (3D) prediction of the current vertex is determined based on one or more neighboring vertices of the current vertex in the mesh. A prediction residue comprising a prediction error of the current vertex is encoded. The prediction residue is obtained based on the selected 2D prediction and the determined 3D prediction of the current vertex.

Texture coordinate coding in mesh compression

In a method, a plurality of candidate predictions of a current vertex is determined. The current vertex is included in a plurality of vertices of a mesh. The plurality of candidate predictions is determined based on a plurality of triangles in the mesh. Each of the plurality of predictions corresponds to a respective candidate prediction. A prediction of the current vertex is determined from the plurality of candidate predictions. The prediction is identified by a prediction index that is predetermined based on a number of the plurality of candidate predictions of the current vertex being equal to or larger than a threshold value. Prediction information of the current vertex is encoded based on the determined prediction. The prediction information includes the prediction index identifying the prediction of the current vertex.

Vertex position coding in mesh compression

Whether each of a plurality of duplicate vertices is skippable in a mesh is determined. One or more skippable duplicate vertices of the plurality of duplicate vertices in the mesh are excluded to generate an updated mesh. Vertices of the updated mesh are divided into a plurality of vertex groups, where the plurality of vertex groups includes a first vertex group. A prediction mode of the first vertex group is determined at least based on estimation errors of vertices in the first vertex group of the updated mesh. Each of the estimation errors indicates a difference between a respective vertex in the first vertex group and a corresponding reference vertex in the reference frame. Prediction information of the vertices in the first vertex group is generated based on the determined prediction mode.

Duplicate vertices based position compression

Aspects of the disclosure provide methods and apparatuses for mesh coding (e.g., compression and decompression). In some examples, an apparatus for mesh coding includes processing circuitry. The processing circuitry decodes an array of attributes from a bitstream carrying a three dimensional (3D) mesh frame that includes a plurality of patches. The array of attributes corresponds to vertices of the 3D mesh frame. The vertices of the 3D mesh frame are ordered into subsets respectively belonging to the plurality of patches. The processing circuitry determines a first portion of the array of attributes corresponding to a first subset of the vertices of the 3D mesh frame. The first subset belongs to a first patch in the plurality of patches. The processing circuitry determines first connectivity information of the first subset of the vertices, and reconstructs the first patch of the 3D mesh frame based on the first portion of the array of attributes and the first connectivity information.

Vertex reordering for mesh compression

A method and apparatus that derives a curve of a mesh sequence from a frame of a video bitstream, the curve comprising a plurality of vertices; subdivides the curve by adding a plurality of subdivided vertices; calculates one or more displacement vectors between each of the plurality of vertices on the curve and each of the plurality of subdivided vertices on the subdivided curve; transforms the one or more displacement vectors to output one or more original coefficients; tracks the current frame with a previous frame to obtain a one-to-one correspondence between the plurality of vertices of the current frame and a plurality of vertices of the previous frame; predicts one or more coefficients of each of the plurality of vertices in current frame based on the one-to-one correspondence; performs entropy coding on one or more prediction residues; and encodes the one or more prediction residues.

Growing community of inventors

Palo Alto, CA, United States of America

Chao Huang