Tencent America LLC

University of California

Apple Inc.

Omnivision Technologies, Inc.

Xiang Zhang

Shan Liu

Wen Gao

Xiaozhong Xu

Chao Huang

Jun Tian

Sehoon Yea

Weiwei Feng

Bing Jian

Thuong Nguyen Canh

Yuan Wang

Kelin Liu

Haichao Zhu

Lu He

Brett W Degner

Paul Xiaopeng Wang

Dinesh C Mathew

Keith J Hendren

Christopher R Wilk

Michael D McBroom

David H Narajowski

Bradley Joseph Hamel

Nicholas Alan Rundle

Simon S Lee

Laura M DeForest

Hao Zhu

Daniel K Boothe

Sadao Ota

Kristopher P Laurent

Edward T Sweet

Peter N Jeziorek

Danny L McBroom

Karen Y Cheng

Lauren M Farrell

Stuart M Nevill

Sunita Venkatesh

Rebecca J Mikolajczyk

Thomas H Tsang

Chanjuan Feng

Ming Liu

David A Jungquist

Ren-min Ma

Stephan Wenger

Seong Yeol Mun

Jonathan A Ellman

Cheng Sun

Fanqing Frank Chen

Haim Suchowski

Heesoo Kang

Xiaobo Yin

Michael Mrejen

Xingjie Ni

Tongcang Li

Hamed Dalir

Yang Xia

Ren-Min Ma

Jeongmin Kim

Chihhui Wu

Kai-Hang Su

Qi-Huo Wei

Volker Jendrik Sorger

Rupert Francis Maximillian Oulton

Anique Akhtar

Aspects of the disclosure provide methods and apparatuses for mesh coding (encoding and/or decoding). In some examples, an apparatus for coding mesh includes processing circuitry. The processing circuitry decodes, three dimensional (3D) coordinates of vertices in a first 3D mesh frame and connectivity information of the vertices from a bitstream that carries the first 3D mesh frame. The first 3D mesh frame represents a surface of an object with polygons. The processing circuitry deduces texture coordinates associated with the vertices, and decodes a texture map for the first 3D mesh frame from the bitstream. The texture map includes first one or more 2D charts with 2D vertices having the texture coordinates. The processing circuitry reconstructs the first 3D mesh frame based on the 3D coordinates of the vertices, the connectivity information of the vertices, the texture map and the texture coordinates.

Mesh compression with deduced texture coordinates

A method performed by at least one processor in a decoder includes receiving a coded video bitstream that includes a compressed two dimensional (2D) mesh corresponding to a surface of three dimensional (3D) volumetric object. The method includes predicting a sampled 2D coordinate from at least one previously coded sampled 2D coordinate included in the compressed mesh. The method includes deriving a prediction residual associated with the sampled 2D coordinate. The method further includes reconstructing a 2D coordinate corresponding to a boundary vertex based on the predicted sampled 2D coordinate and the derived prediction residual.

Predictive coding of boundary UV information for mesh compression

A method and apparatus comprising computer code configured to cause a processor or processors to obtain an input mesh corresponding to volumetric data of at least one three-dimensional (3D) visual content, derive a curve of a mesh sequence from the input mesh corresponding to volumetric data, the curve including a plurality of vertices of the input mesh of the mesh sequence, subdivide the derived curve of the mesh sequence by adding subdivided vertices, computing a displacement vector for each of the plurality of subdivided vertices; and entropy code the volumetric data based on the displacement vector from at least one of the subdivided vertices.

Mesh vertex displacements coding

Method, apparatus, and system for texture coordinate prediction for mesh compression are provided. The process may include receiving, for a mesh, a coordinate of a first vertex and a coordinate of a prediction candidate vertex in a three dimensional (3D) space. The process may include determining a stretch perpendicular distance associated with the first vertex and the prediction candidate vertex, the stretch perpendicular distance being based on a conversion of the 3D space into a two dimensional (2D) space, and determining a 2D texture coordinate of the first vertex based on the stretch perpendicular distance associated with the first vertex and the prediction candidate vertex. The process may also include determining a residual of a predicted coordinate of the first vertex and an actual 2D coordinate of the first vertex; and compressing the mesh based on entropy coding the residual.

Texture coordinate prediction in mesh compression

A method and apparatus comprising computer code configured to cause a processor or processors to obtain an input mesh corresponding to volumetric data of at least one three-dimensional (3D) visual content, partitioning the input mesh into a plurality of submeshes, each of the submeshes comprising a respective one of instances of objects, determine a similarity between at least two of the instance of objects, grouping the submeshes based on the determined similarity, and code the volumetric data based on grouping the submeshes.

Methods for instance-based mesh coding

A method of encoding includes receiving a polygon mesh comprising a plurality of original vertices. The method further includes deriving an initial base mesh from the polygon mesh, the initial base mesh including a first set of base mesh vertices. The method further includes performing a symmetrize process on the initial base mesh to generate a symmetrical base mesh that includes a first side having the first set of base mesh vertices and a second side having a second set of base mesh vertices, each base mesh vertex in the first set of base mesh vertices having a corresponding symmetric vertex in the second set of base mesh vertices. The method further includes determining a first displacement between each original vertex in the polygon mesh located on the second side of the symmetrical base mesh and a nearest vertex included in the second set of base mesh vertices.

Base mesh coding by using surface reflection symmetry

A method performed by at least one processor in a decoder includes receiving a coded video bitstream that includes a compressed two dimensional (2D) mesh corresponding to a surface of three dimensional (3D) volumetric object. The method further includes predicting a current vertex included in the compressed 2D mesh based on an interpolation-based hierarchical prediction scheme that uses at least one sampled vertex included in the compressed 2D mesh. The method further includes deriving a prediction residual associated with the predicted current vertex. The method further includes reconstructing a boundary vertex associated with the 3D volumetric object based on the predicted current vertex and the derived prediction residual.

Coding of boundary UV2XYZ index for mesh compression

In some examples, an apparatus for mesh coding includes processing circuitry. The processing circuitry receives a data structure for a mesh frame with polygons representing a surface of an object. The data structure for the mesh frame includes a UV atlas that associates vertices of the mesh frame to UV coordinates in the UV atlas. The processing circuitry determines respective sampling rates for regions of the mesh frame according to respective characteristics of the regions of the mesh frame and applies, on the UV atlas, the respective sampling rates for the regions of the mesh frame to determine sampling positions on the UV atlas. The processing circuitry generates one or more sampled two dimensional (2D) maps for the mesh frame according to the sampling positions on the UV atlas, and encodes the one or more sampled 2D maps into a bitstream.

Method and apparatus of adaptive sampling for mesh compression by encoders

Method, apparatus, and system for parallel dynamic mesh alignment are provided. The process may include determining that a temporal alignment is present between one or more frames in a received input mesh including a plurality of polygons that describe a surface of a volumetric object, then, determining an intra-frame alignment scheme to spatially align charts within a frame. The process may also include applying the intra-frame alignment scheme to one or more corresponding charts in the one or more frames for inter-frame alignment; and processing the one or more frames in parallel based on the intra-frame alignment scheme and inter-frame alignment.

Parallel approach to dynamic mesh alignment

A plurality of neighboring vertices of a current vertex in a current frame of a mesh is determined. The current frame corresponds to the mesh at a first time instance. Each of the plurality of neighboring vertices is connected to the current vertex through a respective edge in the mesh. A plurality of neighboring estimation errors of the plurality of neighboring vertices is determined. Each of the plurality of neighboring estimation errors indicates a difference between a reference vertex of a corresponding one of the plurality of neighboring vertices in a reference frame and the corresponding one of the plurality of neighboring vertices in the current frame. The reference frame corresponds to the mesh at a second time instance. A prediction residue of the current vertex is determined based on the plurality of neighboring estimation errors. Prediction information of the current vertex is generated based on the determined prediction residue.

Temporal prediction based vertex position compression

Aspects of the disclosure provide methods and apparatuses for mesh coding. In some examples, an apparatus for mesh coding includes processing circuitry. The processing circuitry receives a bitstream carrying a 3D mesh frame. The 3D mesh frame represents a surface of an object with polygons, the polygons are defined by vertices and edges connecting the vertices, the 3D mesh frame is parameterized into a plurality of 2D charts in a 2D UV atlas. The processing circuitry determines respective chart based coding modes for the plurality of 2D charts according to one or more indices decoded from the bitstream, and decodes, from the bitstream, at least respective geometry information associated with the plurality of 2D charts according to the respective chart based coding modes. The processing circuitry reconstructs the 3D mesh frame according to the respective geometry information associated with the plurality of 2D charts.

Method and apparatus for chart based mesh compression

Method, apparatus, and system for atlas sampling-based dynamic mesh compression are provided. The process may include receiving a mesh frame with polygons representing a surface of an object; determining that the mesh frame is associated with an irregular UV chart in response to one or more singularity components in the mesh frame; performing boundary detection to the UV chart to identify one or more boundary loops in the UV chart; and compressing information of the identified one or more boundary loops into a bitstream.

Atlas sampling based mesh compression with charts of general topology

A method, computer program, and computer system is provided for point cloud coding. The method includes receiving, from a bitstream, data corresponding to a point cloud; obtaining from the data a first prediction residual of a first component from among a plurality of components of an attribute associated with the point cloud; reconstructing the first prediction residual; determining a predicted second prediction residual based on the reconstructed first prediction residual and at least one model parameter; obtaining a second prediction residual of a second component from among the plurality of components based on the predicted second prediction residual; reconstructing the second prediction residual; and decoding the data corresponding to the point cloud based on the reconstructed first prediction residual and the reconstructed second prediction residual.

Coding of multiple-component attributes for point cloud coding

There is includes a method and apparatus comprising computer code configured to cause a processor or processors to perform obtaining volumetric data of at least one three-dimensional (3D) visual content, obtaining a plurality of 3D meshes from the volumetric data, projecting vertices of the plurality of 3D meshes onto a plurality of two-dimensional (2D) meshes, signaling connectivity information of the plurality of 3D meshes based on the plurality of 2D meshes, wherein the signaling the connectivity information comprises signaling whether ones of pixels of a 2D occupancy map is to be inferred or explicitly signaled.

Vertex prediction based on mesh triangulation derivation

A method and apparatus that receives a coded video bitstream from an encoder; retrieves, from the coded video bitstream, a mesh including a plurality of polygons that describe one or more surfaces of a volumetric object; predicts a current value of an index of the UV-to-XYZ index array from at least one previously coded value included in another index in the UV-to-XYZ index array, wherein the UV-to-XYZ index array includes a plurality of indices which correspond each two-dimensional (2D) UV vertex associated with the mesh to a three-dimensional (3D) XYZ vertex associated with the mesh; derives a prediction residual associated with the current value of the index of the UV-to-XYZ index array; and reconstructs a boundary UV coordinate based on the predicted current value of the index of the UV-to-XYZ index array and the derived prediction residual.

Predictive coding of boundary UV2XYZ index for mesh compression

Growing community of inventors

Sunnyvale, CA, United States of America

Xiang Zhang