Apple Inc.

Fabrice A Robinet

Khaled Mammou

Alexandros Michael Tourapis

Yeping Su

Jungsun Kim

Valery G Valentin

David William Singer

Hsi-Jung Wu

Andrea Cremaschi

Maneli Noorkami

Afshin Taghavi Nasrabadi

David Flynn

Guy Cote

Jim Chen Chou

Alexandras Tourapis

Thomas Goossens

A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. Additionally, an encoder is configured to signal and/or a decoder is configured to receive a supplementary message comprising volumetric tiling information that maps portions of 2D image representations to objects in the point. In some embodiments, characteristics of the object may additionally be signaled using the supplementary message or additional supplementary messages.

Point cloud compression with supplemental information messages

A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. In some embodiments, a bit stream structure may be used to communicate compressed point cloud data. The bit stream structure may include point cloud compression network abstraction layer (PCCNAL) units that enable use of groups of frames (GOFs), frame, and sub-frame signaling of patch information. Such a bit stream structure may permit low delay streaming and random access reconstruction of point clouds amongst other applications.

Bit stream structure for compressed point cloud data

An example device includes one or more processors, and memory storing instructions that when executed by the processors, cause the processors to receive points that represent a point cloud in three-dimensional space, and generate a data structure representing the point cloud. Generating the data structure includes encoding a position of each point in each dimension as a sequence of bits according to a tree data structure; partitioning each of the sequences into two or more portions according to a scaling depth; determining that a subset of the points is spatially isolated from a remainder of the points; quantizing each of the portions associated with the subset of the points according to a first quantization step size; quantizing each of the portions associated with the remainder of the points according to a second quantization step size; and including the quantized portions in the data structure.

In-tree geometry quantization of point clouds

An example method includes receiving a plurality of points that represent a point cloud; representing a position of the point in each dimension of a three-dimensional space as a sequence of bits, where the position of the point is encoded according to a tree data structure; partitioning at least one of the sequences of bits into a first portion of bits and a second portion of bits; quantizing each of the second portions of bits according to a quantization step size, where the quantization step size is determined according to an exponential function having a quantization parameter value as an input and the quantization step size as an output; and generating a data structure representing the point cloud and including the quantized second portions of bits.

A decoding computing device receives a bit stream for compressed 3D volumetric content. The bit stream includes video encoded image frames comprising packed attribute patch images and depth maps for the 3D volumetric content. Instead of generating a mesh having a vertex for each depth value signaled in the depth map, the decoder performs a real-time mesh simplification process to reduce a resolution of the mesh, such that the mesh resolution is reduced without exceeding an error threshold, which may be dynamically determined. Additionally, the decoder may perform a re-meshing of particular regions of the mesh for the 3D volumetric content to avoid cracks or gaps.

Real time simplification of meshes

A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. A point cloud attribute transfer algorithm may be used to determine distortion between an original point cloud and a reconstructed point cloud. Additionally, the point cloud attribute transfer algorithm may be used to select attribute values for a reconstructed point cloud such that distortion between an original point cloud and a reconstructed version of the original point cloud is minimized.

Point cloud attribute transfer algorithm

A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. The encoder is configured to convert a point cloud into an image based representation. The encoder packs patch images into an image frame and fills empty spaces in the image frame with a padding. The encoder is also configured to determine quantized minimum depths and/or maximum depths patch images in the image frames, wherein depth information is signaled relative to the quantized minimum depth.

Quantized depths for projection point cloud compression

An encoder encodes three-dimensional (3D) volumetric content, such as immersive media, using video encoded attribute patch images packed into a 2D atlas to communicate the attribute values for the 3D volumetric content. The encoder also uses mesh-encoded sub-meshes to communicate geometry information for portions of the 3D object or scene corresponding to the attribute patch images packed into the 2D atlas. The encoder applies decimation operations to the sub-meshes to simplify the sub-meshes before mesh encoding the sub-meshes. A distortion analysis is performed to bound the level to which the sub-meshes are simplified at the encoder. Mesh simplification at the encoder reduces the number of vertices and edges included in the sub-meshes which simplifies rendering at a decoder receiving the encoded 3D volumetric content.

3D volumetric content encoding using 2D videos and simplified 3D meshes

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute information for the point cloud. Attribute values for at least one starting point are included in a compressed attribute information file and attribute correction values used to correct predicted attribute values are included in the compressed attribute information file. Attribute values are predicted based, at least in part, on attribute values of neighboring points and distances between a particular point for whom an attribute value is being predicted and the neighboring points. The predicted attribute values are compared to attribute values of a point cloud prior to compression to determine attribute correction values. A decoder follows a similar prediction process as an encoder and corrects predicted values using attribute correction values included in a compressed attribute information file.

Point cloud compression

A system comprises an encoder configured to compress attribute information and/or spatial for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. To compress the attribute and/or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud.

An example method includes receiving () a plurality of points that represent a point cloud; representing a position of the point in each dimension of a three-dimensional space as a sequence of bits (), where the position of the point is encoded according to a tree data structure; partitioning () at least one of the sequences of bits into a first portion of bits and a second portion of bits; quantizing () each of the second portions of bits according to a quantization step size, where the quantization step size is determined according to an exponential function having a quantization parameter value as an input and the quantization step size as an output; and generating () a data structure representing the point cloud and including the quantized second portions of bits.

An encoder is configured to compress point cloud geometry information using an octree geometric compression technique that utilizes a binary arithmetic encoder, a look-ahead table, a cache, and a context selection process, wherein encoding contexts are selected based, at least in part, on neighborhood configurations. In a similar manner, a decoder is configured to decode compressed point cloud geometry information utilizing a binary arithmetic encoder, a look-ahead table, a cache, and a context selection process.

Point cloud geometry compression using octrees and binary arithmetic encoding with adaptive look-up tables

A system comprises an encoder configured to compress attribute information and/or spatial information for a point cloud and/or a decoder configured to decompress compressed attribute and/or spatial information for the point cloud. The encoder is configured to convert a point cloud into an image based representation. The encoder packs patch images into an image frame and fills empty spaces in the image frame with a padding. Various compression strategies may be used to encode an occupancy map and related block-to-patch information indicating which portions of the image frame correspond to which packed patches. Packed image frames comprising patches and padding, along with an encoded occupancy map and related block-to-patch information are sent to a decoder. The decoder is configured to generate a decompressed point cloud based on the packed image frames, compressed occupancy map, and related block-to-patch information.

Occupancy map block-to-patch information compression

A system comprises an encoder configured to compress attribute information for a point cloud and/or a decoder configured to decompress compressed attribute for the point cloud. To compress the attribute information, attribute values are predicted using one of a plurality of prediction strategies, wherein a selected prediction strategy is selected based at least in part on attribute variability of points in a neighborhood of points. A decoder follows a similar prediction process. Also, attribute correction values may be determined to correct predicted attribute values and may be used by a decoder to decompress a point cloud, wherein the decoder applies the same prediction strategy applied at the encoder.

Growing community of inventors

Sunnyvale, CA, United States of America

Fabrice A Robinet