Tencent America LLC

Weiwei Feng

Shan Liu

Bing Jian

Xiang Zhang

Wen Gao

Xiaozhong Xu

Jun Tian

Kelin Liu

Haichao Zhu

Lu He

Byeongdoo Choi

Meng Xu

Fai Yeung

Shalin Mahendra

Dan Miao

Ye Wang

Methods and apparatuses of encoding a video stream encoded using video point cloud coding include obtaining a source point cloud; generating an occupancy map including one or more pixels based on the source point cloud, an occupancy value associated with each pixel being a non-binary value; encoding the occupancy map to generate an encoded occupancy map, wherein a block of the encoded occupancy map corresponds to the one or more pixels of the occupancy map; selecting an occupancy value of the block based on the one or more pixels; and generating an encoded video stream based on the selected occupancy value.

Non-binary occupancy map for video based point cloud coding

Methods and apparatuses of encoding a video stream using video point cloud coding include obtaining a source point cloud; obtaining a geometry-reconstructed point cloud, wherein the geometry-reconstructed point cloud is reconstructed using lossy coding; obtaining a target point included in the geometry-reconstructed point cloud; performing at least one of a forward K-dimensional (KD)-tree search to determine a first plurality of points of the source point cloud which are nearest neighbors to the target point, and a backward KD-tree search to determine a second plurality of points of the source point cloud for which the target point is a nearest neighbor; determining an average color value based on at least one of a first average color value of the first plurality of points and a second average color value of the second plurality of points; selecting a color value for the target point based on the average color value; and generating an encoded video stream based on the selected color value.

Fast recolor for video based point cloud coding

A method for 3D human model reconstruction and animation includes receiving a two-dimensional (2D) image of a human, segmenting the 2D image into a foreground with the human and a background without the human in the 2D image, generating a parametric model comprising a pose, a shape, and one or more rigging parameters based on the human in the foreground, predicting a textured three-dimensional (3D) model using implicit surface reconstruction of the human in the foreground, aligning the parametric model and the textured 3D model using a 3D registration, and generating a textured 3D clothed human model based on the aligned parametric model and the predicted textured 3D model.

Methods of 3D clothed human reconstruction and animation from monocular image

A method, computer program, and computer system is provided for 3D pose graph reconstruction. 3D reconstruction of indoor buildings for VR/AR applications, such as virtual touring, digital museum, and virtual home sale, may be performed based on estimating pose graph and transformation matrix between cameras by recognizing markers on the ground in panorama images. Image data corresponding to one or more views of a first camera is received. One or more markers corresponding to a second camera are identified within the received image data. A pose graph corresponding to the one or more views of the first camera is constructed based on the identified one or more markers, the pose graph comprising at least an edge.

Methods of estimating pose graph and transformation matrix between cameras by recognizing markers on the ground in panorama images

Reconstructing a coded current picture for video decoding including decoding picture partitioning information corresponding to the current picture; determining whether padding is applied to sub-regions of the current picture using the picture partitioning information; based on determining that padding is not applied, decoding the sub-regions without padding the sub-regions; based on determining that padding is applied, determining whether the padding includes wrap-around padding using the picture partitioning information; based on determining that the padding does not include wrap-around padding, applying repetition padding to the sub-regions, and decoding the sub-regions using the repetition padding; based on determining that the padding includes wrap-around padding, applying the wrap-around padding to the sub-regions, and decoding the sub-regions using the wrap-around padding; and reconstructing the current picture based on the decoded sub-regions.

Method for wrap-around padding for omnidirectional media coding

Methods and apparatuses of encoding a video stream using video point cloud coding include obtaining a source point cloud; generating a geometry map and a texture map based on the source point cloud; generating a reconstructed geometry map and a reconstructed texture map using lossy coding without using recoloring, wherein a resolution of the reconstructed geometry map is same as a resolution of the reconstructed texture map; obtaining a point of the source point cloud corresponding to a geometry pixel of the geometry map and a texture pixel of the texture map; selecting a color of the texture pixel based on a color of the obtained point; and generating an encoded video stream based on the selected color, the geometry map, and the texture map.

Conditional recolor for video based point cloud coding

A method, computer program, and computer system is provided for coding video data. Video data including a reference view and a current view is received. A co-located block in the reference view is identified for a current block in the current view. A predicted block vector is calculated based on an offset vector between the current block and the co-located block, and a disparity vector between the co-located block and the reference block in the reference view. The video data is encoded/decoded based on the calculated predicted block vector.

Methods of intra picture block prediction for multi-view video compression

Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for generating a three-dimensional (3D) geometry of a room from a panorama image of the room. An apparatus includes processing circuitry that determine two-dimensional (2D) positions of wall corner points of the room in the panorama image based on a user input. Each wall corner point is in one of a floor plane or a ceiling plane of the room. The processing circuitry calculates 3D positions of the wall corner points based on the 2D positions of the wall corner points, a size of the panorama image, and a distance between the floor plane and a capture position of a device capturing the panorama image, determines a room layout based on an order of the wall corner points, and generates the 3D geometry based on the room layout and the 3D positions of the wall corner points.

Method and apparatus for constructing a 3D geometry

Aspects of the disclosure include methods, apparatuses, and non-transitory computer-readable storage mediums for generating a floor plan from a point cloud model. An apparatus includes processing circuitry that receives an input three-dimensional point cloud corresponding to a three-dimensional space. The processing circuitry determines a plurality of wall planes in the received input three-dimensional point cloud. The processing circuitry generates a plurality of line segments. Each line segment is generated by projecting a respective wall plane of the plurality of wall planes to a floor plane in the three-dimensional space. The processing circuitry represents the plurality of wall planes in the three-dimensional space using the plurality of line segments in a two-dimensional space corresponding to the floor plan. The processing circuitry adjusts the plurality of line segments in the two-dimensional space to improve the floor plan. The processing circuitry generates the floor plan based on the plurality of adjusted line segments.

Method and apparatus for generating a floor plan

A method, computer program, and computer system is provided for coding video data. Video data including (1) a first frame comprising a first current view and a first reference view and (2) a second frame comprising a second current view and a second reference view is received. A co-located block in the first reference view is identified for a current block in the first current view. A predicted offset vector is calculated based on the motion vector between the co-located block and its reference block in the second reference view. The video data is encoded/decoded based on the predicted offset vector.

Methods of inter picture motion prediction for multi-view video compression

An approach for streaming a coded virtual reality (VR) video stream including receiving a segments of the coded VR video stream; storing the segments in a playback buffer; based on determining that a current playback time is within a threshold time of a playback time of a buffered segment, that a current duration of the playback buffer is larger than a threshold duration, and that a current bandwidth is larger than a threshold bandwidth, and that a current viewport is different from a previous viewport, storing at least one refined tile corresponding to the current viewport in the playback buffer; constructing a frame based on the buffered segment and the at least one refined tile corresponding to the current viewport; and decoding the coded VR video stream based on the constructed frame.

Growing community of inventors

Palo Alto, CA, United States of America

Weiwei Feng