Mediatek Corporation

Shen-Kai Chang

Jian-Liang Lin

Hung-Chih Lin

Chia-Ying Li

Chao-Chih Huang

Ya-Hsuan Lee

Peng Wang

Chi-Cheng Ju

Hui Ou Yang

Han-Liang Chou

Ting-An Lin

Cheng-Hsuan Shih

Hui Ouyang

Shaw-Min Lei

Lin Liu

Yiou-Wen Cheng

Chao-Ling Hsu

Fang-Yi Hsieh

Le Shi

Xin-Wei Shih

An exemplary video processing method includes: receiving an omnidirectional content corresponding to a sphere; obtaining a plurality of projection faces from the omnidirectional content of the sphere according to a pyramid projection; creating at least one padding region; and generating a projection-based frame by packing the projection faces and the at least one padding region in a pyramid projection layout. The projection faces packed in the pyramid projection layout include a first projection face. The at least one padding region packed in the pyramid projection layout includes a first padding region. The first padding region connects with at least the first projection face, and forms at least a portion of one boundary of the pyramid projection layout.

Method and apparatus for reducing artifacts in projection-based frame

A video processing method includes: receiving an omnidirectional content corresponding to a sphere, obtaining projection faces from the omnidirectional content, and creating a projection-based frame by generating at least one padding region and packing the projection faces and said at least one padding region in a 360 VR projection layout. The projection faces packed in the 360 VR projection layout include a first projection face and a second projection face, where there is an image content discontinuity edge between the first projection face and the second projection face if the first projection face connects with the second projection face. The at least one padding region packed in the 360 VR projection layout includes a first padding region, where the first padding region connects with the first projection face and the second projection face for isolating the first projection face from the second projection face in the 360 VR projection layout.

Method and apparatus for generating and encoding projection-based frame that includes at least one padding region and at least one projection face packed in 360-degree virtual reality projection layout

Methods and apparatus of processing 360-degree virtual reality images are disclosed. According to one method, the method receives coded data for an extended 2D (two-dimensional) frame including an encoded 2D frame with one or more encoded guard bands, wherein the encoded 2D frame is projected from a 3D (three-dimensional) sphere using a target projection, wherein said one or more encoded guard bands are based on a blending of one or more guard bands with an overlapped region when the overlapped region exists. The method then decodes the coded data into a decoded extended 2D frame including a decoded 2D frame with one or more decoded guard bands, and derives a reconstructed 2D frame from the decoded extended 2D frame.

Method and apparatus for reduction of artifacts at discontinuous boundaries in coded virtual-reality images

A video processing method includes: obtaining a plurality of projection faces from an omnidirectional content of a sphere, wherein the omnidirectional content of the sphere is mapped onto the projection faces via cubemap projection, and the projection faces comprise a first projection face; obtaining, by a re-sampling circuit, a first re-sampled projection face by re-sampling at least a portion of the first projection face through non-uniform mapping; generating a projection-based frame according to a projection layout of the cubemap projection, wherein the projection-based frame comprises the first re-sampled projection face packed in the projection layout; and encoding the projection-based frame to generate a part of a bitstream.

Method for processing projection-based frame that includes at least one projection face packed in 360-degree virtual reality projection layout

Methods and apparatus of processing cube face images are disclosed. According to embodiments of the present invention, one or more discontinuous boundaries within each assembled cubic frame are determined and used for selective filtering, where the filtering process is skipped at said one or more discontinuous boundaries within each assembled cubic frame when the filtering process is enabled. Furthermore, the filtering process is applied to one or more continuous areas in each assembled cubic frame.

Method and apparatus for selective filtering of cubic-face frames

A video processing method includes receiving a bitstream, and decoding, by a video decoder, the bitstream to generate a decoded frame. The decoded frame is a projection-based frame that has a 360-degree image/video content represented by triangular projection faces packed in a triangle-based projection layout. An omnidirectional image/video content of a viewing sphere is mapped onto the triangular projection faces via a triangle-based projection of the viewing sphere. An equator of the viewing sphere is not mapped along any side of each of the triangular projection faces.

Method and apparatus for decoding projection based frame with 360-degree content represented by triangular projection faces packed in triangle-based projection layout

Methods and apparatus of processing cube face images are disclosed. According one method, each set of six cubic faces is converted into one rectangular assembled image by assembling each set of six cubic faces to maximize a number of continuous boundaries and to minimize a number of discontinuous boundaries. Each continuous boundary corresponds to one boundary between two connected faces with continuous contents from one face to another face. Each discontinuous boundary corresponds to one boundary between two connected faces with discontinuous contents from one face to another face. The method may further comprise applying video coding to the video sequence outputting the compressed data of the video sequence. According to another method, a fully-connected cubic-face image representing an unfolded image from the six faces of the cube is generated and the blank areas are filled with padding data to form a rectangular assembled image.

Method and apparatus of image formation and compression of cubic images for 360 degree panorama display

A projection-based frame is generated according to an omnidirectional video frame and a triangle-based projection layout. The projection-based frame has a 360-degree image content represented by triangular projection faces assembled in the triangle-based projection layout. A 360-degree image content of a viewing sphere is mapped onto the triangular projection faces via a triangle-based projection of the viewing sphere. One side of a first triangular projection face has contact with one side of a second triangular projection face, one side of a third triangular projection face has contact with another side of the second triangular projection face. One image content continuity boundary exists between one side of the first triangular projection face and one side of the second triangular projection face, and another image content continuity boundary exists between one side of the third triangular projection face and another side of the second triangular projection face.

Method and apparatus for generating projection-based frame with 360-degree image content represented by triangular projection faces assembled in triangle-based projection layout

According to one method, at a source side or an encoder side, a selected viewport associated with the 360-degree virtual reality images is determined. One or more parameters related to the selected pyramid projection format are then determined. According to the present invention, one or more syntax elements for said one or more parameters are included in coded data of the 360-degree virtual reality images. The coded data of the 360-degree virtual reality images are provided as output data. At a receiver side or a decoder side, one or more syntax elements for one or more parameters are parsed from the coded data of the 360-degree virtual reality images. A selected pyramid projection format associated with the 360-degree virtual reality images is determined based on information including said one or more parameters. The 360-degree virtual reality images are then recovered according to the selected viewport.

Method and apparatus of signalling syntax for immersive video coding

For omnidirectional video such as 360-degree Virtual Reality (360VR) video, a video system that support independent decoding of different views of the omnidirectional video is provided. A decoder for such a system can extract a specified part of a bitstream to decode a desired perspective/face/view of an omnidirectional image without decoding the entire image while suffering minimal or no loss in coding efficiency.

View-independent decoding for omnidirectional video

A video processing method includes receiving a bitstream, and decoding, by a video decoder, the bitstream to generate a decoded frame. The decoded frame is a projection-based frame that has a 360-degree image/video content represented by triangular projection faces packed in an octahedron projection layout. An omnidirectional image/video content of a viewing sphere is mapped onto the triangular projection faces via an octahedron projection of the viewing sphere. An equator of the viewing sphere is not mapped along any side of each of the triangular projection faces.

Method and apparatus for decoding projection-based frame with 360-degree content represented by triangular projection faces packed in octahedron projection layout

A video processing method includes receiving an omnidirectional content corresponding to a sphere, generating a projection-based frame according to the omnidirectional content and a pyramid projection layout, and encoding, by a video encoder, the projection-based frame to generate a part of a bitstream. The projection-based frame has a 360-degree content represented by a base projection face and a plurality of lateral projection faces packed in the pyramid projection layout. The base projection face and the lateral projection faces are obtained according to at least projection relationship between a pyramid and the sphere.

Method and apparatus for generating and encoding projection-based frame with 360-degree content represented by base projection face and lateral projection faces packed in pyramid projection layout

A projection-based frame is generated according to an omnidirectional video frame and an octahedron projection layout. The projection-based frame has a 360-degree image content represented by triangular projection faces assembled in the octahedron projection layout. A 360-degree image content of a viewing sphere is mapped onto the triangular projection faces via an octahedron projection of the viewing sphere. One side of a first triangular projection face has contact with one side of a second triangular projection face, one side of a third triangular projection face has contact with another side of the second triangular projection face. One image content continuity boundary exists between one side of the first triangular projection face and one side of the second triangular projection face, and another image content continuity boundary exists between one side of the third triangular projection face and another side of the second triangular projection face.

Method and apparatus for generating projection-based frame with 360-degree image content represented by triangular projection faces assembled in octahedron projection layout

Methods and apparatus of processing 360-degree virtual reality images are disclosed. According to one method, a 2D (two-dimensional) frame is divided into multiple blocks. The multiple blocks are encoded or decoded using quantization parameters by restricting a delta quantization parameter to be within a threshold for any two blocks corresponding to two neighboring blocks on a 3D sphere. According to another embodiment, one or more guard bands are added to one or more edges that are discontinuous in the 2D frame but continuous in the 3D sphere. Fade-out process is applied to said one or more guard bands to generate one or more faded guard bands. At the decoder side, the reconstructed 2D frame is generated from the decoded extended 2D frame by cropping said one or more decoded faded guard bands or by blending said one or more decoded faded guard bands and reconstructed duplicated areas.

A video processing method includes receiving an omnidirectional content corresponding to a sphere, generating a projection-based frame according to at least the omnidirectional content and a segmented sphere projection (SSP) format, and encoding, by a video encoder, the projection-based frame to generate a part of a bitstream. The projection-based frame has a 360-degree content represented by a first circular projection face, a second circular projection face, and at least one rectangular projection face packed in an SSP layout. A north polar region of the sphere is mapped onto the first circular projection face. A south polar region of the sphere is mapped onto the second circular projection face. At least one non-polar ring-shaped segment between the north polar region and the south polar region of the sphere is mapped onto said at least one rectangular projection face.

Method and apparatus for generating and encoding projection-based frame with 360-degree content represented in projection faces packed in segmented sphere projection layout

Growing community of inventors

Hsinchu County, Taiwan

Shen-Kai Chang