Telefonaktiebolaget Lm Ericsson (publ)

Sarel Cohen

Chris Phillips

Robert Hammond Forsman

Rickard Sjöberg

Nir Drang

The embodiments herein relate to a method performed by a server device for handling media data streams. The sever device obtains multiple streams of source media data of varying resolution each having multiple source frames. Each source frame is divided into a plurality of tiles. The server device selects a subset of tiles from least one of the streams of source media data. The selected subset of tiles is comprised in an output frame. The server device determines layout information for the selected subset of tiles in the output frame. The layout information comprises scale, source tile index and output tile index.

Methods and devices for handling media data streams

A system and method for providing quality control in immersive video during pausing of a video streaming session. In one embodiment, a paused video frame may comprise a plurality of mixed quality video tiles depending on user gaze vector information. Under pause control, the video quality of all tiles of the paused video frame is equalized such that it is of same value for all the video tiles, which may be the video quality of the tiles presented in a viewport of the client device. The paused video frame having the same quality tiles throughout is used as a replacement video frame, which is presented to the client device player for decoding and displaying instead of the mixed quality video frame while the streaming session is paused.

Video client optimization during pause

A system and method for providing immersive video based on gaze vector information. A gaze vector is obtained from a client device operating to display an immersive video asset to a user, wherein each video frame comprises an array of tiles projected on a 3-dimensional (3D) display environment viewed by the user in which the user is immersed, the gaze vector defining a gaze direction in the 3D display environment where the user is viewing (i.e., a viewport) at any particular time. Angular separations between the viewport and remaining portions of the video frame are used in selecting different video qualities of the immersive video asset such that the viewport comprises a higher video quality than the remaining portions when displayed by the client device.

Video optimization system and method based on gaze vector information

Immersive video system and method based on gaze vector information

A system and method for providing immersive video based on gaze vector information. A gaze vector is obtained from a client device operating to display a 360&#xb0; immersive video asset to a user, wherein each video frame comprises an array of tiles projected on a 3-dimensional (3D) display environment viewed by the user in which the user is immersed, the gaze vector defining a gaze direction in the 3D display environment where the user is viewing (i.e., a viewport) at any particular time. Angular separations between the viewport and remaining portions of the video frame are used in selecting different video qualities of the 360&#xb0; immersive video asset such that the viewport comprises a higher video quality than the remaining portions when displayed by the client device.

There is provided mechanisms for determining a time offset for a video segment of a video stream using metadata. The metadata comprises time information of at least one of a start time and an end time of the video segment. A method is performed by a video network node. The method comprises extracting a first video part and a second video part from the video stream. Each of the first video part and the second video part comprises a common video segment. The method comprises identifying a sequence of video frames in the first video part that represents the common video segment. The method comprises determining the time offset based on a time difference between an end-point frame of the Identify sequence of video identified sequence of video frames and the time information in the metadata.

Handling of video segments in a video stream

System and method for providing 360&#xb0; immersive video based on gaze vector information

A system and method for providing quality control in 360&#xb0; immersive video during pausing of a video streaming session. In one embodiment, a paused video frame may comprise a plurality of mixed quality video tiles depending on user gaze vector information. Under pause control, the video quality of all tiles of the tiled video frame is equalized to a highest video quality corresponding to the quality of the tiles presented in a viewport. A replacement video frame is generated having tiles with the same equalized video quality throughout the full 360&#xb0; immersive video frame, which is presented to the client device player for decoding and displaying instead of the mixed quality video frame while the streaming session is paused.

Client optimization for providing quality control in 360&#xb0; immersive video during pause

A tile selection and bandwidth optimization scheme for effectuating optimized 360&#xb0; immersive video viewing experience. Stream manifest files describing characteristics of tile groupings per frame for each tile-encoded bitstream corresponding to a particular one of a plurality of bitrate representations of a media input stream are retrieved. Each bitrate representation is encoded having a separate video quality that is related to a corresponding QP value used for the bitrate representation. Each frame of a tile-encoded bitstream comprises an array of tiles, with a plurality of frames forming a GOP structure of the tile-encoded bitstream. Tile weights based on respective tile positions and variant weights corresponding to the plurality of tile-encoded bitstreams of the media input stream are used in determining adequacy metrics for each tile/GOP combination, based on which tiles having different bitrate qualities are selected from corresponding tile-coded bitstreams in an iterative process for effectuating knapsack optimization with respect to an overall bandwidth allocation.

Tile selection and bandwidth optimization for providing 360&#xb0; immersive video

A system and method for effectuating optimized 360&#xb0; immersive video viewing experience. A media input stream is coded into a plurality of phase-encoded bitstreams in multiple bitrate representations with varying video qualities. Each phase-encoded bitstream that belongs to a particular bitrate representation includes a number (N) of frames with a specific GOP structure having a GOP size (p), wherein the plurality of phase-encoded bitstreams equals the GOP size. For each pphase-encoded bitstream, N frames are encoded as follows: at least a first frame is encoded as an I-frame; and a frame at a frame position i, for 2&#x2264;i&#x2264;N, that satisfies {i Mod (GOP size)}=p is encoded as an X-frame that has a slice header of a P-frame and includes blocks of only intra-coded data, otherwise, that frame is encoded as a P-frame having blocks of predictive-coded data (P-blocks) with a P-slice header.

System and method for encoding 360&#xb0; immersive video

A system and method for providing multisession 360&#xb0; immersive video services in a network portion having a dedicated bandwidth capacity. In one embodiment, a bandwidth optimization method is operative to determine respective bandwidth allocations for the video sessions responsive to receiving an indication of the dedicated bandwidth capacity, a plurality of weights corresponding to the video sessions and a plurality of bitrate caps configured for the video sessions. Respective bandwidth allocations may be provided to a tile selection process configured to select tiles for a corresponding video session on a frame-by-frame basis from a plurality of tile-encoded bitrate representations of a media input stream associated with the video session, wherein each tile-encoded bitrate representation is provided with a separate video quality.

System and method for providing multiple 360&#xb0; immersive video sessions in a network

A system and method for providing quality control in 360&#xb0; immersive video during pausing of a video streaming session, wherein a paused video frame may comprise a plurality of mixed quality video tiles depending on user gaze vector information. Under pause control of a video optimization node, the video quality of all tiles of the tiled video frame is equalized to a highest video quality corresponding to the quality of the tiles presented in a viewport. In one embodiment, a custom replacement video frame is generated having tiles with the same equalized video quality throughout the full 360&#xb0; immersive video frame, e.g., an X-frame, is presented to the client device player for decoding and displaying instead of the mixed quality video frame while the streaming session is paused. In another embodiment, a still image based on the custom replacement video frame is provided to the client device to display instead of the mixed quality paused video frame.

System and method for providing quality control in 360&#xb0; immersive video during pause

A system and method for effectuating optimized 360&#xb0; immersive video viewing experience. A media input stream is coded into a first set of bitstreams having different qualities, wherein each GOP starts with an intra-coded (I) frame followed by a set of frames including at least one predictive-coded (P) frame. The media input stream is also coded into a second set of bitstreams having different qualities, wherein each GOP starts with an I-frame followed by a plurality of X-frames, each X-frame having a slice header of a P-frame and comprising blocks of intra-coded data only (e.g., I-blocks). During playout of the 360&#xb0; immersive video content, when a viewport changes during mid-GOP and/or if bandwidth allocation changes, higher quality data in an X-frame is muxed into the coded video sequence for providing enhanced viewing experience with minimum latency.

Growing community of inventors

Tel Aviv, Israel

Sarel Cohen