Interdigital Madison Patent Holdings, Sas

Vid Scale, Inc.

Interdigital Vc Holdings, Inc.

Interdigital Patent Holdings, Inc.

Ahmed Hamza

Yong He

Yan Ye

Eduardo Asbun

Srinivas Gudumasu

Ali C Begen

Yuwen He

A method and apparatus include receiving a timed-metadata track identifying point cloud tiles corresponding to one or more spatial regions within a point cloud scene. A decoding device determines one or more point cloud tiles to be used for rendering an image. One or more geometry tile tracks are retrieved, via a communications network, corresponding to the determined one or more point cloud tiles. Each geometry tile track comprises point cloud geometry data for a respective tile. The retrieved geometry tile tracks are processed.

Tile tracks for geometry-based point cloud data

Systems, devices, and methods are described herein for partial access support in SOBMFF containers for video-based point cloud streams. A video encoding device may partition a 3D space into a first spatial region and a second spatial region, The video encoding device may map the first spatial region to a first set of V-PCC tiles and the second spatial region to a second set of V-PCC tiles. The video encoding device may determine a first track to carry first mapping information associated with the first spatial region that is mapped to the first set of V-PCC tiles. The video encoding device may determine a second track to carry second mapping information associated with the second spatial region that is mapped to the second set of V-PCC files. The video encoding device may send In a timed-metadata V-PCC bitstream the first track and the second track,

Partial access support in ISOBMFF containers for video-based point cloud streams

Systems, methods, and instrumentalities are disclosed for receiving content related to geometry-based point clouds. A client device may receive a media presentation description (MPD) file from a content server, A set of pre-selection elements may be identified from the MPD file. One or more adaptation sets associated with at least one pre-selection element of the set of pre-selection elements may be identified. The adaptation set(s) may be indicated by an attribute associated with one of the pre-selection elements. A geometry-based point cloud compression (GPCC) tile identifier associated with a viewport may be determined. The GPCC tile identifier may be determined based on a first descriptor received in the MPD file. One or more adaptation sets associated with the GPCC tile identifier may be selected using a second descriptor. A point cloud component associated with the selected one or more adaptation sets may be requested. Point cloud component may be received.

Adaptive streaming of geometry-based point clouds

Methods, apparatus, systems, architectures and interfaces for encoding and/or decoding point cloud bitstreams including coded point cloud sequences are provided. Included among such methods, apparatuses, systems, architectures, and interfaces is an apparatus that may include a processor and memory. A method may include any of: mapping components of the point cloud bitstream into tracks; generating information identifying any of geometry streams or texture streams according to the mapping of the components; generating information associated with layers corresponding to respective geometry component streams; and generating information indicating operation points associated with the point cloud bitstream.

Methods and apparatus for point cloud compression bitstream format

A client device adaptively streams a 360-degree video. A first segment is displayed based on a first viewing direction at a first time, where the first viewing direction is associated with a first viewport. The client requests a first base buffer segment based on the first viewport. The first base buffer segment has a presentation time after the first segment. At a second time, the viewing direction changes to a second viewing direction associated with a second viewport. The client requests, prior to the presentation time, a first viewport buffer segment based on the second viewport, with the same presentation time. The client device displays a second segment at the presentation time, wherein the second segment is either the first viewport buffer segment or the first base buffer segment. The client provides reports on viewport switching latency and on the most-requested segments.

Methods and apparatus to reduce latency for 360-degree viewport adaptive streaming

Systems, methods, and instrumentalities are disclosed that relate to the processing of a media container file associated with 3D video data. The media container file may indicate that certain video-based point cloud compression (V-PCC) component tracks may be played together as a playout group. These V-PCG component tracks may represent respective encoded versions of one or more V-PCC components, and a video decoding device may play the tracks together in response to determining that the tracks belong to the same playout track group. The video decoding device may also determine from the media container file that certain PCC component tracks include tile groups that correspond to different objects in a point cloud or different parts of a same object in the point cloud. The video decoding device may decode these tile groups independently from each other so that a subset of the objects or parts of the point cloud may be accessed without also accessing the rest of the objects or parts.

Video-based point cloud streams

Methods are described herein for signaling information regarding different viewpoints in a multi-viewpoint omnidirectional media presentation. Techniques disclosed include receiving, from a server, information identifying groups of viewpoints, including information identifying a first group of viewpoints and a second group of viewpoints. And further receiving, from the server, information identifying one or more omnidirectional videos captured from respective viewpoints belonging to the first group and information identifying one or more omnidirectional videos captured from respective viewpoints belonging to the second group. Based on the received information identifying groups of viewpoints, the disclosed techniques include transitioning from a first omnidirectional video of the one or more omnidirectional videos captured from respective viewpoints belonging to the first group to a second omnidirectional video of the one or more omnidirectional videos captured from respective viewpoints belonging to the second group.

Viewpoint metadata for omnidirectional video

Systems, methods, and instrumentalities are disclosed herein that related to video-based point cloud streams in one or more ISO Base Media File Format (ISOBMFF) container files, A container format for point cloud data is provided and the container format indicates at least a relationship between a 3D region of the point cloud and one or more video-based point cloud compression (V-PCC) tracks. The V-PCC tracks may be grouped together and linked to the 3D region to allow spatial access to the 3D region.

Methods, apparatus, and systems directed to adaptive streaming of V-PCC (Video-based Point Cloud Compression) data using an adaptive HTTP streaming protocol, such as MPEG DASH. A method includes signaling the point cloud data of the point cloud in a DASH MPD including: a main AdaptationSet for the point cloud, including at least (1) a @codecs attribute that is set to a unique value signifying that the corresponding AdaptationSet corresponds to V-PCC data and (2) an initialization segment containing at least one V-PCC sequence parameter set for a representation of the point cloud; and a plurality of component AdaptationSets, each corresponding to one of the V-PCC components and including at least (1) a VPCCComponent descriptor identifying a type of the corresponding V-PCC component and (2) at least one property of the V-PCC component; and transmitting the DASH bitstream over the network.

Methods and apparatus for adaptive streaming of point clouds

A streaming device may request and download multi-layer video segments based on a number of factors including the artistic interest associated with the video segments and/or the status of a buffer area managed by the streaming device. The multi-layer segments may be coded using scalable coding techniques or a combination of scalable coding and simulcast coding techniques by which each of the video segments may be coded into one or more representations of different qualities and/or bitrates. When requesting the multi-layer segments, the streaming device may ensure that the fullness of the buffer area falls between a buffer underflow threshold and a backfilling threshold under various network conditions. The streaming device may estimate the available network bandwidth in order to facilitate the scheduling decisions. The streaming device may consider the artistic interest associated with the video segments during scheduling and may give priority to those segments with higher artistic interest.

Scheduling multiple-layer video segments

A secondary content such as an advertisement may be inserted based on users' interests in 360 degree video streaming. Users may have different interests and may watch different areas within a 360 degree video. The information about area(s) of 360 degree scenes that users watch the most may be used to select an ad(s) relevant to their interests. One or more secondary content viewports may be defined within a 360 degree video frame. Secondary content viewport parameter(s) may be tracked. For example, statistics of the user's head orientation for some time leading to the presentation of the ad(s) may be collected. Secondary content may be determined based on the tracked secondary content viewport parameter(s).

Secondary content insertion in 360-degree video

A media content processing device may decode visual volumetric content based on one or more messages, which may indicate which attribute sub-bitstream of one or more attribute sub-bitstreams indicated in a parameter set is active, The parameter set may include a visual volumetric video-based parameter set. The message indicating one or more active attribute sub-bitstreams may be received by the decoder, A decoder may perform decoding, such as determining which attribute sub-bitstream to use for decoding visual media content, based on the one or more messages, The one or more messages may be generated and sent to a decoder, for example, to indicate the deactivation of the one or more attribute sub-bitstreams. The decoder may determine an inactive attribute sub-bitstream and skip the inactive attribute sub-bitstream for decoding the visual media content based on the one or more messages.

Dynamic adaptation of volumetric content component sub-bitstreams in streaming services

Systems and methods described herein provide for monitoring of video streaming events. Techniques disclosed include rendering a first omnidirectional video associated with a first viewpoint. In response to a request to access an omnidirectional video associated with another viewpoint, a time of the request and information identifying the other viewpoint is recorded. Techniques disclosed also include identifying a time of switching into rendering of a second omnidirectional video associated with a second viewpoint, and reporting a viewpoint switching latency, representing a difference between the identified switching time and a recorded request time associated with information identifying the second viewpoint.

Monitoring of video streaming events

Systems and methods are disclosed including techniques for rendering a 360-degree media content. Techniques disclosed include receiving a 360-degree media content and associated metadata that include a classification of a first spatial region from the received content. Techniques disclosed further include determining that a detected user movement is associated with a rendering of the first spatial region and determining whether the classification associated with the first spatial region complies with a stored user preference. If the classification violates the user preference, a path for gradually shifting the content rendering from a currently rendered spatial region to a spatial region that complies with the user preference is determined, and the received content is rendered according to the determined path.

Customized 360-degree media viewing

Processing a 360-degree video content for video coding may include receiving the video content in a first geometry. The video content may include unaligned chroma and luma components associated with a first chroma sampling scheme. The unaligned chroma and luma components may be aligned to a sampling grid associated with a second chroma sampling scheme that has aligned chroma and luma components. A geometric conversion to the video content may be performed. The video content, that may comprise the aligned chroma and luma components, in the first geometry may be converted to a second geometry. The first geometry may be a stitched geometry, and the second geometry may be a coding geometry. The converted video content in the second geometry may include the chroma and luma components aligned to the sampling grid associated with the second chroma sampling scheme.

Growing community of inventors

Montreal, Canada

Ahmed Hamza