AT&T Intellectual Property I, LP

Indiana University

University of Minnesota

The Directv Group, Inc.

Jackson Jarrell Pair

Bo Han

Cheuk Yiu Ip

Vijay Gopalakrishnan

Vikash Sharma

Feng Qian

Tan Xu

Dean Sakihama

Matthew T Rasmussen

Harry Porudominsky

Qingyang Xiao

A network processing system obtains a viewport of a client device for volumetric video and a two-dimensional (2D) subframe of a frame of volumetric video is obtained associated with the viewport. Viewports may be obtained from the client device or be predicted. 2D subframes and reduced resolution versions of frames can be transmitted to the client device. A client device may request volumetric video from the network processing system and provides a viewport to the network processing system. The client device may obtain from the network processing system reduced resolution versions of volumetric video frames and 2D subframes in accordance with the viewport. The client device may determine whether a current viewport matches the viewport associated with the obtained 2D subframe and provides a display based on either that subframe (upon a match) or a 2D perspective of the reduced resolution frame associated with the current viewport (if no match).

Processing system for streaming volumetric video to a client device

Aspects of the subject disclosure may include, for example, a device, that includes a processing system including a processor and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations including receiving a manifest for a point cloud, wherein the point cloud is partitioned into a plurality of cells; determining an occlusion level for a cell of the plurality of cells with respect to a predicted viewport; reducing a point density for the cell provided in the manifest based on the occlusion level, thereby determining a reduced point density; and requesting delivery of points in the cell, based on the reduced point density. Other embodiments are disclosed.

Method of determining occlusion based visibility for volumetric video streaming

Aspects of the subject disclosure may include, for example, embodiments for obtaining video content from a video content server, determining a quality parameter for each of the plurality of adjusted video content, and adjusting the video content to generate the plurality of adjusted video content. Each of the plurality of adjusted video content comprises a different point cloud density according to the quality parameter. Further embodiments can include receiving a request for the video content from a video content application on a communication device, and determining a distance from a viewpoint to a point-cloud object on the video content application of the communication device. Additional embodiments can include selecting an adjusted video content based on the distance from the viewpoint to the point-cloud object and the quality parameter of the adjusted video content, and providing the adjusted video content to the communication device for presentation. Other embodiments are disclosed.

Method, systems and devices for providing adjusted video content according to viewing distance

Aspects of the subject disclosure may include, for example, a device having a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising predicting a viewpoint within a volumetric video based on a movement input provided by a viewer, resulting in a predicted viewpoint at a future time, retrieving a cell occupancy bitmap for a point cloud of the volumetric video for the future time, determining visible cells based on the cell occupancy bitmap and the predicted viewpoint, where the visible cells are not obscured by points in other cells, and retrieving points of the point cloud that are within the visible cells prior to the future time.

System and method for streaming visible portions of volumetric video

Aspects of the subject disclosure may include, for example, identifying a field of view (FoV) within a first frame of a video that occupies a sub-portion of the frame. A size is determined by subdividing a portion of the video according to a tiling scenario. Tiles overlapping the FoV are detected, and an updated size is determined based on an encoded version of the tiles and compared to a reference size. Responsive to the updated size being less than the reference size, the reference size is adjusted to the updated size, the tiling scenario is adjusted according to an adjusted tiling scenario, and the determining of the size is repeated based on the adjusted tiling scenario. Responsive to the updated size not being less than the reference size, a preferred tiling scenario is identified based on the adjusted tiling scenario for tiling the video. Other embodiments are disclosed.

System and method for segmenting immersive video

Aspects of the subject disclosure may include, for example, obtaining a natural language instruction, interpreting the instruction to obtain a machine interpretation, and analyzing the machine interpretation to obtain an intent of the natural language instruction. An actionable command adapted to cause a digital manipulation tool to digitally manipulate a content item is determined according to the intent of the natural language instruction. The actionable command is provided to the digital manipulation tool to obtain the manipulated content item. Other embodiments are disclosed.

System and method for digital content development using a natural language interface

Aspects of the subject disclosure may include, for example, a process that analyzes image content of an immersive video frame and identifies a first group of segments of the immersive video frame based on the analysis. The segments collectively span an entire space portrayed by the immersive video frame and the first group of segments spans less than the entire space. A size of a display region of an immersive video viewer is determined that spans less than the entire space and a second group of segments is determined based on the first group of segments and the size of the display region. Transport of the second group of segments, spanning less than the entire space, is facilitated via a communication network to the immersive video viewer for presentation, without requiring transmission of all of the segments. Other embodiments are disclosed.

System and method of predicting field of view for immersive video streaming

Aspects of the subject disclosure may include, for example, encoding a point cloud in a first frame of volumetric video in an octree format and identifying a first group of points of the point cloud, resulting in a first base layer including a base layer, a first enhancement layer and a second enhancement layer. Each layer includes different points from the point cloud, with the second enhancement layer including all points in each octet of leaf nodes in the octree format of the point cloud that are not in the base layer and the first enhancement layer. The first base layer of the point cloud is provided over a communication network to a media device and the first enhancement layer is also provided responsive to a determination of sufficient available bandwidth in the communication network. Other embodiments are disclosed.

Method for scalable volumetric video coding

Aspects of the subject disclosure may include, for example, a device that has a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations including selecting a downsampling technique for removing points from a point cloud in a volumetric video; selecting a data reduction percentage for a target bitrate for streaming the volumetric video; downsampling the point cloud in each frame of the volumetric video for the target bitrate using the downsampling technique and data reduction percentage selected, thereby creating a downsampled volumetric video; and compressing and encoding the downsampled volumetric video, thereby creating an encoded volumetric video. Other embodiments are disclosed.

System and method of enabling adaptive bitrate streaming for volumetric videos

In one example, a processing system including at least one processor may obtain at least one frame of a volumetric video, obtain a first viewport of a client device, render a plurality of two-dimensional subframes comprising two-dimensional projections of the at least one frame of the volumetric video, based upon the first viewport of the client device, and transmit the plurality of two-dimensional subframes to the client device. In another example, a processing system including at least one processor may request a volumetric video from a network-based proxy, provide a viewport to the network-based proxy, obtain a plurality of two-dimensional subframes from the network-based proxy in accordance with the viewport, select at least one of the plurality of two-dimensional subframes to present, and display the at least one two-dimensional subframe that is selected.

Multi-viewport transcoding for volumetric video streaming

Aspects of the subject disclosure may include, for example, obtaining data indicative, for a frame of a video, of a field of view of a user; determining from a set of homogeneous tiles of the frame a subset of the homogeneous tiles that covers the field of view, wherein the set of homogeneous tiles has been constructed prior to the obtaining of the data; and merging at least a first tile of the set of homogeneous tiles and a second tile of the set of homogenous tiles into a larger tile, wherein the merging is based upon at least one parameter. Other embodiments are disclosed.

Apparatus, storage medium and method for heterogeneous segmentation of video streaming

Aspects of the subject disclosure may include, for example, a device having a processing system including a processor; and a memory that stores executable instructions that, when executed by the processing system, facilitate performance of operations, the operations comprising: predicting a viewpoint within a volumetric video based on movement input by a viewer, resulting in a predicted viewpoint, wherein a point cloud of the volumetric video is segmented into cells, and wherein at least some of the cells are visible to the viewer at a future time of playback based on the predicted viewpoint; retrieving a cell occupancy bitmap for the point cloud for the future time of playback; determining visible cells based on the cell occupancy bitmap and the predicted viewpoint; receiving the visible cells for the future time of playback; and rendering and displaying the point cloud within the visible cells during playback at the future time. Other embodiments are disclosed.

Aspects of the subject disclosure may include, for example, a machine-readable medium, comprising executable instructions that, when executed by a processing system including a processor, facilitate performance of operations including receiving a base layer of a first frame of a point cloud of a volumetric video encoded in an octree format over a communication network; receiving a first enhancement layer of the first frame of the point cloud of the volumetric video encoded in the octree format responsive to available sufficient bandwidth in the communication network for delivery of the first enhancement layer; receiving a second enhancement layer of the first frame of the point cloud of the volumetric video encoded in the octree format responsive to available sufficient bandwidth in the communication network for delivery of the second enhancement layer; and rendering the first frame on a display device. Other embodiments are disclosed.

Growing community of inventors

Marina Del Rey, CA, United States of America

Jackson Jarrell Pair