Tencent America LLC

Arianne Hinds

Stephan Wenger

Rohit Abhishek

Paul Spencer Dawkins

Shan Liu

Byeongdoo Choi

There is included a method and apparatus including computer code configured to cause a processor or processors to parse a scene file to extract scene file data, send the scene file data to a converter, and translate, by the converter and based on a metadata framework, the scene file data from a first scene graph format to a second scene graph format compatible with a renderer interface, and the metadata framework comprises an organization of metadata into at least one of systems and subsystems of the systems comprising collections of information common across a plurality of scene graph formats, and the metadata is specified in an Immersive Technologies Media Format.

Independent mapping space for asset interchange using ITMF

Aspects of the disclosure provide methods and apparatuses for video encoding/decoding. In some examples, an apparatus for video decoding includes processing circuitry. The processing circuitry decodes pictures associated with views in a bitstream and determines, from a supplemental enhancement information (SEI) message, positions of multidimensional coordinates in a multidimensional space respectively for the views. In an example, a position of a view is defined in the SEI message by at least a vertical view position and a horizontal view position in the multidimensional space. Further, the processing circuitry determines a rendering picture from the pictures based on a rendering view that is defined by at least the vertical view position and the horizontal view position in the multidimensional space.

Techniques for signaling multiview view positions in SEI message

A method, computer program, and computer system is provided for streaming immersive media. Content is ingested in a first two-dimension format or a first three-dimensional format, whereby the format references a neural network. The ingested content is converted to a second two-dimensional or a second three-dimensional format based on the referenced neural network. The converted content is streamed to a client end-point, such as a television, a computer, a head-mounted display, a lenticular light field display, a holographic display, an augmented reality display, or a dense light field display.

Reference of neural network model by immersive media for adaptation of media for streaming to heterogenous client end-points

A method including parsing a scene file to extract relevant scene file data; sending the relevant scene file data to a converter; translating, by the converter, the relevant scene file data into a format compatible with a respective rendering engine; mapping the translated scene file data using the respective rendering engine into a scene representation, wherein the scene representation is compatible with an Immersive Technologies Media Format (ITMF).

Mapping architecture of immersive technologies media format (ITMF) specification with rendering engines

A method for receiving, by a plurality of user devices within a Local Area Network (LAN), a request to create a local cloud within the LAN, selecting, by one of the plurality of user devices, a master device from among the plurality of user devices, and one or more devices from among the plurality of user devices based on a resource availability, requesting, by the master device, each of the one or more devices to share resources of the device for the local cloud, and configuring, by the master device, the one or more devices as the local cloud. The master device receives a service request for a service to be performed by the local cloud, and distributes a plurality of tasks corresponding to the service to the one or more devices configured as the local cloud according to the resource availability of the one or more devices.

Method and apparatus for using local area network as service for edge cloud

Media data of a first scene graph is received. The first scene graph is created based on a first scene graph format. The first scene graph format includes a plurality of description components indicating attributes of the first scene graph. A plurality of metadata identifiers associated with the first scene graph format and a plurality of metadata identifiers associated with a second scene graph format are determined based on a lookup table. The lookup table identifies metadata identifiers associated with the first and second scene graph formats. A plurality of description components of the second scene graph format that indicates the attributes of the first scene graph is determined based on (i) the plurality of description components of the first scene graph format, (ii) the plurality of metadata identifiers associated with the first scene graph format, and (iii) the plurality of metadata identifiers associated with the second scene graph format.

Scene graph translation

Method, device, and non-transitory storage medium for generating crowdsourced holographic views are provided. The method may include receiving one or more images of a same scene from one or more imaging devices, where each of the one or more imaging devices is at different locations. The method may further include generating a 3D model of the same scene based on the one or more images and streaming immersive media for rendering a holographic display of the same scene based on the 3D model based on determining that a display device is capable of displaying immersive media.

Generating holographic or lightfield views using crowdsourcing

Analyzing the complexity of an object of a scene in a media steam (or media data) performed by at least one processor, is provided, including receiving immersive media data comprising a plurality of scenes from a content source; obtaining a respective object of a respective scene in the plurality of scenes, from the immersive media data; analyzing the respective scene to generate complexity information associated with the respective object of the respective scene; generating metadata associated with the respective object of the respective scene, the metadata comprising the complexity information; and determining whether to distribute the respective scene to a client for processing based on the generated metadata.

Immersive media data complexity analyzer for transformation of asset formats

A method of streaming immersive media includes receiving, from a client end-point, a video content request that identifies video content; transmitting, to the client end-point based on the video content request, a profile request; receiving, from the client end-point, at least one neural network model associated with one or more characteristics of the client end-point; ingesting the identified video content in a two-dimensional video format; converting the ingested video content to a three-dimensional video format based on the at least one neural network model such that the ingested video content converted to the three-dimensional video format includes at least one scene interpolated from one or more scenes of the ingested video content in the two-dimensional video format; and streaming the converted video content to the client end-point.

Reference of neural network model for adaptation of 2D video for streaming to heterogeneous client end-points

Method, device, and non-transitory storage medium for adaptive streaming of immersive media are provided. The method may include determining characteristics associated with a scene to be transmitted to the end client, adjusting at least a part of the scene to be transmitted to the end client based on the determined characteristics, and transmitting an adaptive stream of the lightfield or holographic immersive media comprising the adjusted scene based on the determined characteristics.

Adaptive streaming for lightfield or holographic media

Packaging media for optimizing immersive media distribution of a media steam performed by at least one processor, is provided, including: receiving immersive media data for an immersive presentation; acquiring asset information associated with media assets corresponding to a set of scenes included in the immersive media data for the immersive presentation; analysing characteristics of the media assets based on the asset information, the characteristics comprising an asset type associated with a respective media asset and a frequency that indicates a number of times the respective media asset is used among the set of scenes included in the immersive presentation; ordering the media assets in a sequence based on the asset type and the frequency associated with each of the media assets; and streaming the immersive media data for the immersive presentation based on the ordered sequence of the media assets.

Immersive media streaming prioritized by frequency of asset reuse

A method for packaging media for optimizing media distribution in a media streaming network may be provided. The method may include: receiving, by a media streaming server, an immersive media stream comprising one or more immersive media assets associated with one or more scenes; identifying a subset of the one or more immersive media assets that comprise essential elements of a respective scene; ordering the one or more immersive media assets in a sequence, based on the identified subset of the one or more immersive media assets that comprise essential elements of a respective scene in the one or more scenes; and streaming the one or more immersive media assets in the ordered sequence from the media streaming server to a client device.

Immersive media streaming prioritized by asset complexity

A method is performed by a network node connected to one or more additional network nodes via a local area network (LAN). The network node and the one or more additional network nodes form a cloud. The method includes setting a first timer to a first value, the first timer being associated with determining whether one of the one or more additional network nodes is an active controller in the cloud. The method includes starting the first timer. The method includes determining, before the first timer expires, whether a first message that indicates the one of the one or more additional network nodes is the active controller is received. Based on determining the first message is received before the first timer expires, the method includes synchronizing with the one of the one or more additional network nodes that is the active controller, and restarting the first timer.

Method and apparatus for using nonstop controller with local area network (LAN) for local cloud

Aspects of the disclosure provide methods and apparatuses for media processing. In some examples, an apparatus includes processing circuitry. The processing circuitry can exchange, with a server device, a plurality of control messages over a control plane channel that uses a first transport protocol. The plurality of control messages belongs to a control plane of a bidirectional protocol for immersive media distribution. The processing circuitry receives, from the server device, a first plurality of data messages over a first data plane channel that uses a second transport protocol. The first plurality of data messages belongs to a data plane of the bidirectional protocol and carries immersive media content. The processing circuitry presents the immersive media content carried by the first plurality of data messages.

Bidirectional presentation datastream using control and data plane channels

Optimizing media distribution of a media steam performed by at least one processor, is provided, including: receiving immersive media data for an immersive presentation from a content source; acquiring asset information corresponding to media assets in a set of scenes included in the immersive media data; analyzing characteristics of the media assets, from the asset information, used in the set of scenes to determine whether a respective media asset is unique among the media assets; and generating metadata information that uniquely identifies the respective media asset for reusing the respective media asset in the set of scenes, based on determining that the respective media asset is not unique among the media assets in the set of scenes included in the immersive media data for the immersive presentation.

Growing community of inventors

Palo Alto, CA, United States of America

Arianne Hinds