Koninklijke Kpn N.v.

Nederlandse Organisatie Voor Toegepast-natuurwetenschappelijk Onderzoek Tno

Tiledmedia B.v.

Imec Vzw

Universiteit Gent

Ray Van Brandenburg

Mattijs Oskar Van Deventer

Emmanuel Thomas

Hans Maarten Stokking

Omar Aziz Niamut

Lucia D'Acunto

Menno Remco Bangma

Martin Prins

Bastiaan Wissingh

Konstantinos Trichias

Arjen Veenhuizen

Arian Koster

Anton Havekes

Peter Joannes Mathias Veugen

Jeroen Famaey

Xavier Albert Jean-Pierre Corbillon

Eelco Cramer

Dolf Schinkel

Emmanuel Didier Rémi Thomas

Amritpal Singh Gill

Thomas Attema

Arjen Timotheus Veenhuizen

Marinus Thomas Van Eck

Frederik Harm Klok

Robert Hendrik Koenen

A computer-implemented method is disclosed for requesting tile streams for rendering a spatial part of an immersive video on a display of a client device. In this method, a server system stores a plurality of tile streams. Each tile stream represents a spatial part of the immersive video and each tile stream comprises a sequence of tile frames that are associated with respective times in the video. Further, the plurality of tile streams comprises a first set of high quality tile streams having a relatively high quality and comprises a second set of one or more low quality tile streams having a relatively low quality and/or a third set of one or more intermediate quality tile streams having a relatively intermediate quality. The method comprises based on a determined value for a parameter indicative of how many requested high quality tile frames are received not in time for properly rending them on the display, refraining, at least for some time period, from requesting high quality tile streams.

Method and system for requesting tile streams

A method is described for rendering a spatial part of an immersive video on a display of a client device comprising a tile frame retrieval device, a decoder device and a renderer device, wherein the immersive video comprises video frames, each video frame being spatially divided in tile frames, and the immersive video comprises tile streams, each tile stream representing a spatial part of the immersive video and each tile stream comprising a plurality of said tile frames, and wherein the client has stored tile stream mapping information, that indicates for each tile stream a respective position on a surface of a two-dimensional or three-dimensional model, such as a sphere or cube, the method comprising based on said tile stream mapping information and a viewport, the tile frame retrieval device determining a plurality of tile streams, and requesting encoded video data from a server, the encoded video data comprising, for each determined tile stream, an encoded tile frame that comprises encoded data representative of a tile frame comprised in the tile stream; the tile frame retrieval device receiving the encoded tile frames and forming an encoded video frame comprising the received encoded tile frames, each encoded tile frame having a position in the encoded video frame, and the tile frame retrieval device generating tile frame arrangement information indicating the position of each encoded tile frame within the encoded video frame; the decoder device decoding the encoded video frame to obtain a decoded video frame comprising the tile frames at respective positions within the decoded video frame; based on the tile frame arrangement information and based on the tile stream mapping information, the renderer device mapping the decoded video frame onto one or more surfaces of the two-dimensional or three-dimensional model so that each tile frame is mapped onto the position of the one or more surfaces of the model as indicated; based on the mapped decoded video frame, the renderer device rendering at least part of the decoded video frame on the display of the client device.

Methods and devices for rendering a video on a display

The invention provides a streaming client with a streaming presentation of an event, with said presentation being obtained by dynamically switching between a plurality of media streams which represent different recordings of the event. For that purpose, source data is obtained from a plurality of stream sources. The source data is constituted by content data and/or metadata of concurrent portions of the plurality of media streams. By applying a quality assessment function to the source data, a quality score is assigned to each one of the concurrent portions. This allows selection data to be generated which enables the streaming client to access a selected one of the concurrent portions of a respective media stream. For example, a segment may be selected which provides a highest quality score. By doing so, it is therefore not required anymore for a consumer to manually switch between different media streams.

Establishing a streaming presentation of an event

A first system () is configured to transmit an interest message, e.g. to a router (). The interest message specifies criteria identifying data units from an area of interest, e.g. an intersection, which are to be aggregated and comprises at least one parameter specifying how the data units are requested to be aggregated. A second system () is configured to receive the interest message from a sender, e.g. a router (), determine whether the system will return an aggregation of data units matching the criteria, and return a data message comprising an aggregation of data units matching the criteria to the sender () if the processing means have determined that the system will return an aggregation of data units matching the criteria. The data units are aggregated in the data message in dependence on how the data units were requested to be aggregated.

Transmitting and receiving an interest message specifying an aggregation parameter

A method is described for processing an omnidirectional video by a client device, said omnidirectional video being associated with a 3D coordinate system for determining a spatial position of video data of said omnidirectional video on a curved surface, preferably a sphere, said method comprising: receiving a manifest file, the manifest file comprising a plurality of tile stream identifiers for identifying a plurality of tile streams, the tile streams comprising video frames having image views, whereby the image views of video frames of different tile streams cover different regions of a 2D projection of the omnidirectional video, a region defining a tile; selecting on the basis of spatial relation information in the manifest file and on the basis of a viewpoint of a user of the client device a first tile streams associated with a first resolution and a first tile position and a second tile stream associated with a second resolution and a second tile position, the second resolution being lower than the first resolution, wherein the spatial relation information defines for each tile a tile position and wherein the viewpoint defines a viewing direction of the user, tile position and the viewpoint being defined on the basis of coordinates of the 3D coordinate system.

Spatially tiled omnidirectional video streaming

Encoded video data representing partial video images is decoded from a first video stream such as a HEVC bitstream. The first video stream comprises video images divided into a spatial array of independently decodable slices. The first video stream comprises network abstraction layer units that each contain video content data for a respective one of the slices preceded by a header that comprises parameters relating to the respective one of the slices in relation to the first video stream. Previously prepared information such as meta data about positions of parameters in the network abstraction layer units or a plurality of different versions of the network abstraction layer units for different partial image sizes (different picture sizes) is used. The meta data may be used for partially rewriting the network abstraction layer units for a subarray of slices, the network abstraction layer units being rewritten at a location selected on the basis of meta data, to make the network abstraction layer units consistent with a second video stream of encoded video data only for the sub-array of slices. Versions of the network abstraction layer units may be selected for the sub-array of slices, from the plurality of different versions. Subsequently, the partial video images are generated by decoding the encoded video data from the rewritten network abstraction layer units.

Partial video decoding method, device and system

The invention relates to a system for receiving an interest message. The system comprises at least one communication interface, at least one memory and at least one processor. The at least one processor is configured to receive an interest message on a first one of the at least one communication interface. The interest message comprises a name of a requested data object. At least one name component of the name of the requested data object comprises a range. The at least one processor is further configured to determine whether the at least one memory comprises a named data object having a name matching the name of the requested data object and to provide the named data object on the first one of the at least one communication interface if the memory comprises the named data object. The at least one processor is also configured to, if the memory does not comprise the named data object, forward the interest message on one or more further ones of the at least one communication interface, associate the name of the requested data object with an identifier of the first one of the at least one communication interface in the at least one memory, receive the named data object on the one or more further ones of the at least one communication interface and forward the named data object on the first one of the at least one communication interface based on the association.

Systems and methods for transmitting and receiving interest messages

A method is described for providing one or more patches to a client device. The method may comprise determining manifest file version information of a manifest file that is used by the client device; detecting an indication that one or more patches have been missed, a patch comprising information for updating at least part of information in the manifest file; requesting one or more missed patches on the basis of the manifest file version information (and the version information of the manifest file that the client would obtain after having applied the last missed patch&#x2014;if available); and, receiving the one or more missed patches for updating the information in the manifest file.

Updating part of a manifest file on the basis of patches

A method is described for providing one or more tile streams to a client device for rendering a region-of-interest (ROI), wherein said ROI defines a region of a tile representation of a source video and wherein said method comprises: rendering a region-of-interest (ROI) video stream comprising a ROI of said source video; receiving ROI position information of at least one frame of said ROI video stream; and, receiving one or more tile streams that are requested the basis of said ROI position information and a spatial manifest file, said spatial manifest file defining one or more tile representations of said source video, a tile representation comprising one or more tile identifiers for identifying one or more tile streams and a tile position information defining the spatial position of said tile streams.

Providing tile video streams to a client

Methods and systems are described for forming one or more tile streams comprising: providing media data of at least one composite video stream to the input of at least one encoder, the media data of said composite video stream representing video frames comprising an image region having one or more subregions, each of said one or more subregions comprising visual content of one of said one or more video streams; providing encoder information to said encoder for configuring said encoder to generate encoded media data representing tiled video frames, a tile in said tiled video frames matching at least one of said one or more subregions; encoding media data of said composite video stream, wherein media data associated with a subregion of a video frame are encoded into independently encoded media data that have no spatial decoding dependency on media data of other subregions in said video frame, said independently encoded media data forming a tile of a tiled video frame, preferably said tile being arranged in said image region on the basis of a tile grid; and, storing said encoded media data as one or more tile streams on a storage medium, preferably on a storage medium of one or more network nodes, a tile stream comprising media data of at least one tile and tile position information for signaling a decoder to decode the media data of said tile stream into tiled video frames comprising a tile at a position as indicated by the tile position information.

Forming one or more tile streams on the basis of one or more video streams

The invention relates to a network node () comprising at least one communication interface, at least one memory, and at least one processor. The at least one processor is configured to receive an interest message on the at least one communication interface from a downstream node (), the interest message comprising a name of a requested data object, to determine whether to bid for storing the requested data object in the at least one memory, to calculate a bid value, to forward the interest message to an upstream node () on the at least one communication interface, the interest message including the calculated bid value in dependence on at least the determination whether to bid, to receive a data message comprising the requested data object from the upstream node on the at least one communication interface, to determine from at least the data message whether the network node won the bid, to store the data object in the at least one memory if the network node won the bid, and to forward the data message to the downstream node on the at least one communication interface.

Network node, endpoint node and method of receiving an interest message

A system is provided for enabling correlating timeline information between a first media stream and a second media stream which are both associated with a common play-out timeline. The system comprises at least two stream monitors arranged for providing identification information for a media stream by obtaining a persistent identification of one or more media samples of the media stream, determining a timestamp value associated with the persistent identification, and providing the timestamp value and the persistent identification as the identification information. Furthermore, a correlation subsystem is provided for combining the identification information to enable correlating timeline information of the first media stream to timeline information of the second media stream by matching a first persistent identification of the first media stream to a second persistent identification of the second media stream and thereby matching a first timestamp value of the first media stream to a second timestamp value of the second media stream. The system enables media synchronization to be provided for a plurality of devices in a segment of a media distribution network with less computational complexity and/or bandwidth.

Correlating timeline information between media streams

A method is described of streaming HEVC-tiled video data to a client device comprising: providing said client device with a spatial manifest file comprising one or more spatial segments identifiers for locating one or more delivery nodes configured for delivering video data associated with a spatial segment identified by at least one of said one or more spatial segments identifiers, preferably (part of) one or more URLs, to said client device, a spatial segment being associated with HEVC-tiled video data comprising a subset of HEVC tiles of a HEVC-tiled (panorama) video stream; and, selecting a spatial segment identifier in said spatial manifest file for requesting a delivery node to deliver at least part of the video data of a spatial segment as a HEVC-tiled video stream to the client device.

HEVC-tiled video streaming

A method for enhancing a region of interest in video frames of a video stream is described. The method includes receiving media data defining base video frames of at least one base stream, said base stream being associated with one or more enhancement tile streams, an enhancement tile stream comprising media data defining tiled video frames comprising a tile, said tile comprising media data for enhancing visual content in a subregion of the image region of the base video frames; requesting media data of one or more enhancement tile streams, preferably one or more HEVC enhancement tile streams, the one or more enhancement tile streams being associated with media data for enhancing visual content in the at least one region of interest; generating base video frames on the basis of media data of the at least one base stream and generating tiled video frames on the basis of media data of the one or more enhancement tile streams, the tiled video frames comprising one or more tiles forming visual content the at least one region of interest; and, replacing or blending at least part of the visual content of said at least one region of interest in the video frames of said base stream with at least part of said enhanced visual content of the tiled video frames.

Enhancing a region of interest in video frames of a video stream

Methods and systems are described for retrieving segmented content originating from two or more content delivery networks. The method may comprise the steps of: retrieving a manifest file template associated with said segmented content comprising one or more pointers to database entries of said first content location database; using said one or more database entries for determining a location of a delivery node in said first or second content delivery network, said delivery node being configured for delivering a segment associated with said database entry; and, generating a manifest file for locating said segmented content by replacing pointers with said determined locations.

Growing community of inventors

The Hague, Netherlands

Ray Van Brandenburg