Koninklijke Kpn N.v.

Nederlandse Organisatie Voor Toegepast-Natuurwetenschappelijk Onderzoek Tno

Koninklijke Ptt Nederland N.v.

Imec Vzw

Universiteit Gent

Mattijs Oskar Van Deventer

Ray Van Brandenburg

Hans Maarten Stokking

Omar Aziz Niamut

Fabian Arthur Walraven

Victor Bastiaan Klos

Emmanuel Thomas

John Gerard Beerends

Lucia D'Acunto

Jeroen Laarakkers

Johannes Jacobus Van Der Tol

Martin Prins

Pieter Albert Nooren

Bastiaan Wissingh

Peter Joannes Mathias Veugen

Menno Remco Bangma

Arjen Veenhuizen

Antonius Norp

Alexandre Da Silva Pratas Gabriel

Arian Koster

Anton Havekes

Hendrikus Nathaniël Hindriks

Jeroen Famaey

Robert Koenen

Simon Gunkel

Dolf Schinkel

Peter Veugen

Herman Pals

Johan P Bekooij

Adrianus Holtzer

Emmanuel Didier Rémi Thomas

Hendrikus Nathaniel Hindriks

Edsger Jan Jager

Nicolaas Wijnand Keesmaat

Ferdinand Wouter De Vries

Eduani Thomas Metz

Ronaldus Albertus Cornelis Van Riet

Johannes Marinus Maria De Nijs

Karim Mohammed El Assal

Adrianus Cornelis Gerardus Holtzer

A method is described for transmitting a multimedia stream from a first terminal and receiving an associated second multimedia stream at a second terminal. The first and second terminal are connected to at least one gateway for enabling the transmission of the stream and the receiving of the associated stream. The method comprises: —initiating the exchange of first multimedia session information between the first terminal and the gateway, using a first protocol; —providing a trigger to the second terminal to initiate the exchange of second multimedia session information between the second terminal and the gateway, using a second protocol; —in response to providing the trigger, the second terminal initiating the exchange of second multimedia session information between the second terminal and the gateway, using a third protocol; —transmitting a first multimedia stream from the first terminal and receiving a second associated multimedia stream at the second terminal.

Method and system for transmitting a multimedia stream

A processor system and computer-implemented method may be provided for generating a composite video stream which may combine a background video and a foreground video stream into one stream. For that purpose, a spatially segmented encoding of the background video may be obtained, for example in the form of a tiled stream. The foreground video stream may be received, for example, from a(nother) client device. The foreground video stream may be a real-time stream, e.g., when being used in real-time communication. The image data of the foreground video stream may be inserted into the background video by decoding select segments of the background video, inserting the foreground image data into the decoded background image data of these segments, and by encoding the resulting composite image data to obtain composite segments which, together with the non-processed segments of the background video, form a spatially segmented encoding of a composite video.

Generating composite video stream for display in VR

Processor systems and computer-implemented methods are provided for rendering a multiuser virtual environment in which different panoramic images are provided as image-based backdrop for different viewing positions in the virtual environment. When a user switches from a first viewing position for which a first panoramic image is rendered to a second viewing position for which a second panoramic image is rendered, a transition may be rendered for the user, for example as a mathematical transformation of the panoramic images. To avoid other users perceiving the representation of the user in the virtual environment from abruptly switching to the second viewing position, transition data may be provided which enables another processor system to render a representation of the transition in the virtual environment, e.g., by moving an avatar of the user along a movement trajectory.

Generating and signaling transition between panoramic images

A method is described for processing an omnidirectional video by a client device, said omnidirectional video being associated with a 3D coordinate system for determining a spatial position of video data of said omnidirectional video on a curved surface, preferably a sphere, said method comprising: receiving a manifest file, the manifest file comprising a plurality of tile stream identifiers for identifying a plurality of tile streams, the tile streams comprising video frames having image views, whereby the image views of video frames of different tile streams cover different regions of a 2D projection of the omnidirectional video, a region defining a tile; selecting on the basis of spatial relation information in the manifest file and on the basis of a viewpoint of a user of the client device a first tile streams associated with a first resolution and a first tile position and a second tile stream associated with a second resolution and a second tile position, the second resolution being lower than the first resolution, wherein the spatial relation information defines for each tile a tile position and wherein the viewpoint defines a viewing direction of the user, tile position and the viewpoint being defined on the basis of coordinates of the 3D coordinate system.

Spatially tiled omnidirectional video streaming

A projection metadata system and method are provided for providing projection metadata to a rendering system for use in rendering a panoramic scene. A rendering system and method are provided for rendering the panoramic scene using the projection metadata. The projection metadata may comprise machine readable data which may directly provide the coordinate mapping to be used in by the rendering system to render the panoramic scene. For example, an executable script may be provided which, when executed, carries out the coordinate mapping. Compared to known ways of signalling a rendering system which projection to use, which may conventionally involve selecting a projection from a number of predefined projections and signalling the selection, the use of projection metadata may provide a high degree of flexibility, as different projections may be defined for, e.g., different spatial and/or temporal parts of the panoramic scene, different rendering techniques, etc.

Coordinate mapping for rendering panoramic scene

A method is described for providing one or more patches to a client device. The method may comprise determining manifest file version information of a manifest file that is used by the client device; detecting an indication that one or more patches have been missed, a patch comprising information for updating at least part of information in the manifest file; requesting one or more missed patches on the basis of the manifest file version information (and the version information of the manifest file that the client would obtain after having applied the last missed patch&#x2014;if available); and, receiving the one or more missed patches for updating the information in the manifest file.

Updating part of a manifest file on the basis of patches

A method for processing spherical video data is described comprising: receiving region of interest (ROI) information associated with a ROI in a spherical representation of video data, the ROI information including a spherical ROI coordinate associated with a position of the ROI; receiving a manifest file comprising one or more tile stream identifiers for identifying one or more tile streams, each tile stream comprising spherical video data of a tile on the spherical surface, each tile defining a subpart on the spherical surface, the manifest file further comprising spatial description information for signaling the client apparatus information on the position of the tiles on the spherical surface; receiving projection information for signaling the client apparatus about the type of projection that is used for projection of the spherical video data onto a planar surface; and, receiving and/or generating boundary information for defining the boundary of the ROI on the basis of the spherical ROI coordinate, and, using the ROI area for selecting one or more tile streams on the basis of the spatial description information and the projection information, wherein each of the selected tile streams defines a tile on the sphere that overlaps with the ROI area.

Processing spherical video data on the basis of a region of interest

A method of facilitating capturing visual information at a first location () for display at a second location () is disclosed. At the first location, a virtual reality device () can be configured to render visual information representing a virtual environment, and at least one capturing device () can be configured to capture information representing a real environment comprising the virtual reality device (). At the second location, at least one monitor () can be configured to render the information captured by the at least one capturing device (). The method comprises determining an orientation of at least one capturing device () relative to the virtual reality device (), and providing in the virtual environment a visual indication of said orientation.

Capturing and rendering information involving a virtual environment

A method is described for providing one or more tile streams to a client device for rendering a region-of-interest (ROI), wherein said ROI defines a region of a tile representation of a source video and wherein said method comprises: rendering a region-of-interest (ROI) video stream comprising a ROI of said source video; receiving ROI position information of at least one frame of said ROI video stream; and, receiving one or more tile streams that are requested the basis of said ROI position information and a spatial manifest file, said spatial manifest file defining one or more tile representations of said source video, a tile representation comprising one or more tile identifiers for identifying one or more tile streams and a tile position information defining the spatial position of said tile streams.

Providing tile video streams to a client

Methods and systems are described for forming one or more tile streams comprising: providing media data of at least one composite video stream to the input of at least one encoder, the media data of said composite video stream representing video frames comprising an image region having one or more subregions, each of said one or more subregions comprising visual content of one of said one or more video streams; providing encoder information to said encoder for configuring said encoder to generate encoded media data representing tiled video frames, a tile in said tiled video frames matching at least one of said one or more subregions; encoding media data of said composite video stream, wherein media data associated with a subregion of a video frame are encoded into independently encoded media data that have no spatial decoding dependency on media data of other subregions in said video frame, said independently encoded media data forming a tile of a tiled video frame, preferably said tile being arranged in said image region on the basis of a tile grid; and, storing said encoded media data as one or more tile streams on a storage medium, preferably on a storage medium of one or more network nodes, a tile stream comprising media data of at least one tile and tile position information for signaling a decoder to decode the media data of said tile stream into tiled video frames comprising a tile at a position as indicated by the tile position information.

Forming one or more tile streams on the basis of one or more video streams

A system is provided for enabling correlating timeline information between a first media stream and a second media stream which are both associated with a common play-out timeline. The system comprises at least two stream monitors arranged for providing identification information for a media stream by obtaining a persistent identification of one or more media samples of the media stream, determining a timestamp value associated with the persistent identification, and providing the timestamp value and the persistent identification as the identification information. Furthermore, a correlation subsystem is provided for combining the identification information to enable correlating timeline information of the first media stream to timeline information of the second media stream by matching a first persistent identification of the first media stream to a second persistent identification of the second media stream and thereby matching a first timestamp value of the first media stream to a second timestamp value of the second media stream. The system enables media synchronization to be provided for a plurality of devices in a segment of a media distribution network with less computational complexity and/or bandwidth.

Correlating timeline information between media streams

A method for enhancing a region of interest in video frames of a video stream is described. The method includes receiving media data defining base video frames of at least one base stream, said base stream being associated with one or more enhancement tile streams, an enhancement tile stream comprising media data defining tiled video frames comprising a tile, said tile comprising media data for enhancing visual content in a subregion of the image region of the base video frames; requesting media data of one or more enhancement tile streams, preferably one or more HEVC enhancement tile streams, the one or more enhancement tile streams being associated with media data for enhancing visual content in the at least one region of interest; generating base video frames on the basis of media data of the at least one base stream and generating tiled video frames on the basis of media data of the one or more enhancement tile streams, the tiled video frames comprising one or more tiles forming visual content the at least one region of interest; and, replacing or blending at least part of the visual content of said at least one region of interest in the video frames of said base stream with at least part of said enhanced visual content of the tiled video frames.

Enhancing a region of interest in video frames of a video stream

Methods and systems are described for retrieving segmented content originating from two or more content delivery networks. The method may comprise the steps of: retrieving a manifest file template associated with said segmented content comprising one or more pointers to database entries of said first content location database; using said one or more database entries for determining a location of a delivery node in said first or second content delivery network, said delivery node being configured for delivering a segment associated with said database entry; and, generating a manifest file for locating said segmented content by replacing pointers with said determined locations.

Locating and retrieving segmented content

Methods and systems for enabling streaming of segmented content from a content delivery network to an adaptive streaming client on the basis of a manifest file are described wherein the said manifest file may comprise one or more segment identifiers. The method may comprise: selecting at least one segment identifier from said manifest file, said identifier identifying a segment that has not been requested by said client; and, sending pre-announce information to said content delivery network for announcing in advance that said segment may be requested by said client, said pre-announce information comprising said at least one segment identifier, said pre-announce information being further arranged for triggering said content delivery network to ensure availability of said segment upon receiving said request.

Streaming of segmented content

Methods and systems are described for enabling network-initiated control of streaming of segmented content from a content delivery node to at least one client, the client being configured to access at least part of the segmented content on the basis of a manifest file. A first manifest file is received identifying one or more segments and location information for locating one or more content delivery nodes configured to transmit one or more segments to at least one client. In response to reception of the first manifest file, channel set-up information is provided. At least one streaming control channel is established between at least one client and a control channel server function associated with the content delivery node on the basis of the control channel set-up information. The at least one client is configured for receiving at least one manifest file update message via the streaming control channel.

Network-initiated content streaming control

Growing community of inventors

Leidschendam, Netherlands

Mattijs Oskar Van Deventer