Koninklijke Kpn N.v.

Nederlandse Organisatie Voor Toegepast-natuurwetenschappelijk Onderzoek Tno

Omar Aziz Niamut

Mattijs Oskar Van Deventer

Hans Maarten Stokking

Emmanuel Thomas

Fabian Arthur Walraven

Ray Van Brandenburg

Martin Prins

John Gerard Beerends

Victor Bastiaan Klos

Robert Koenen

Alexandre Da Silva Pratas Gabriel

Anton Havekes

Peter Veugen

Sylvie Dijkstrasoudarissanane

Van Mattijs Oskar Deventer

Sylvie Dijkstra- Soudarissanane

Ray Brandenburg Van

A method is described for transmitting a multimedia stream from a first terminal and receiving an associated second multimedia stream at a second terminal. The first and second terminal are connected to at least one gateway for enabling the transmission of the stream and the receiving of the associated stream. The method comprises: —initiating the exchange of first multimedia session information between the first terminal and the gateway, using a first protocol; —providing a trigger to the second terminal to initiate the exchange of second multimedia session information between the second terminal and the gateway, using a second protocol; —in response to providing the trigger, the second terminal initiating the exchange of second multimedia session information between the second terminal and the gateway, using a third protocol; —transmitting a first multimedia stream from the first terminal and receiving a second associated multimedia stream at the second terminal.

Method and system for transmitting a multimedia stream

A method is described for forming output video frames by a decoder device, comprising: receiving a bitstream comprising bitstream parts representing encoded media data of video tiles and tile positioning information associated with the encoded video tiles; extracting tile positioning information from the bitstream, the tile positioning information comprising tile identifiers for identifying encoded video tiles, a tile identifier being associated with one or more boundary identifiers, each identifying a boundary of a video tile identified by a tile identifier; determining a tile map based on the boundary identifiers, the tile map representing a spatial layout of video tiles in an output video frame; and forming an output video frame, including identifying parts of the bitstream representing encoded media data of video tiles identified in the tile map, decoding the identified encoded media data and copying the decoded media data in the output video frame buffer according to spatial layout.

System and methods for video encoding and decoding

Systems and methods for video encoding and decoding

The invention provides a streaming client with a streaming presentation of an event, with said presentation being obtained by dynamically switching between a plurality of media streams which represent different recordings of the event. For that purpose, source data is obtained from a plurality of stream sources. The source data is constituted by content data and/or metadata of concurrent portions of the plurality of media streams. By applying a quality assessment function to the source data, a quality score is assigned to each one of the concurrent portions. This allows selection data to be generated which enables the streaming client to access a selected one of the concurrent portions of a respective media stream. For example, a segment may be selected which provides a highest quality score. By doing so, it is therefore not required anymore for a consumer to manually switch between different media streams.

Establishing a streaming presentation of an event

A processor system and computer-implemented method are provided for generating a composite video stream which may include a non-volumetric video and a volumetric video, at least part of which is to be displayed in front of the non-volumetric video. The volumetric video may be included in the composite video stream in the form of a non-volumetric representation of the volumetric video, for example inserted into a spatial subregion of the non-volumetric video which may be occluded by the volumetric video during display. The encoding, transmission and decoding may thus not have to be modified to support volumetric video. Signaling data may be provided which may be indicative of the composite video stream containing the non-volumetric representation of the volumetric video. A processor system and computer-implemented method may be provided for rendering the composite video stream using the signaling data.

Streaming volumetric and non-volumetric video

A method of determining a bitstream is described, the method including: determining a motion vector (MV) of a current block of a current video frame of a sequence of video frames comprising video data captured by a camera, the MV defining an offset of the current block relative to a prediction block, wherein part of the offset is caused by global motion in the video data if the camera is moving relative to a scene; determining motion vector predictor MW candidates for predicting the MV, the MW candidates being determined based on MVs of already encoded blocks; determining a motion compensation vector MCV for MW candidates, determining global motion compensated MW candidates based on a difference between a MW candidate and an associated MCV; selecting a compensated MW; and, determining a motion vector difference MVD based on the compensated MW and the MW; and, using entropy coding to code, the MVD, an indication of the selected compensated MW; and, conversion information for converting a MVP into a compensated MW, into a bitstream.

Video coding based on global motion compensated motion vector predictors

A system and method are provided for managing a concurrent streaming of a first media stream () and a second media stream (). The media streams () represent different recordings of an event. The concurrent streaming of the media streams () is enabled by a resource having a resource limitation (). To address this resource limitation, when it is determined that the resource limitation is exceeded by the concurrent streaming of the first media stream () streaming to a first streaming client () and the second media stream () streaming to a second streaming client (), the first media stream () is selected for substituting the second media stream () in the streaming to the second streaming client (). As a result of the substitution, the first and the second streaming client () stream the same media stream rather than different media streams. This may have as advantage that the allocation of the resource is reduced, since the streaming of a same media stream to different streaming clients () typically represents a lower resource allocation than the streaming of different media streams to different streaming clients.

Managing concurrent streaming of media streams

A method is described of forming a bitstream by an encoder apparatus comprising: a processor of the encoder apparatus receiving a projected video frame including one or more pixel regions, the one or more pixel regions representing pixels of spherical video data projected onto one or more 2D projection surfaces of a projection model, the projection model being adapted to project spherical video data onto a rectangular 2D surface of a video frame; the processor generating a plurality of different candidate video frames on the basis of the received projected video frame, each candidate video frame being formed by applying at least one operation to one or more groups of pixels in one or more pixel regions of the projected video frame, wherein said at least operation includes a shift, mirror and/or rotation operation; and, wherein said operation is different for each of said plurality of different candidate video frames; the processor applying a rate distortion analysis to each of the candidate video frames to determine a rate distortion cost for each of the candidate video frames; and, the processor selecting a candidate video frame from said plurality of candidate video frames on the basis of the rate distortion cost; and, generating an output bitstream on the basis of the selected candidate video frame.

Coding spherical video data

A client device is facilitated to synchronize streaming and processing a second stream to streaming and processing a first stream. Both streams may share a common content timeline. The second stream may be buffered in a data storage. A request for streaming a content part of the second stream may be received. The request may be indicative of a selected time point on the common content timeline from which point onwards the second stream is to be processed. The second stream may then be transcoded to obtain a transcoded second stream. The transcoding may create a random access point in the transcoded second stream from which point onwards the client device is able to decode the transcoded second stream, wherein the random access point is nearer in time to the selected time point than any random access point of the second stream. The transcoded second stream may then be streamed to the client device instead of the non-transcoded second stream, thereby providing the client device with direct access to a part, such as a frame, of said second stream.

Synchronizing processing between streams

A system and method are provided for generating an output video, such as a video panorama, from a plurality of video streams representing different recordings of a scene. The plurality of video streams may be analyzed to identify at least one part of at least one of the plurality of video streams which is to be used in the output video, thereby identifying a contributing part of a video stream. Orchestration metadata may be generated identifying the contributing part. The orchestration metadata may be provided to a stream source from which the video stream originated to enable the stream source to selectively stream the contributing part of the video stream. Effectively, a selection of the stream's video data may be made to avoid or reduce unnecessary bandwidth usage.

Generating output video from video streams

A method for processing spherical video data is described including a data processor providing spherical video data in the form of projected video frames to an input of an encoder process, a projected video frame including one or more pixel regions, the one or more pixel regions representing pixels of spherical video data projected onto one or more 2D projection surfaces of a projection model, preferably an equirectangular or a 3D polyhedron projection model, for projecting spherical video data onto a rectangular 2D surface of a video frame; wherein the providing of spherical video data may comprise: generating a set of modified projected video frames on the basis of a projected video frame, the generating including applying different shift and/or rotation operations to pixels, preferably one or more pixel regions, of the selected projected video frame; applying an image analysis algorithm to the modified projected video frames, the image analysis algorithm determining an image analysis parameter for a least part of the modified projected video frames, the image analysis parameter being indicative for the encoding efficiency and/or image quality; and, selecting a modified projected video frame from the set of modified projected video frames, the selected modified projected video frame being associated with an image analysis parameter indicative of the best encoding efficiency and/or image quality.

Processing spherical video data

A method of facilitating capturing visual information at a first location () for display at a second location () is disclosed. At the first location, a virtual reality device () can be configured to render visual information representing a virtual environment, and at least one capturing device () can be configured to capture information representing a real environment comprising the virtual reality device (). At the second location, at least one monitor () can be configured to render the information captured by the at least one capturing device (). The method comprises determining an orientation of at least one capturing device () relative to the virtual reality device (), and providing in the virtual environment a visual indication of said orientation.

Capturing and rendering information involving a virtual environment

A method of forming a video signal by a decoder device is described wherein the method comprises receiving a bitstream and decoder information, the decoder information signaling the decoder device the presence of one or more resolution components in the bitstream for forming a video signal, a resolution component representing a spatially subsampled version, preferably a polyphase subsampled version, of a first video signal having a first resolution, the one or more resolution components being part of a resolution component group, the group comprising a plurality of resolution components on the basis of which the first video signal is reconstructable; and, the decoder device parsing the bitstream and decoding the one or more resolution components into video frames on the basis of the decoder information.

Spatial scalable video coding

Methods and systems are described for retrieving segmented content originating from two or more content delivery networks. The method may comprise the steps of: retrieving a manifest file template associated with said segmented content comprising one or more pointers to database entries of said first content location database; using said one or more database entries for determining a location of a delivery node in said first or second content delivery network, said delivery node being configured for delivering a segment associated with said database entry; and, generating a manifest file for locating said segmented content by replacing pointers with said determined locations.

Locating and retrieving segmented content

Methods and systems are described for enabling network-initiated control of streaming of segmented content from a content delivery node to at least one client, the client being configured to access at least part of the segmented content on the basis of a manifest file. A first manifest file is received identifying one or more segments and location information for locating one or more content delivery nodes configured to transmit one or more segments to at least one client. In response to reception of the first manifest file, channel set-up information is provided. At least one streaming control channel is established between at least one client and a control channel server function associated with the content delivery node on the basis of the control channel set-up information. The at least one client is configured for receiving at least one manifest file update message via the streaming control channel.

Growing community of inventors

Vlaardingen, Netherlands

Omar Aziz Niamut