Koninklijke Kpn N.v.

Nederlandse Organisatie Voor Toegepast-natuurwetenschappelijk Onderzoek Tno

Koninklike Kpn N.v.

Martin Prins

Hans Maarten Stokking

Omar Aziz Niamut

Emmanuel Thomas

Ray Van Brandenburg

Anton Havekes

Mattijs Oskar Van Deventer

Victor Bastiaan Klos

Peter-Jan Doets

Hendrikus Nathaniël Hindriks

Simon Gunkel

Robert-Ernst Kooij

Frank Bart Ter Haar

Robert Koenen

Simon Norbert Bernhard Gunkel

Hendrikus Nathaniel Hindriks

Robert-Enrst Kooij

Nanda Van Der Stap

Victor Klos

The invention provides a streaming client with a streaming presentation of an event, with said presentation being obtained by dynamically switching between a plurality of media streams which represent different recordings of the event. For that purpose, source data is obtained from a plurality of stream sources. The source data is constituted by content data and/or metadata of concurrent portions of the plurality of media streams. By applying a quality assessment function to the source data, a quality score is assigned to each one of the concurrent portions. This allows selection data to be generated which enables the streaming client to access a selected one of the concurrent portions of a respective media stream. For example, a segment may be selected which provides a highest quality score. By doing so, it is therefore not required anymore for a consumer to manually switch between different media streams.

Establishing a streaming presentation of an event

A method and processor system are provided which analyze a depth map, which may be obtained from a range sensor capturing depth information of a scene, to identify where an object is located in the scene. Accordingly, a region of interest may be identified in the scene which includes the object, and image data may be selectively obtained of the region of interest, rather than of the entire scene containing the object. This image data may be acquired by an image sensor configured for capturing visible light information of the scene. By only selectively obtaining the image data within the region of interest, rather than all of the image data, improvements may be realized in the computational complexity of a possible further processing of the image data, the storage of the image data and/or the transmission of the image data.

Obtaining image data of an object in a scene

A method of determining a group of user devices suitable for contributing user content to an interactive media broadcast may comprise: &#x2014;transmitting a set of requirements to a first plurality of user devices (), each user device having a set of capabilities, the set of requirements relating to using at least part of the user content in the interactive media broadcast, &#x2014;transmitting an invitation to a second plurality of user devices (), the invitation being configured to cause a user device to produce a conditional response if the set of capabilities of the user device matches the set of requirements, and &#x2014;identifying a group of user devices from which a conditional response was received. The method may further comprise selecting a group of user devices from the identified user devices. The selected group of user devices may include all identified user devices. The media broadcast may include an interactive television broadcast.

Identifying user devices for interactive media broadcast participation

A system and method are provided for managing a concurrent streaming of a first media stream () and a second media stream (). The media streams () represent different recordings of an event. The concurrent streaming of the media streams () is enabled by a resource having a resource limitation (). To address this resource limitation, when it is determined that the resource limitation is exceeded by the concurrent streaming of the first media stream () streaming to a first streaming client () and the second media stream () streaming to a second streaming client (), the first media stream () is selected for substituting the second media stream () in the streaming to the second streaming client (). As a result of the substitution, the first and the second streaming client () stream the same media stream rather than different media streams. This may have as advantage that the allocation of the resource is reduced, since the streaming of a same media stream to different streaming clients () typically represents a lower resource allocation than the streaming of different media streams to different streaming clients.

Managing concurrent streaming of media streams

A content distribution system is provided comprising a content client and a content server connected through a network. The content server is configured to send the content client a session description comprising at least one media description to enable it to receive over the network a content item. The system comprises a network monitor for monitoring network traffic going through the network, and a recommender for marking in dependency upon the network monitor, one or more of the media descriptions in the session description to obtain a marked session description. The marked media description recommends the content client to prefer certain content above others in view of the current network condition. For example, if available bandwidth is low, content is recommended which requires low bandwidth. Network congestion is thereby avoided for all users of the system.

Content distribution system

Systems and methods for screen sharing are provided. The screen sharing may comprise providing image data from a sender system to a receiver system. The receiver system may be a Virtual Reality [VR] system configured to render the image data in a virtual environment by projecting the image data onto a virtual canvas. At the sender system, visual content may be identified to share with the receiver system. The rendering or display of the virtual environment by the receiver may impose legibility constraints on the visual content. An adjusted version of the visual content may be generated which provides improved legibility when viewed in the virtual environment. The image data representing the adjusted version may then be provided to the receiver system. Accordingly, the legibility of the visual content shared by the sender system may be improved when rendered, displayed and subsequently viewed by a user in the virtual environment.

Screen sharing for display in VR

A system and method are provided for generating an output video, such as a video panorama, from a plurality of video streams representing different recordings of a scene. The plurality of video streams may be analyzed to identify at least one part of at least one of the plurality of video streams which is to be used in the output video, thereby identifying a contributing part of a video stream. Orchestration metadata may be generated identifying the contributing part. The orchestration metadata may be provided to a stream source from which the video stream originated to enable the stream source to selectively stream the contributing part of the video stream. Effectively, a selection of the stream's video data may be made to avoid or reduce unnecessary bandwidth usage.

Generating output video from video streams

Some embodiments involve obtaining access to a number of omnidirectional images each representing a different viewpoint relative to a scene, and to metadata indicating a physical relation between the respective viewpoint and a physical object in the scene. Viewer data may be obtained that is indicative of a current physical relation between a viewpoint of the user and a same type of physical object in a current viewing environment of the user. An omnidirectional image may then be selected for display by comparing the viewer data to the metadata of the omnidirectional images. In other embodiments, an omnidirectional image may be selected, e.g., from omnidirectional images or by being the only available image, and the user may be instructed to assume a viewpoint in which the physical relation to the physical object in the user's current viewing environment corresponds to the physical relation as indicated by the selected image's metadata.

Selecting an omnidirectional image for display

Systems and methods are provided for generating and rendering an enhanced audiovisual recording of a user, which may be used for multiuser communication, e.g., in Virtual Reality. Such an enhanced recording may be generated by determining a face orientation of the user in the audiovisual recording, and generating orientation data specifying an orientation which represents said determined face orientation. During rendering, the audio data may be rendered based on the orientation data, namely by rendering the audio data as a spatial audio source having a spatial direction which is congruent with the face orientation of the user in a visual representation of the user. Accordingly, the spatial direction of the voice of the user may better match the user's face direction in the user's visual representation.

Enhanced audiovisual multiuser communication

A system and method are provided for determining a size of a virtual object in a virtual environment. The virtual object may represent an object in physical space. The system and method may obtain image data of the object from a camera, obtain camera metadata indicative of an angle of view of the camera, estimate an object distance between the object and the camera, and estimate a physical size of the object in physical space by determining an image size of the object in the image data, and determining a relation between the image size and the physical size of the object on the basis of the camera metadata and the object distance. Accordingly, the size of the virtual object in the virtual environment may be determined in accordance with the physical size of the object. Virtual objects may thus be given realistic relative sizes in the virtual environment. A further advantage may be that there is no need for a calibrated system of multiple cameras and a calibrated rendering environment.

Determining size of virtual object

A method of facilitating capturing visual information at a first location () for display at a second location () is disclosed. At the first location, a virtual reality device () can be configured to render visual information representing a virtual environment, and at least one capturing device () can be configured to capture information representing a real environment comprising the virtual reality device (). At the second location, at least one monitor () can be configured to render the information captured by the at least one capturing device (). The method comprises determining an orientation of at least one capturing device () relative to the virtual reality device (), and providing in the virtual environment a visual indication of said orientation.

Capturing and rendering information involving a virtual environment

A system and method are provided for enhancing an audio recording which comprises a recording of a sound signal obtained from the play-out of an audio signal via a speaker. The audio signal, and thereby the sound signal, may represent certain audio content, e.g., a radio station or TV audio. To perform the enhancing, the recording of the sound signal is suppressed using the audio signal, thereby obtaining an intermediate audio recording. An original version of the audio content is then added to the intermediate audio recording to obtain an enhanced audio recording. This original version is generally of higher quality as it generally does not represent a background audio component but rather was purposefully recorded or generated.

Enhancing an audio recording

Methods and systems are described for processing spatially segmented content originating from a content delivery network. One method comprises a client in a media processing device receiving a spatial manifest information comprising one or more spatial representations of a source stream, each spatial representation identifying one or more spatial segment streams, location information for locating one or more delivery nodes in said content delivery network and, optionally, position information for stitching spatial segment frames in said segment streams into a video frame for display; selecting one or more spatial segment streams and on the basis of said spatial manifest information requesting at least one delivery node in said content delivery network to transmit said one or more selected spatial segment streams to said client; and, receiving said one or more selected spatial segment streams from said at least one delivery node.

Spatially segmented content delivery

A token-based validation method for delivery of at least part of a segmented content item and a content delivery system configured for executing such method are described. Said segmented content item may be associated with at least one manifest file comprising one or more segment identifiers. The method may comprises the steps of: a content processing device sending a first segment request message comprising a first segment identifier associated with a first segment to said at least one delivery node; generating first validation information for use with a further second segment request message, said first validation information comprising at least a first token and associated first timing information; and, sending a first response message and said first validation information to said content processing device, said first response message comprising at least part of said segment or location information associated with at least one delivery node for delivering said segment.

Growing community of inventors

The Hague, Netherlands

Martin Prins