Fraunhofer-gesellschaft Zur Foerderung Der Angewandten Forschung E.v.

Bernd Czelhan

Harald Fuchs

Ingo Hofmann

Nikolaus Rettelbach

Max Neuendorf

Stefan Doehla

Daniel Fischer

Nikolaus Faerber

Jan Plogsties

Herbert Thoma

Stephan Schreiner

Adrian Murtaza

Matteo Agnelli

Stefan Döhla

Nikolaus Farber

Nikolaus Färber

There are disclosed techniques, systems, methods and instructions for a virtual reality, VR, augmented reality, AR, mixed reality, MR, or 360-degree video environment. In one example, the system includes at least one media video decoder configured to decode video signals from video streams for the representation of VR, AR, MR or 360-degree video environment scenes to a user. The system includes at least one audio decoder configured to decode audio signals from at least one audio stream. The system is configured to request at least one audio stream and/or one audio element of an audio stream and/or one adaptation set to a server on the basis of at least the user's current viewport and/or head orientation and/or movement data and/or interaction metadata and/or virtual positional data.

Optimizing audio delivery for virtual reality applications

A method and a system for virtual reality, augmented reality, mixed reality, or 360-degree Video environment is disclosed. The system receives Video Streams associated to audio and video scenes to be reproduced and Audio Streams associated to audio and video scenes to be reproduced. There are provided a Video decoder which decodes signal from the Video Stream for the representation of the audio and video scene; an Audio decoder which decodes signal from the Audio Stream for the representation of the audio and video scene to the user; and a region of interest processor deciding, based e.g. on the user's viewport, head orientation, movement data, or metadata, whether an Audio information message is to be reproduced. At the decision, the reproduction of the Audio information message is caused.

Method and apparatus for efficient delivery and usage of audio messages for high quality of experience

An audio decoder decodes a bit stream of encoded audio data, which bit stream represents a sequence of audio sample values and includes a plurality of frames, wherein each frame includes associated encoded audio sample values. The audio decoder includes a determiner configured to determine whether a frame of the encoded audio data is a special frame including encoded audio sample values associated with the special frame and additional information, wherein the additional information include encoded audio sample values of a number of frames preceding the special frame, wherein the encoded audio sample values of the preceding frames are encoded using the same codec configuration as the special frame, wherein the number of preceding frames is sufficient to initialize the decoder to be in a position to decode the audio sample values associated with the special frame if the special frame is the first frame upon start-up of the decoder.

Audio decoder, apparatus for generating encoded audio output data and methods permitting initializing a decoder

Audio video synchronization and alignment or alignment of audio to some other external clock are rendered more effective or easier by treating fragment grid and frame grid as independent values, but, nevertheless, for each fragment the frame grid is aligned to the respective fragment's beginning. A compression effectiveness lost may be kept low when appropriately selecting the fragment size. On the other hand, the alignment of the frame grid with respect to the fragments' beginnings allows for an easy and fragment-synchronized way of handling the fragments in connection with, for example, parallel audio video streaming, bitrate adaptive streaming or the like.

Fragment-aligned audio coding

Growing community of inventors

Happurg, Germany

Bernd Czelhan