Fraunhofer-Gesellschaft Zur Foerderung Der Angewandten Forschung E.V.

Harald Fuchs

Adrian Murtaza

Oliver Hellmuth

Leon Terentiv

Jouni Paulus

Sascha Disch

Bernd Czelhan

Falko Ridderbusch

Stefan Doehla

Juergen Herre

Ingo Hofmann

Stephan Schreiner

Nikolaus Faerber

Bernhard Grill

Nikolaus Rettelbach

Max Neuendorf

Daniel Fischer

Stefan Kraegeloh

Jurgen Herre

Herbert Thoma

Sebastian Scharrer

Josef Bernhard

Jürgen Herre

Jan Plogsties

Simone Neukam

Arne Borsum

Michael Kratz

Marc Gayer

Roberta Camilleri

Manfred Lutzky

Harald Popp

Thomas Sporer

Bert Greevenbosch

Christian Uhle

Stefan Döhla

Sandra Brix

Matteo Agnelli

Mercè Serra Joan

Thorsten Kastner

Andreas Walther

Stefan Krägeloh

Ulf Jennehag

Anja Becker

Mercé Serra Joan

Simone FÜG

Nikolaus Färber

Matteo Torcoli

Merce Serra Joan

Simone Fug

Alapan Arnab

Thomas Mayenfels

Christian Simon

Andreas Turnwald

There are disclosed techniques, systems, methods and instructions for a virtual reality, VR, augmented reality, AR, mixed reality, MR, or 360-degree video environment. In one example, the system includes at least one media video decoder configured to decode video signals from video streams for the representation of VR, AR, MR or 360-degree video environment scenes to a user. The system includes at least one audio decoder configured to decode audio signals from at least one audio stream. The system is configured to request at least one audio stream and/or one audio element of an audio stream and/or one adaptation set to a server on the basis of at least the user's current viewport and/or head orientation and/or movement data and/or interaction metadata and/or virtual positional data.

Optimizing audio delivery for virtual reality applications

Audio data processor, having: a receiver interface for receiving encoded audio data and metadata related to the encoded audio data; a metadata parser for parsing the metadata to determine an audio data manipulation possibility; an interaction interface for receiving an interaction input and for generating, from the interaction input, interaction control data related to the audio data manipulation possibility; and a data stream generator for obtaining the interaction control data and the encoded audio data and the metadata and for generating an output data stream, the output data stream having the encoded audio data, at least a portion of the metadata, and the interaction control data.

Data processor and transport of user control data to audio decoders and renderers

A multi-channel audio decoder for providing at least two output audio signals on the basis of an encoded representation is configured to render a plurality of decoded audio signals, which are obtained on the basis of the encoded representation, in dependence on one or more rendering parameters, to obtain a plurality of rendered audio signals. The multi-channel audio decoder is configured to derive one or more decorrelated audio signals from the rendered audio signals, and to combine the rendered audio signals, or a scaled version thereof, with the one or more decorrelated audio signals, to obtain the output audio signals. A multi-channel audio encoder provides a decorrelation method parameter to control an audio decoder.

Multi-channel audio decoder, multi-channel audio encoder, methods, computer program and encoded audio representation using a decorrelation of rendered audio signals

A sound reproduction/simulation system includes at least sound reproduction device driven by one or more audio signals and a processor for processing an input audio stream to generate the one or more audio signals. The processor performs processing based on processing parameters defined by a sound characteristic of a target system.

Sound reproduction/simulation system and method for simulating a sound reproduction

A method and a system for virtual reality, augmented reality, mixed reality, or 360-degree Video environment is disclosed. The system receives Video Streams associated to audio and video scenes to be reproduced and Audio Streams associated to audio and video scenes to be reproduced. There are provided a Video decoder which decodes signal from the Video Stream for the representation of the audio and video scene; an Audio decoder which decodes signal from the Audio Stream for the representation of the audio and video scene to the user; and a region of interest processor deciding, based e.g. on the user's viewport, head orientation, movement data, or metadata, whether an Audio information message is to be reproduced. At the decision, the reproduction of the Audio information message is caused.

Method and apparatus for efficient delivery and usage of audio messages for high quality of experience

An audio decoder decodes a bit stream of encoded audio data, which bit stream represents a sequence of audio sample values and includes a plurality of frames, wherein each frame includes associated encoded audio sample values. The audio decoder includes a determiner configured to determine whether a frame of the encoded audio data is a special frame including encoded audio sample values associated with the special frame and additional information, wherein the additional information include encoded audio sample values of a number of frames preceding the special frame, wherein the encoded audio sample values of the preceding frames are encoded using the same codec configuration as the special frame, wherein the number of preceding frames is sufficient to initialize the decoder to be in a position to decode the audio sample values associated with the special frame if the special frame is the first frame upon start-up of the decoder.

Audio decoder, apparatus for generating encoded audio output data and methods permitting initializing a decoder

Audio video synchronization and alignment or alignment of audio to some other external clock are rendered more effective or easier by treating fragment grid and frame grid as independent values, but, nevertheless, for each fragment the frame grid is aligned to the respective fragment's beginning. A compression effectiveness lost may be kept low when appropriately selecting the fragment size. On the other hand, the alignment of the frame grid with respect to the fragments' beginnings allows for an easy and fragment-synchronized way of handling the fragments in connection with, for example, parallel audio video streaming, bitrate adaptive streaming or the like.

Fragment-aligned audio coding

An apparatus for downmixing three or more audio input channels to obtain two or more audio output channels is provided. The apparatus includes a receiving interface for receiving the three or more audio input channels and for receiving side information. Moreover, the apparatus includes a downmixer for downmixing the three or more audio input channels depending on the side information to obtain the two or more audio output channels. The number of the audio output channels is smaller than the number of the audio input channels. The side information indicates a characteristic of at least one of the three or more audio input channels, or a characteristic of one or more sound waves recorded within the one or more audio input channels, or a characteristic of one or more sound sources which emitted one or more sound waves recorded within the one or more audio input channels.

Apparatus and method for providing enhanced guided downmix capabilities for 3D audio

A decoder for generating an audio output signal having one or more audio output channels is provided, having a receiving interface for receiving an audio input signal having a plurality of audio object signals, for receiving loudness information on the audio object signals, and for receiving rendering information indicating whether one or more of the audio object signals shall be amplified or attenuated, further having a signal processor for generating the one or more audio output channels of the audio output signal, configured to determine a loudness compensation value depending on the loudness information and depending on the rendering information, and configured to generate the one or more audio output channels of the audio output signal from the audio input signal depending on the rendering information and depending on the loudness compensation value. One or more by-pass audio object signals are employed for generating the audio output signal. Moreover, an encoder is provided.

Decoder, encoder and method for informed loudness estimation employing by-pass audio object signals in object-based audio coding systems

Growing community of inventors

Roettenbach, Germany

Harald Fuchs