Apple Inc.

Agere Systems Inc.

Lucent Technologies Inc.

Avago Technologies General IP Pte, Ltd.

Frank M Baumgarte

Eric A Allamanche

Dipanjan Sen

Christof Faller

Shyh-Shiaw Kuo

Stefan K O Strommer

Tomlinson M Holman

Jiashu Chen

Jonathan D Sheaffer

Moo Young Kim

Juha O Merimaa

Symeon Delikaris Manias

Peter Kroon

Gaetan R Lorho

Sina Zamani

Aram M Lindahl

Joshua D Atkins

William George Stewart

Peter A Raffensperger

Eric J Day

J Kevin Calhoun

Stefan KO Strommer

Christoff Faller

Petter Kroon

A method that includes receiving a bitstream that comprises: an encoded version of an audio signal that is associated with a sound source that is within a first 3D scene, a scene tree structure that includes an origin of the first scene relative to an origin of a second scene, and a position of the sound source within the first scene relative to the origin of the first scene, wherein the position references the origin of the first scene using an identifier, wherein the scene tree structure defines an initial configuration of the sound source with respect to the first and second scenes; determining a position of a listener; producing a set of spatially rendered audio signals by spatially rendering the audio signal according to the position of the sound source with respect to the position of the listener; and using the spatially rendered audio signals to drive speakers.

Method and system for efficiently encoding scene positions

A method that includes receiving a first bitstream that includes an encoded version of an audio signal for a three-dimensional (3D) scene and a first set of metadata that has 1) a position of a 3D sub-scene within the scene and 2) a position of a sound source associated with the audio signal within the sub-scene; determining a position of a listener; spatially rendering the scene to produce the sound source with the audio signal at the position of the sound source with respect to the position of the listener; receiving a second bitstream that includes a second set of metadata that has a different position of the sub-scene; and adjusting the spatial rendering of the scene such that the position of the sound source changes to correspond to movement of the sub-scene from the position of the sub-scene to the different position of the sub-scene.

Method and system for spatially rendering three-dimensional (3D) scenes

Disclosed is a hierarchical spatial resolution codec that adaptively adjusts the representations of immersive audio content as the target bandwidth for delivering the audio content changes. The audio content may be represented by an adaptive number of content types such as channels/objects, higher-order ambisonics (HOA), and encoded by adaptive spatial coding techniques to support the target bitrate of a transmission channel or user. Adaptive spatial coding techniques may include adaptive channel/object spatial encoding techniques to generate an adaptive number of channels/objects, and adaptive HOA spatial encoding or HOA compression techniques to generate an adaptive order of the HOA. The adaptation may be a function of the target bitrate that is associated with a desired quality, and an analysis that determines the priority of the channels, objects, and HOA. High priority channels/objects may be encoded into a high quality bit-stream while low priority channels/objects may be converted and encoded as HOA.

Hierarchical spatial resolution codec

Disclosed is an audio codec that represents an immersive signal by a two-channel stereo signal that is a stereo rendering of the immersive signal and directional parameters. The directional parameters may be based on a perceptual model describing the direction of virtual speaker pairs to recreate the perceived location of dominant sounds. Audio processing at the decoder may be performed on the stereo signal in the frequency domain for multiple channel pairs using time-frequency tiles. Spatial localization of the audio signals may use a panning approach by applying weightings to the time-frequency tiles of the stereo signal for each output channel pair. The weightings for the time-frequency tiles may be derived based on the directional parameters, an analysis of the stereo signal, and the output channel layout. The weightings may be used to adaptively process the time-frequency tiles using a de-correlator to reduce or minimize spectral distortions from spatial rendering.

Stereo-based immersive coding

Disclosed are methods and systems for decoding immersive audio content encoded by an adaptive number of scene elements for channels, audio objects, higher-order ambisonics (HOA), and/or other sound field representations. The decoded audio is rendered to the speaker configuration of a playback device. For bit streams that represent audio scenes with a different mixture of channels, objects, and/or HOA in consecutive frames, fade-in of the new frame and fade-out of the old frame may be performed. Crossfading between consecutive frames happen in the speaker layout after rendering, in the spatially decoded content type before rendering, or between the transport channels as the output of the baseline decoder but before spatial decoding and rendering. Crossfading may use an immediate fade-in and fade-out frame (IFFF) for the transition frame or may use an overlap-add synthesis technique such as time-domain aliasing cancellation (TDAC) of MDCT.

Seamless scalable decoding of channels, objects, and HOA audio content

A decoder-side method for outputting several audio channels of a sound program is described. An audio channel of the sound program, a residual signal, a gain parameter, and a delay parameter are received, for example within a bitstream. The audio channel is adjusted in accordance with the gain parameter and the delay parameter, to produce an adjusted audio signal, and is then combined with the residual signal to produce a combined signal. The audio channel is output as a first audio channel of the sound program for playback, while the combined signal is output as a second audio channel of the sound program. Other aspects are also described and claimed.

Channel-aligned audio coding

A bitstream is obtained by a decoder side, that contains an encoded version of an audio signal and an instantaneous loudness sequence of the audio signal. The instantaneous loudness sequence has not been loudness normalized. A dynamic range control, DRC, gain sequence is produced by applying the instantaneous loudness sequence to a DRC characteristic, with loudness normalization. The DRC gain sequence is applied to the decoded audio signal. Other aspects are also described and claimed.

Deferred loudness adjustment for dynamic range control

A device with microphones can generate microphone signals during an audio recording. The device can store, in an electronic audio data file, the microphone signals, and metadata that includes impulse responses of the microphones. Other aspects are described and claimed.

Spatial audio file format for storing capture metadata

An audio normalization gain value is applied to an audio signal to produce a normalized signal. The normalized signal is processed to compute dynamic range control (DRC) gain values in accordance with a selected one of several pre-defined DRC characteristics. The audio signal is encoded, and the DRC gain values are provided as metadata associated with the encoded audio signal. Several other embodiments are also described and claimed.

Metadata for loudness and dynamic range control

A system for producing an encoded digital audio recording has an audio encoder that encodes a digital audio recording having a number of audio channels or audio objects. An equalization (EQ) value generator produces a sequence of EQ values which define EQ filtering that is to be applied when decoding the encoded digital audio recording, wherein the EQ filtering is to be applied to a group of one or more of the audio channels or audio objects of the recording independent of any downmix. A bitstream multiplexer combines the encoded digital audio recording with the sequence of EQ values, the latter as metadata associated with the encoded digital audio recording. Other embodiments are also described including a system for decoding the encoded audio recording.

Encoded audio metadata-based equalization

A first layer of data having a first set of Ambisonic audio components can be decoded where the first set of Ambisonic audio components is generated based on ambience and one or more object-based audio signals. A second layer of data is decoded having at least one of the one or more object-based audio signals. One of the object-based audio signals is subtracted from the first set of Ambisonic audio components. The resulting Ambisonic audio components are rendered to generate a first set of audio channels. The one or more object-based audio signals are spatially rendered to generate a second set of audio channels. Other aspects are described and claimed.

Layered coding of audio with discrete objects

A system for encoding and applying Dynamic Range Control/Compression (DRC) gain values to a piece of sound program content is described. In particular, a set of DRC gain values representing a DRC gain curve for the piece of content may be divided into frames corresponding to frames of the piece of content. A set of fields may be included with an audio signal representing the piece of content. The additional fields may represent the DRC gain values using linear or spline interpolation. The additional fields may include 1) an initial gain value for each DRC frame, 2) a set of slope values at particular points in the DRC curve, 3) a set of time delta values for each consecutive pair of slope values, and/or 4) one or more gain delta values representing changes of DRC gain values in the DRC gain curve between points of the slope values.

Metadata driven dynamic range control

Transfer functions can describe responses of microphones or ears to sounds at different locations on a sphere. The transfer functions can be compressed by determining, based on transfer functions, a) one or more basis transfer functions, and b) spherical harmonics coefficients that describe variations of the transfer functions with respect to spherical coordinates. Other aspects are described and claimed.

Compressing spatial acoustic transfer functions

An audio encoding device and an audio decoding device are described herein. The audio encoding device may examine a set of audio channels/channel groups representing a piece of sound program content and produce a set of ducking values to associate with one of the channels/channel groups. During playback of the piece of sound program content, the ducking values may be applied to all other channels/channel groups. Application of these ducking values may cause (1) the reduction in dynamic range of ducked channels/channel groups and/or (2) movement of channels/channel groups in the sound field. This ducking may improve intelligibility of audio in the non-ducked channel/channel group. For instance, a narration channel/channel group may be more clearly heard by listeners through the use of selective ducking of other channels/channel groups during playback.

Growing community of inventors

Sunnyvale, CA, United States of America

Frank M Baumgarte