Qualcomm Incorporated

Apple Inc.

Dipanjan Sen

Nils Günther Peters

Moo Young Kim

Martin James Morrell

S M Akramus Salehin

Shankar Thagadur Shivappa

Pei Xiang

Siddhartha Goutham Swaminathan

Sang-Uk Ryu

Thomas Stockhammer

Frank M Baumgarte

Ferdinando Olivieri

Ricardo De Jesus Bernal Castillo

Richard Paul Walters

Jeongook Song

Juha O Merimaa

Eric A Allamanche

Isaac Garcia Munoz

Nils Gunther Peters

Wenliang Lu

Michael Ericson

Jonathan D Sheaffer

Joshua D Atkins

Symeon Delikaris Manias

Wade Lyle Heimbigner

Eric Stephen Mattis

Peter A Raffensperger

Jason Filos

Mark Adam Cherry

Gaetan R Lorho

Eric J Day

Daniel Weber

Sina Zamani

J Kevin Calhoun

Martin Morrell

Kerry Titus Hartman

Disclosed are methods and systems for decoding immersive audio content encoded by an adaptive number of scene elements for channels, audio objects, higher-order ambisonics (HOA), and/or other sound field representations. The decoded audio is rendered to the speaker configuration of a playback device. For bit streams that represent audio scenes with a different mixture of channels, objects, and/or HOA in consecutive frames, fade-in of the new frame and fade-out of the old frame may be performed. Crossfading between consecutive frames happen in the speaker layout after rendering, in the spatially decoded content type before rendering, or between the transport channels as the output of the baseline decoder but before spatial decoding and rendering. Crossfading may use an immediate fade-in and fade-out frame (IFFF) for the transition frame or may use an overlap-add synthesis technique such as time-domain aliasing cancellation (TDAC) of MDCT.

Seamless scalable decoding of channels, objects, and HOA audio content

An example device includes a memory configured to store a plurality of representations of a soundfield, each representation of the soundfield comprising a different set of ambisonic coefficients representative of the same soundfield at concurrent periods of time. The device also includes a processor, coupled to the memory, and the processor is configured to perform audio playback based on a field of view and on a particular representation of the soundfield from the plurality of representations.

Mixed-order ambisonics (MOA) audio data for computer-mediated reality systems

An example audio decoding device includes a memory configured to store at least a portion of a coded audio bitstream; and one or more processors configured to: decode, based on the coded audio bitstream, a representation of a soundfield; decode, based on the coded audio bitstream, a syntax element indicating a selection of either a head-related transfer function (HRTF) or a binaural room impulse response (BRIR); and render, using the selected HRTF or BRIR, speaker feeds from the soundfield.

Signaling for rendering tools

In general, disclosed is a device that includes one or more processors, coupled to the memory, configured to perform an energy analysis with respect to one or more audio objects, in the ambisonics domain, in the first time segment. The one or more processors are also configured to perform a similarity measure between the one or more audio objects, in the ambisonics domain, in the first time segment, and the one or more audio objects, in the ambisonics domain, in the second time segment. In addition, the one or more processors are configured to perform a reorder of the one or more audio objects, in the ambisonics domain, in the first time segment with the one or more audio objects, in the ambisonics domain, in the second time segment, to generate one or more reordered audio objects in the first time segment.

Reordering of foreground audio objects in the ambisonics domain

A device and method for backward compatibility for virtual reality (VR), mixed reality (MR), augmented reality (AR), computer vision, and graphics systems. The device and method enable rendering audio data with more degrees of freedom on devices that support fewer degrees of freedom. The device includes memory configured to store audio data representative of a soundfield captured at a plurality of capture locations, metadata that enables the audio data to be rendered to support N degrees of freedom, and adaptation metadata that enables the audio data to be rendered to support M degrees of freedom. The device also includes one or more processors coupled to the memory, and configured to adapt, based on the adaptation metadata, the audio data to provide the M degrees of freedom, and generate speaker feeds based on the adapted audio data.

Six degrees of freedom and three degrees of freedom backward compatibility

A device with microphones can generate microphone signals during an audio recording. The device can store, in an electronic audio data file, the microphone signals, and metadata that includes impulse responses of the microphones. Other aspects are described and claimed.

Spatial audio file format for storing capture metadata

A device configured to decode a bitstream, where the device includes a memory configured to store a temporally encoded representation of spatial audio signals. The device is also configured to receive the bitstream that includes an indication of a spatial transformation, and includes a temporal decoding unit, coupled to the memory, configured to decode one or more spatial audio signals represented in a spatial domain, where the one or more spatial audio signals are associated with different angles in the spatial domain. In addition, the device includes an inverse spatial transformation unit, coupled to the temporal decoding unit, is configured to convert the one or more spatial audio signals represented in the spatial domain into at least three ambisonic coefficients that, in part, represent a soundfield in an ambisonics domain, and perform a spatial transformation of the soundfield based on the indication of the spatial transformation received in the bitstream.

Spatial transformation of ambisonic audio data

A first layer of data having a first set of Ambisonic audio components can be decoded where the first set of Ambisonic audio components is generated based on ambience and one or more object-based audio signals. A second layer of data is decoded having at least one of the one or more object-based audio signals. One of the object-based audio signals is subtracted from the first set of Ambisonic audio components. The resulting Ambisonic audio components are rendered to generate a first set of audio channels. The one or more object-based audio signals are spatially rendered to generate a second set of audio channels. Other aspects are described and claimed.

Layered coding of audio with discrete objects

In general, techniques are described by which to provide priority information for higher order ambisonic (HOA) audio data. A device comprising a memory and a processor may perform the techniques. The memory stores HOA coefficients of the HOA audio data, the HOA coefficients representative of a soundfield. The processor may decompose the HOA coefficients into a sound component and a corresponding spatial component, the corresponding spatial component defining shape, width, and directions of the sound component, and the corresponding spatial component defined in a spherical harmonic domain. The processor may also determine, based on one or more of the sound component and the corresponding spatial component, priority information indicative of a priority of the sound component relative to other sound components of the soundfield, and specify, in a data object representative of a compressed version of the HOA audio data, the sound component and the priority information.

Higher order ambisonic audio data

One or more processors may obtain a first distance between a first audio zone of the two or more audio zones associated with the one or more interest points within the first audio zone, and a first device position of a device, obtain a second distance between a second audio zone of the two or more audio zones associated with the one or more interest points within the second audio zone, and the first device position of the device, and obtain an updated first distance and updated second distance after movement of the device has changed from the first device position to a second device position. The one or more processor(s) may independently control the first audio zone and the second audio zone, such that the audio data within the first audio zone and the second audio zone are adjusted based on the updated first distance and updated second distance.

Rendering audio data from independently controlled audio zones

In general, techniques are described for rendering metadata to control user movement based audio rendering. A device comprising a memory and one or more processors may be configured to perform the techniques. The memory may be configured to store audio data representative of a soundfield. The one or more processors may be coupled to the memory, and configured to obtain rendering metadata indicative of controls for enabling or disabling adaptations, based on an indication of a movement of a user of the device, of a renderer used to render audio data representative of a soundfield, specify, in a bitstream representative of the audio data, the rendering metadata, and output the bitstream.

Rendering metadata to control user movement based audio rendering

An example device includes a memory device, and a processor coupled to the memory device. The memory is configured to store audio spatial metadata associated with a soundfield and video data. The processor is configured to identify one or more foreground audio objects of the soundfield using the audio spatial metadata stored to the memory device, and to select, based on the identified one or more foreground audio objects, one or more viewports associated with the video data. Display hardware coupled to the processor and the memory device is configured to output a portion of the video data being associated with the one or more viewports selected by the processor.

Audio-driven viewport selection

In general, techniques are described for compressing decomposed representations of a sound field. A device comprising one or more processors may be configured to perform the techniques. The one or more processors may be configured to obtain a bitstream comprising a compressed version of a spatial component of a sound field, the spatial component generated by performing a vector based synthesis with respect to a plurality of spherical harmonic coefficients.

Compression of decomposed representations of a sound field

In general, techniques are described for signaling layers for scalable coding of higher order ambisonic audio data. A device comprising a memory and a processor may be configured to perform the techniques. The memory may be configured to store the bitstream. The processor may be configured to obtain, from the bitstream, an indication of a number of layers specified in the bitstream, and obtain the layers of the bitstream based on the indication of the number of layers.

Signaling layers for scalable coding of higher order ambisonic audio data

In general, techniques are described for modeling occlusions when rendering audio data. A device comprising a memory and one or more processors may perform the techniques. The memory may store audio data representative of a soundfield. The one or more processors may obtain occlusion metadata representative of an occlusion within the soundfield in terms of propagation of sound through the occlusion, the occlusion separating the soundfield into two or more sound spaces. The one or more processors may obtain a location of the device, and obtain, based on the occlusion metadata and the location, a renderer by which to render the audio data into one or more speaker feeds that account for propagation of the sound in one of the two or more sound spaces in which the device resides. The one or more processors may apply the renderer to the audio data to generate the speaker feeds.

Growing community of inventors

San Diego, CA, United States of America

Dipanjan Sen