Dolby Laboratories Licensing Corporation

Dolby International Ab

Lianwu Chen

Lie Lu

Dirk Jeroen Breebaart

Nicolas R Tsingos

Joshua Brandon Lando

Mingqing Hu

David Matthew Fischer

Ziyu Yang

Chunmao Zhang

Alan Jeffrey Seefeldt

Ming Wang

Brett Graham Crockett

Xuejing Sun

Antonio Mateos Sole

Rhonda J Wilson

Example embodiments disclosed herein relate to audio object clustering. A method for metadata-preserved audio object clustering is disclosed. The method comprises classifying an audio object into at least a category based rendering mode information metadata. The method further comprises assigning a predetermined number of clusters to the categories and rendering the audio object based on the rendering mode. Corresponding system and computer program product are also disclosed.

Metadata-preserved audio object clustering

A method of processing audio content including a plurality of audio elements comprises: clustering the plurality of audio elements into a plurality of clusters of audio elements; and for a cluster among the plurality of clusters: for each audio element in the cluster, determining a measure of energy that the audio element contributes to the cluster; for at least one audio element in the cluster, determining a compensation gain based at least in part on the measures of energy for the audio elements in the cluster; and applying the compensation gain to the at least one audio element in the cluster.

Adaptive loudness normalization for audio object clustering

An apparatus and method of blind detection of binauralized audio. If the input content is detected as binaural, a second binauralization may be avoided. In this manner, the user experience avoids audio artifacts introduced by multiple binauralizations.

Blind detection of binauralized stereo content

Example embodiments disclosed herein relate to audio object clustering. A method for metadata-preserved audio object clustering is disclosed. The method comprises classifying a plurality of audio objects into a number of categories based on information to be preserved in metadata associated with the plurality of audio objects. The method further comprises assigning a predetermined number of clusters to the categories and allocating an audio object in each of the categories to at least one of the clusters according to the assigning. Corresponding system and computer program product are also disclosed.

Example embodiments disclosed herein relate to audio object clustering based on renderer-aware perceptual difference. A method of processing audio objects is provided. The method includes obtaining renderer-related information indicating a configuration of a renderer. The method also includes determining, based on the obtained renderer-related information, a rendering difference between a first audio object and a second audio object among the audio objects with respect to the renderer. The method further includes clustering the audio objects at least in part based on the rendering difference. Corresponding system, device, and computer program product are also disclosed.

Audio object clustering based on renderer-aware perceptual difference

Embodiments of the example embodiment relate to audio object extraction. A method for audio object extraction from audio content is disclosed. The method comprises determining a sub-band object probability for a sub-band of the audio signal in a frame of the audio content, the sub-band object probability indicating a probability of the sub-band of the audio signal containing an audio object. The method further comprises splitting the sub-band of the audio signal into an audio object portion and a residual audio portion based on the determined sub-band object probability. Corresponding system and computer program product are also disclosed.

Audio object extraction with sub-band object probability estimation

Audio objects that are present in input audio content in one or more frames are determined. Output clusters that are present in output audio content in the one or more frames are also determined. Here, the audio objects in the input audio content are converted to the output clusters in the output audio content. One or more spatial error metrics are computed based at least in part on positional metadata of the audio objects and positional metadata of the output clusters.

Spatial error metrics of audio content

Example embodiments disclosed herein relates to upmixing of audio signals. A method of upmixing an audio signal is described. The method includes decomposing the audio signal into a diffuse signal and a direct signal, generating an audio bed at least in part based on the diffuse signal, the audio bed including a height channel, extracting an audio object from the direct signal, estimating metadata of the audio object, the metadata including height information of the audio object; and rendering the audio bed and the audio object as an upmixed audio signal, wherein the audio bed is rendered to a predefined position and the audio object is rendered according to the metadata. Corresponding system and computer program product are described as well.

Upmixing of audio signals

Example embodiments disclosed herein relate to audio object clustering with single channel quality preservation. A method of clustering audio objects is disclosed. The method includes determining cluster positions based on object positions of the audio objects and a reference speaker layout, the reference speaker layout indicating speakers located at different speaker positions. The method also includes determining object-to-cluster gains based on the determined cluster positions, the object positions and the reference speaker layout, an object-to-cluster gain defining a proportion of the respective audio object that is assigned to a cluster associated with one of the determined cluster positions. The method further includes clustering the audio objects based on the object-to-cluster gains and the cluster positions for generating cluster signals. Corresponding system, computer program product and device for clustering audio objects are also disclosed.

Audio object clustering with single channel quality preservation

Example embodiments disclosed herein relate to audio signal processing. The audio signal has multiple audio objects. A method of processing an audio signal is disclosed. The method includes obtaining an object position for each of the audio objects; and determining cluster positions for grouping the audio objects into clusters based on the object positions, a plurality of object-to-cluster gains, and a set of metrics. The metrics indicate a quality of the cluster positions and a quality of the object-to-cluster gains, each of the cluster positions is a centroid of a respective one of the clusters, and one of the object-to-cluster gains defines a ratio of the respective audio object in one of the clusters. The method also includes determining the object-to-cluster gains based on the object positions, the cluster positions and the set of metrics; and generating a cluster signal based on the determined cluster positions and object-to-cluster gains. Corresponding system and computer program product are also disclosed.

Processing object-based audio signals

A method is disclosed for audio object extraction from an audio content which includes identifying a first set of projection spaces including a first subset for a first channel and a second subset for a second channel of the plurality of channels. The method may further include determining a first set of correlations between the first and second channels, each of the first set of correlations corresponding to one of the first subset of projection spaces and one of the second subset of projection spaces. Still further, the method may include extracting an audio object from an audio signal of the first channel at least in part based on a first correlation among the first set of correlations and the projection space from the first subset corresponding to the first correlation, the first correlation being greater than a first predefined threshold. Corresponding system and computer program products are also disclosed.

Projection-based audio object extraction from audio content

Embodiments of the present invention relate to video content assisted audio object extraction. A method of audio object extraction from channel-based audio content is disclosed. The method comprises extracting at least one video object from video content associated with the channel-based audio content, and determining information about the at least one video object. The method further comprises extracting from the channel-based audio content an audio object to be rendered as an upmixed audio signal based on the determined information. Corresponding system and computer program product are also disclosed.

Video content assisted audio object extraction

Embodiments of the present invention relate to audio object clustering by utilizing temporal variation of audio objects. There is provided a method of estimating temporal variation of an audio object for use in audio object clustering. The method comprises obtaining at least one segment of an audio track associated with the audio object, the at least one segment containing the audio object; estimating variation of the audio object over a time duration of the at least one segment based on at least one property of the audio object and adjusting, at least partially based on the estimated variation of the audio object, a contribution of the audio object to the determination of a centroid in the audio object clustering. Corresponding system and computer program product are disclosed.

Growing community of inventors

Beijing, China

Lianwu Chen