Dolby International Ab

Dolby Laboratories Licensing Corporation

Giulio Cengarle

Antonio Mateos Sole

Daniel Arteaga

Ming Wang

Juan Felix Torres

Davide Scaini

Brett Graham Crockett

Cristina Michel Vasco

Philip Nicol

Alan Jeffrey Seefeldt

Dirk Jeroen Breebaart

David Matthew Fischer

Nicolas R Tsingos

Mark Franklin Davis

C Phillip Brown

Louis Dunn Fielder

Rhonda J Wilson

David Matthew Cooper

Sunil Bharitkar

Mark David De Burgh

Yuanxing Ma

Suraj Suhas Barkale

Dirk Jeroen Breebart

Antonio Mateos Solé

Nicholas Laurence Engel

Patrick Winfrey Scannell

Natanael David Olaiz

Chunghsin Yeh

Kenneth Robert Hunold

Antonio Mateos SolÉ

Disclosed herein are method, systems, and computer-program products for segmenting a binaural recording of speech into parts containing self-speech and parts containing external speech, and processing each category with different settings, to obtain an enhanced overall presentation. The segmentation is based on a combination of: i) feature-based frame-by-frame classification, and ii) detecting dissimilarity by statistical methods. The segmentation information is then used by a speech enhancement chain, where independent settings are used to process the self- and external speech parts.

Detection and enhancement of speech in binaural recordings

Embodiments are disclosed for automatic leveling of speech content. In an embodiment, a method comprises: receiving, using one or more processors, frames of an audio recording including speech and non-speech content; for each frame: determining, using the one or more processors, a speech probability; analyzing, using the one or more processors, a perceptual loudness of the frame; obtaining, using the one or more processors, a target loudness range for the frame; computing, using the one or more processors, gains to apply to the frame based on the target loudness range and the perceptual loudness analysis, where the gains include dynamic gains that change frame-by-frame and that are scaled based on the speech probability; and applying the gains to the frame so that a resulting loudness range of the speech content in the audio recording fits within the target loudness range.

Automatic leveling of speech content

A method of audio processing includes performing spatial analysis on a binaural signal to estimate level differences and phase differences characteristic of a binaural filter of the binaural signal, performing object extraction on the binaural audio signal using the estimated level and phase differences to generate a left/right main component signal and a left/right residual component signal. The system may process the left/right main and left/right residual components differently using different object processing parameters for e.g. repositioning, equalization, compression, upmixing, channel remapping or storage to generate a processed binaural signal that provides an improved listening experience. Repositioning may be based on head tracking sensor data.

Binaural signal post-processing

In an embodiment, a method comprises: filtering reference audio content items to separate the reference audio content items into different frequency bands; for each frequency band, extracting a first feature vector from at least a portion of each of the reference audio content items, wherein the first feature vector includes at least one audio characteristic of the reference audio content items; obtaining at least one semantic label from at least a portion of each of the reference audio content items; obtaining a second feature vector consisting of the first feature vectors per frequency band and the at least one semantic label; generating, based on the second feature vector, cluster feature vectors representing centroids of clusters; separating the reference audio content items according to the cluster feature vectors; and computing an average target profile for each cluster based on the reference audio content items in the cluster.

Automatic generation and selection of target profiles for dynamic equalization of audio content

A method for generating mastered audio content, the method comprising obtaining an input audio content comprising a number, M, of audio signals, obtaining rendered presentation of the input audio content, the rendered presentation comprising a number, M, of audio signals, obtaining a mastered presentation generated by mastering the rendered presentation, comparing the mastered presentation with the rendered presentation to determine one or more indications of differences between the mastered presentation and the rendered presentation, modifying one or more of the audio signals of the input audio content based on the indications of differences to generate the mastered audio content. With this approach, conventional, typically stereo, channel-based mastering tools can be used to provide a mastered version of any input audio content, including object-based immersive audio content.

Presentation independent mastering of audio content

Methods, systems, and computer program products of automatic de-essing are disclosed. An automatic de-esser can be used without manually setting parameters and can perform reliable sibilance detection and reduction regardless of absolute signal level, singer gender and other extraneous factors. An audio processing device divides input audio signals into buffers each containing a number of samples, the buffers overlapping one another. The audio processing device transforms each buffer from the time domain into the frequency domain and implements de-essing as a multi-band compressor that only acts on a designated sibilance band. The audio processing device determines an amount of attenuation in the sibilance band based on comparison of energy level in sibilance band of a buffer to broadband energy level in a previous buffer. The amount of attenuation is also determined based on a zero-crossing rate, as well as a slope and onset of a compression curve.

Audio de-esser independent of absolute signal level

Embodiments are disclosed for noise floor estimation and noise reduction, In an embodiment, a method comprises: obtaining an audio signal; dividing the audio signal into a plurality of buffers; determining time-frequency samples for each buffer of the audio signal; for each buffer and for each frequency, determining a median (or mean) and a measure of an amount of variation of energy based on the samples in the buffer and samples in neighboring buffers that together span a specified time range of the audio signal; combining the median (or mean) and the measure of the amount of variation of energy into a cost function; for each frequency: determining a signal energy of a particular buffer of the audio signal that corresponds to a minimum value of the cost function; selecting the signal energy as the estimated noise floor of the audio signal; and reducing, using the estimated noise floor, noise in the audio signal.

Noise floor estimation and noise reduction

An audio object including audio content and object metadata is received. The object metadata indicates an object spatial position of the audio object to be rendered by audio speakers in a playback environment. Based on the object spatial position and source spatial positions of the audio speakers, initial gain values for the audio speakers are determined. The initial gain values can be used to select a set of audio speakers from among the audio speakers. Based on the object spatial position and a set of source spatial positions at which the set of audio speakers are respectively located in the playback environment, a set of non-negative optimized gain values for the set of audio speakers is determined. The audio object at the object spatial position is rendered with the set of optimized gain values for the set of audio speakers.

Adaptive panner of audio objects

Methods, systems, and computer program products for network-based processing and distribution of multimedia content of a live performance are disclosed. In some implementations, recording devices can be configured to record a multimedia event (e.g., a musical performance). The recording devices can provide the recordings to a server while the event is ongoing. The server automatically synchronizes, mixes and masters the recordings. The server performs the automatic mixing and mastering using reference audio data previously captured during a rehearsal. The server streams the mastered recording to multiple end users through the Internet or other public or private network. The streaming can be live streaming.

Network-based processing and distribution of multimedia content of a live musical performance

Methods, systems, and computer program products that provide streaming capabilities to audio input and output devices are disclosed. An audio processing device connects an upstream device to a downstream device. The upstream device is plugged into an input port of the audio processing device. The audio processing device intercepts a signal from the upstream device to the downstream device. The audio processing device converts the signal to digital data and streams the digital data to a server. The digital data can include metadata, e.g., an input gain. The audio processing device can adjust the input gain in response to instructions from the server. The audio processing device feeds a pass-through copy of the audio signal to an output port. A user can connect the downstream device in a usual signal chain into the output port of the audio processing device. The streaming does not affect the user's workflow.

Audio input and output device with streaming capabilities

Methods, systems, and computer program products for rending an audio object having an apparent size are disclosed. An audio processing system receives audio panning data including a first grid mapping first virtual sound sources in a space and speaker positions to speaker gains. The first grid specifies first speaker gains of the first virtual sound sources in the space. The audio processing system determines a second grid of second virtual sound sources in the space, including mapping the first virtual sound sources into the second virtual sound sources of the second virtual sources. The audio processing system selects at least one of the first grid or second grid for rendering an audio object based on an apparent size of the audio object. The audio processing system renders the audio object based on the selected grid or grids.

Rendering audio objects having apparent size

Methods, systems, and computer program products for synchronizing audio signals captured by multiple independent devices during an audio event are described. Multiple recording devices, e.g. several smartphones, record the audio event. A computer system receives audio signals from the devices. The system determines a first delay between two audio signals based on cross-correlation of waveforms of the two audio signals. Subsequently, the system detects attacks that are present in each audio signal by computing the derivative of a respective envelope for each audio signal. The system determines a second delay between the two audio signals based on cross-correlation of attacks of the two audio signals. The system synchronizes the audio signals using the second delay upon determining that using the second delay improves sound quality over using the first delay.

Hybrid audio signal synchronization based on cross-correlation and attack analysis

Various embodiments are disclosed for (possibly simultaneously) applying EQ and DRC to audio signals. In an embodiment, a method comprises: dividing an input audio signal into n frames, where n is a positive integer greater than one; dividing each frame of the input audio signal into Nb frequency bands, where Nb is a positive integer greater than one; for each frame n: computing an input level of the input audio signal in each band f, resulting in a input audio level distribution for the input audio signal; computing a gain for each band f based at least in part on a mapping of one or more properties of the input audio level distribution to a reference N audio level distribution computed from one or more reference audio signals; and applying each computed gain for each band f to each corresponding band f of the input audio signal.

Dynamic EQ

Embodiments are described for a method of simultaneously localizing a set of speakers and microphones, having only the times of arrival between each of the speakers and microphones. An autodiscovery process uses an external input to set: a global translation (3 continuous parameters), a global rotation (3 continuous parameters), and discrete symmetries, i.e., an exchange of any axis pairs and/or reversal of any axis. Different time of arrival acquisition techniques may be used, such as ultrasonic sweeps or generic multitrack audio content. The autodiscovery algorithm is based in minimizing a certain cost function, and the process allows for latencies in the recordings, possibly linked to the latencies in the emission.

Growing community of inventors

Barcelona, Spain

Giulio Cengarle