Northwestern University

Adobe, Inc.

Bryan A Pardo

Andrew Todd Sabin

Gautham J Mysore

Nicholas J Bryan

Zeyu Jin

Maxwell Morrison

Juan Pablo Caceres Chomali

Prem Seetharaman

Mark B Cartwright

Zafar Rafii

Methods for modifying audio data include operations for accessing audio data having a first prosody, receiving a target prosody differing from the first prosody, and computing acoustic features representing samples. Computing respective acoustic features for a sample includes computing a pitch feature as a quantized pitch value of the sample by assigning a pitch value, of the target prosody or the audio data, to at least one of a set of pitch bins having equal widths in cents. Computing the respective acoustic features further includes computing a periodicity feature from the audio data. The respective acoustic features for the sample include the pitch feature, the periodicity feature, and other acoustic features. A neural vocoder is applied to the acoustic features to pitch-shift and time-stretch the audio data from the first prosody toward the target prosody.

Neural pitch-shifting and time-stretching

Embodiments of the present invention provide systems, methods, and computer storage media for sound quality prediction and real-time feedback about sound quality, such as room acoustics quality and background noise. Audio data can be sampled from a live sound source and stored in an audio buffer. The audio data in the buffer is analyzed to calculate a stream of values of one or more sound quality measures, such as speech transmission index and signal-to-noise ratio. Speech transmission index can be calculated using a convolution neural network configured to predict speech transmission index from reverberant speech. The stream of values can be used to provide real-time feedback about sound quality of the audio data. For example, a visual indicator on a graphical user interface can be updated based on consistency of the values over time. The real-time feedback about sound quality can help users optimize their recording setup.

Sound quality prediction and interface to facilitate high-quality voice recordings

Systems, methods, and apparatus are provided for equalization preference learning for digital audio modification. A method for listener calibration of an audio signal includes modifying a reference sound using at least one equalization curve; playing the modified reference sound for a listener; accepting listener feedback regarding the modified reference sound; and generating a weighting function based on listener feedback. A listener audio configuration system includes an output providing a sound for listener review; an interface accepting listener feedback regarding the sound; and a processor programming an audio device based on listener feedback.

Systems, methods, and apparatus for equalization preference learning

Systems, methods, and apparatus for equalization preference learning are provided. An example method includes receiving an audio input with respect to a target sound. The example method includes extracting one or more features from the audio input to provide one or more examples for rating based on the audio input. The example method includes generating a query based on the audio input and the one or more rated examples. The example method includes providing one or more synthesizer suggestion results identified in a search based on the query. The example method includes evaluating the one or more results with respect to the target sound. When one of the results matches the target sound, the example method includes outputting synthesizer parameters associated with the result. When none of the results matches the target sound, the example method includes refining the query for a second search based on feedback with respect to the one or more results.

Systems, methods, and apparatus to search audio synthesizers using vocal imitation

A method includes determining a first spectrogram of the audio signal, defining a similarity matrix of the audio signal based on the first spectrogram and a transposed version of the first spectrogram, identifying two or more similar frames in the similarity matrix that are more similar to a designated frame than to one or more other frames in the similarity matrix, creating a repeating spectrogram model based on the two or more similar frames that are identified in the similarity matrix, and deriving a mask based on the repeating spectrogram model and the first spectrogram of the audio signal. The mask is representative of similarities between the repeating spectrogram model and the first spectrogram of the audio signal. The method also includes extracting a repeating structure from the audio signal by applying the mask to the audio signal.

Growing community of inventors

Evanston, IL, United States of America

Bryan A Pardo