Dolby Laboratories Licensing Corporation

International Business Machines Corporation

Nuance Communications, Inc.

Zhiwei Shuang

Grant Allen Davidson

Yong Qin

Qin Shi

Zhong Su

Jian Ye Chen

Xuejing Sun

Mark Stuart Vinton

Louis Dunn Fielder

Rhonda J Wilson

David Matthew Cooper

Kuan-Chieh Yen

Xiguang Zheng

Shenghua Bao

Dirk Jeroen Breebaart

Shi Lei Zhang

Glenn N Dickins

William F Ganong, Iii

Wen Liu

Liu Wen

Shilei Zhang

David S McGrath

Lie Lu

Bin Cheng

Jundai Sun

Ming Wang

David Gunawan

Shen Huang

Jie Zhou

Dong Shi

Michael William Mason

Sen Xu

Fan Ping Meng

Xuemei Yu

Huiqun Deng

Ziyu Yang

Guiping Wang

Jia Dai

David Mcgrath

Qingyuan Bin

Libin Luo

Shaofan Yang

David S Mcgrath

Jun Wang

Systems, methods, and computer program products for audio processing based on convolutional neural network (CNN) are described. The CNN architecture may comprise a multi-scale input block and a multi-scale nested block. The multi-scale input block may be configured to receive input data and to generate a first downsampled input data set by downsampling the input data. The multi-scale nested block may comprise a first encoding layer configured to generate a first encoded data set by performing a convolution based on the input data. The multi-scale nested block may comprise a second encoding layer configured to generate a second encoded data set by performing a convolution based on the first downsampled input data set. Furthermore, the multi-scale nested block may comprise a first convolutional layer configured to generate a first output data set by upsampling the second encoded data set, concatenating the first encoded data set and the upsampled second encoded data set, and performing a convolution. The first convolutional layer may be nested between the encoding layers and decoding layers, thereby increasing the number of communication channels with the CNN and simplifying the underlying optimization problem.

Method and appartus for audio processing using a nested convolutional neural network architechture

The present disclosure relates to reverberation generation for headphone virtualization. A method of generating one or more components of a binaural room impulse response (BRIR) for headphone virtualization is described. In the method, directionally-controlled reflections are generated, wherein directionally-controlled reflections impart a desired perceptual cue to an audio input signal corresponding to a sound source location. Then at least the generated reflections are combined to obtain the one or more components of the BRIR. Corresponding system and computer program products are described as well.

Reverberation generation for headphone virtualization

In some embodiments, virtualization methods for generating a binaural signal in response to channels of a multi-channel audio signal, which apply a binaural room impulse response (BRIR) to each channel including by using at least one feedback delay network (FDN) to apply a common late reverberation to a downmix of the channels. In some embodiments, input signal channels are processed in a first processing path to apply to each channel a direct response and early reflection portion of a single-channel BRIR for the channel, and the downmix of the channels is processed in a second processing path including at least one FDN which applies the common late reverberation. Typically, the common late reverberation emulates collective macro attributes of late reverberation portions of at least some of the single-channel BRIRs. Other aspects are headphone virtualizers configured to perform any embodiment of the method.

Generating binaural audio in response to multi-channel audio using at least one feedback delay network

Described herein is a method for Convolutional Neural Network (CNN) based speech source separation, wherein the method includes the steps of: (a) providing multiple frames of a time-frequency transform of an original noisy speech signal; (b) inputting the time-frequency transform of said multiple frames into an aggregated multi-scale CNN having a plurality of parallel convolution paths; (c) extracting and outputting, by each parallel convolution path, features from the input time-frequency transform of said multiple frames; (d) obtaining an aggregated output of the outputs of the parallel convolution paths; and (e) generating an output mask for extracting speech from the original noisy speech signal based on the aggregated output. Described herein are further an apparatus for CNN based speech source separation as well as a respective computer program product comprising a computer-readable storage medium with instructions adapted to carry out said method when executed by a device having processing capability.

Method and apparatus for speech source separation based on a convolutional neural network

Apparatus and methods for controlling a jitter buffer are described. In one embodiment, the apparatus for controlling a jitter buffer includes an inter-talkspurt delay jitter estimator for estimating an offset value of the delay of a first frame in the current talkspurt with respect to the delay of a latest anchor frame in a previous talkspurt, and a jitter buffer controller for adjusting a length of the jitter buffer based on a long term length of the jitter buffer for each frame and the offset value.

Controlling a jitter buffer

A method for steering binauralization of audio is provided. The method comprises steps of: receiving () an audio input signal, calculating () a confidence value indicating a likelihood that a current audio frame of the audio input signal comprises binauralized audio; determining () a state signal based on the confidence value; determining () a steering signal, based on the first confidence value, the state signal and an energy value of the audio frame; and generating () an audio output signal with steered binauralization by processing the audio input signal according to the steering signal.

Steering of binauralization of audio

In some embodiments, virtualization methods for generating a binaural signal in response to channels of a multi-channel audio signal, which apply a binaural room impulse response (BRIR) to each channel including by using at least one feed-back delay network (FDN) to apply a common late reverberation to a downmix of the channels. In some embodiments, input signal channels are processed in a first processing path to apply to each channel a direct response and early reflection portion of a single-channel BRIR for the channel, and the downmix of the channels is processed in a second processing path including at least one FDN which applies the common late reverberation. Typically, the common late reverberation emulates collective macro attributes of late reverberation portions of at least some of the single-channel BRIRs. Other aspects are headphone virtualizers configured to perform any embodiment of the method.

Growing community of inventors

Beijing, China

Zhiwei Shuang