Systems and methods for audio source separation using band-split transformers

Abstract

The present disclosure describes techniques for implementing improved audio source separation. A complex spectrum X is split into a plurality of K bands along a frequency axis by applying band-split operations on the complex spectrum X. The complex spectrum is a time-frequency representation of audio signals. Each of the plurality of K bands is denoted as X, k=1, . . . , K. Each band Xcomprises one or more frequency bins. Each individual multilayer perceptron is applied to each band Xto extract latent representations and obtain outputs H. A time-domain transformer and a frequency-domain transformer are applied on a stacked representation H. Time-domain and frequency-domain transformers are repeatedly applying in an interleaved manner for L times to obtain Houtput from the transformer blocks. The His input into a multi-band mask estimation sub-model. A complex ideal ratio mask is generated based on outputs from the multi-band mask estimation sub-model.