Neural network based time-frequency mask estimation and beamforming for speech pre-processing

Abstract

Techniques are provided for pre-processing enhancement of a speech signal. A methodology implementing the techniques according to an embodiment includes performing de-reverberation processing on signals received from an array of microphones, the signals comprising speech and noise. The method also includes generating time-frequency masks (TFMs) for each of the signals. The TFMs indicate the probability that a time-frequency component of the signal associated with that TFM element includes speech. The TFM generation is based on application of a recurrent neural network to the signals. The method further includes generating steering vectors based on speech covariance matrices and noise covariance matrices. The TFMs are employed to filter speech components of the signals, for calculation of the speech covariance, and noise components of the signals for calculation of the noise covariance. The method further includes performing beamforming on the signals, based on the steering vectors, to generate the enhanced speech signal.