Systems, methods, and apparatuses for restoring degraded speech via a modified diffusion model

Abstract

Systems, methods, and apparatuses to restore degraded speech via a modified diffusion model are described. An exemplary system is specially configured to train a diffusion-based vocoder containing an upsampler, based on pairing original speech x and degraded speech mel-spectrum msamples; train a deep convoluted neural network (CNN) upsampler based on a mean absolute error loss to match the estimated original speech {circumflex over (x)}′ outputted by the diffusion-based vocoder by extracting the upsampler, generating a reference conditioner, and generating a weighted altered conditioner c′. The system further optimizes speech quality to invert non-linear transformation and estimate lost data by feeding the degraded mel-spectrum mthrough the CNN upsampler and feeding the degraded mel-spectrum mthrough the diffusion-based vocoder. The system then generates estimated original speech {circumflex over (x)}′ based on the corresponding degraded speech mel-spectrum m. Other related embodiments are described.