Neural modulation codes for multilingual and style dependent speech and language processing

Abstract

Computer-implemented methods and apparatus that use neural modulation codes as an alternative to training many individual recognition models or to loosing performance by training mixed models. Large neural models are modulated by codes that model the different conditions. The codes directly alter (modulate) the behavior of connections in a multiconditional perceptual classifier, so as to permit the most appropriate neuronal units and their features to be applied to each condition. The approach may be applied to multilingual ASR, where the resulting multilingual network arrangement is able to achieve performance that is competitive or better than individually trained mono-lingual network. Moreover, the approach requires no adaptation data or extensive adaptation/training time to operate in a manner tuned to each condition. Beyond multilingual speech processing systems the approach can be applied to many other perceptual processing problems (e.g. speech recognition, speech synthesis, language translation, image processing) to factor the processing task from the conditioning variables that drive the actual realization. Instead of adapting or retraining neural systems to individual conditions, it modulates a large invariant network to operate in different modes based on conditioning codes that are provided by auxiliary networks that model these conditions.