Nippon Telegraph and Telephone Corporation

The University of Tokyo

Hirokazu Kameoka

Takehiro Moriya

Yutaka Kamamoto

Noboru Harada

Ryosuke Sugiura

Takuhiro Kaneko

Ko Tanaka

Nobukatsu Hojo

Takahito Kawanishi

Yasunori Oishi

Li Li

Aaron Valero Puche

A voice signal conversion model learning device includes: a generation unit configured to execute generation processing of generating a conversion destination voice signal on the basis of an input voice signal that is a voice signal of an input voice, conversion source attribute information that is information indicating an attribute of an input voice that is a voice represented by the input voice signal, and conversion destination attribute information indicating an attribute of a voice represented by the conversion destination voice signal that is a voice signal of a conversion destination of the input voice signal; and an identification unit configured to execute voice estimation processing of estimating whether or not a voice signal that is a processing target is a voice signal representing a vocal sound actually uttered by a person on the basis of the conversion source attribute information and the conversion destination attribute intonation, wherein the conversion destination voice signal is input to the identification unit, the processing target is a voice signal input to the identification unit, and the generation unit and the identification unit pertain learning on the basis of an estimation result of the voice estimation processing.

Audio signal conversion model learning apparatus, audio signal conversion apparatus, audio signal conversion model learning method and program

A voice conversion device is provided with a linguistic information extraction unit that extracts linguistic information corresponding to utterance content from a conversion source voice signal, an appearance feature extraction unit that extracts appearance features expressing features related to the look of a person's face from a captured image of the person, and a converted voice generation unit that generates a converted voice on a basis of the linguistic information and the appearance features.

Voice conversion apparatus, voice conversion learning apparatus, image generation apparatus, image generation learning apparatus, voice conversion method, voice conversion learning method, image generation method, image generation learning method, and computer program

A conversion learning device includes: a source encoding unit that converts, by using a first machine learning model, a feature amount sequence of a source domain that is a characteristic of conversion-source content data, into a first internal representation vector sequence that is a matrix in which internal representation vectors at individual locations of the feature amount sequence of the source domain are arranged; a target encoding unit that converts, by using a second machine learning model, a feature amount sequence of a target domain that is a characteristic of conversion-target content data, into a second internal representation vector sequence that is a matrix in which internal representation vectors at individual locations of the feature amount sequence of the target domain are arranged; an attention matrix calculation unit that calculates, by using the first internal representation vector sequence and the second internal representation vector sequence, an attention matrix that is a matrix mapping the individual locations of the feature amount sequence of the source domain to the individual locations of the feature amount sequence of the target domain, and calculates a third internal representation vector sequence that is a product of an internal representation vector sequence calculated by linear conversion of the first internal representation vector sequence and the attention matrix; a target decoding unit that calculates, by using the third internal representation vector sequence, a feature amount sequence of a conversion domain that is used to convert the source domain into the conversion domain, by using a third machine learning model; and a learning execution unit that causes at least one of the target encoding unit and the target decoding unit to learn such that a distance between a submatrix of the feature amount sequence of the target domain and a submatrix of the feature amount sequence of the conversion domain becomes shorter.

Conversion learning apparatus, conversion learning method, conversion learning program and conversion apparatus

It is possible to stably learn, in a short time, a model that can output embedded vectors for calculating a set of time frequency points at which the same sound source is dominant. Parameters of the neural network are learned based on a spectrogram of a sound source signal formed by a plurality of sound sources such that embedded vectors for time frequency points at which the same sound source is dominant are similar to embedded vectors for each of time frequency points output by a neural network, which is a CNN.

Sound signal model learning device, sound signal analysis device, method and program

To be able to convert to a voice of the desired attribution. The present invention includes learning, on the basis of parallel data of a sound feature value series in a conversion-source voice signal and a latent variable series in the conversion-source voice signal, and an attribution code indicating attribution of the conversion-source voice signal, an encoder for estimating a latent variable series from input of a sound feature value series and an attribution code, and a decoder for reconfiguring the sound feature value series from input of the latent variable series and the attribution code, to maximize a value of an objective function, the objective function being represented using attribution code similarity of a sound feature value series reconfigured by the decoder from input of an error between the sound feature value series reconfigured by the decoder and the sound feature value series in the conversion-source voice signal in the parallel data, a distance between the latent variable series estimated by the encoder and the latent variable series in the conversion-source voice signal in the parallel data, and any attribution code, the attribution code similarity being similarity to the any attribution code identified by an attribution identifier.

Voice conversion learning device, voice conversion device, method, and program

To be able to convert to a voice of the desired attribution. A learning unit learns a converter to minimize a value of a learning criterion of the converter, learns a voice identifier to minimize a value of a learning criterion of the voice identifier, and learns an attribution identifier to minimize a value of a learning criterion of the attribution identifier.

A parameter included in a fundamental frequency pattern of a voice can be estimated from the fundamental frequency pattern with high accuracy and the fundamental frequency pattern of the voice can be reconstructed from the parameter included in the fundamental frequency pattern. A learning unitlearns a deep generation model including an encoder which regards a parameter included in a fundamental frequency pattern in a voice signal as a latent variable of the deep generation model and estimates the latent variable from the fundamental frequency pattern in the voice signal on the basis of parallel data of the fundamental frequency pattern in the voice signal and the parameter included in the fundamental frequency pattern in the voice signal, and a decoder which reconstructs the fundamental frequency pattern in the voice signal from the latent variable.

Device, method, and program for analyzing speech signal

To be able to convert to a voice of the desired attribution. Learning an encoder for, on the basis of parallel data of a sound feature vector series in a conversion-source voice signal and a latent vector series in the conversion-source voice signal, and an attribution label indicating attribution of the conversion-source voice signal, estimating a latent vector series from input of a sound feature vector series and an attribution label, and a decoder for reconfiguring the sound feature vector series from input of the latent vector series and the attribution label.

Audio conversion learning device, audio conversion device, method, and program

The present invention relates to methods of converting a speech into another speech that sounds more natural. The method includes learning for a target conversion function and a target identifier according to an optimal condition in which the target conversion function and the target identifier compete with each other. The target conversion function converts source speech into target speech. The target identifier identifies whether the converted target speech follows the same distribution as actual target speech. The methods include learning for a source conversion function and a source identifier according to an optimal condition in which the source conversion function and the source identifier compete with each other. The source conversion function converts target speech into source speech, and the source identifier identifies whether the converted source speech follows the same distribution as actual source speech.

Device for learning speech conversion, and device, method, and program for converting speech

The present invention reduces encoding distortion in frequency domain encoding compared to conventional techniques, and obtains LSP parameters that correspond to quantized LSP parameters for the preceding frame and are to be used in time domain encoding from coefficients equivalent to linear prediction coefficients resulting from frequency domain encoding. When p is an integer equal to or greater than 1, a linear prediction coefficient sequence which is obtained by linear prediction analysis of audio signals in a predetermined time segment is represented as a[1], a[2], . . . , a[p], and &#x3c9;[1], &#x3c9;[2], . . . , &#x3c9;[p] are a frequency domain parameter sequence derived from the linear prediction coefficient sequence a[1], a[2], . . . , a[p], an LSP linear transformation unit () determines the value of each converted frequency domain parameter &#x2dc;&#x3c9;[i] (i=1, 2, . . . , p) in a converted frequency domain parameter sequence &#x2dc;&#x3c9;[1], &#x2dc;&#x3c9;[2], . . . , &#x2dc;&#x3c9;[p] using the frequency domain parameter sequence &#x3c9;[1], &#x3c9;[2], . . . , &#x3c9;[p] as input, through linear transformation which is based on the relationship of values between &#x3c9;[i] and one or more frequency domain parameters adjacent to &#x3c9;[i].

Decoding method, apparatus and recording medium

The present invention reduces encoding distortion in frequency domain encoding compared to conventional techniques, and obtains LSP parameters that correspond to quantized LSP parameters for the preceding frame and are to be used in time domain encoding from coefficients equivalent to linear prediction coefficients resulting from frequency domain encoding. When p is an integer equal to or greater than 1, a linear prediction coefficient sequence which is obtained by linear prediction analysis of audio signals in a predetermined time segment is represented as a[1], a[2], . . . , a[p], and &#x3c9;[1], &#x3c9;[2], . . . , &#x3c9;[p] are a frequency domain parameter sequence derived from the linear prediction coefficient sequence a[1], a[2], . . . , a[p], an LSP linear transformation unit () determines the value of each converted frequency domain parameter {tilde over (&#x2003;)}&#x3c9;[i] (i=1, 2, . . . , p) in a converted frequency domain parameter sequence {tilde over (&#x2003;)}w[1], {tilde over (&#x2003;)}w[2], . . . , {tilde over (&#x2003;)}&#x3c9;[p] using the frequency domain parameter sequence &#x3c9;[1], &#x3c9;[2], . . . , &#x3c9;[p] as input, through linear transformation which is based on the relationship of values between &#x3c9;[i] and one or more frequency domain parameters adjacent to &#x3c9;[i].

Frequency domain parameter sequence generating method, encoding method, decoding method, frequency domain parameter sequence generating apparatus, encoding apparatus, decoding apparatus, program, and recording medium

The present invention reduces encoding distortion in frequency domain encoding compared to conventional techniques, and obtains LSP parameters that correspond to quantized LSP parameters for the preceding frame and are to be used in time domain encoding from coefficients equivalent to linear prediction coefficients resulting from frequency domain encoding. When p is an integer equal to or greater than 1, a linear prediction coefficient sequence which is obtained by linear prediction analysis of audio signals in a predetermined time segment is represented as a[], a[], . . . , a[p], and &#x3c9;[], . . . , &#x3c9;[p] are a frequency domain parameter sequence derived from the linear prediction coefficient sequence a[], a[], . . . , a[p], an LSP linear transformation unit () determines the value of each converted frequency domain parameter &#x2dc;&#x3c9;[i] (i=1, 2, . . . , p) in a converted frequency domain parameter sequence &#x2dc;&#x3c9;[], . . . , &#x2dc;&#x3c9;[p] using the frequency domain parameter sequence &#x3c9;[], . . . , &#x3c9;[p] as input, through linear transformation which is based on the relationship of values between &#x3c9;[i] and one or more frequency domain parameters adjacent to &#x3c9;[i].

A linear predictive coding apparatus is provided that performs linear predictive analysis using a pseudo correlation function signal sequence obtained by performing inverse Fourier transform regarding the &#x3b7;-th power of the absolute values of the frequency domain sample sequence corresponding to the time-series signal as a power spectrum to obtain coefficients transformable to linear predictive coefficients. The apparatus further adapts values of &#x3b7; for a plurality of candidates for coefficients transformable to linear predictive coefficients stored in a code book and the coefficients transformable to linear predictive coefficients are obtained by the linear predictive analysis. The apparatus further obtains a linear predictive coefficient code corresponding to the coefficients transformable to linear predictive coefficients obtained by the linear predictive analysis, using the plurality of candidates for coefficients transformable to linear predictive coefficients and the coefficients transformable to linear predictive coefficients for which the values of &#x3b7; have been adapted.

Coding and decoding a sound signal by adapting coefficients transformable to linear predictive coefficients and/or adapting a code book

A parameter determination device includes: a spectral envelope estimating portion performing estimation of a spectral envelope using a parameter &#x3b7;specified in a predetermined method, regarding the &#x3b7;-th power of absolute values of a frequency domain sample sequence corresponding to a time-series signal as a power spectrum on the assumption that the parameter &#x3b7;and a parameter &#x3b7; are positive numbers; a whitened spectral sequence generating portion obtaining a whitened spectral sequence which is a sequence obtained by dividing the frequency domain sample sequence by the spectral envelope; and a parameter acquiring portion determining such a parameter &#x3b7; that generalized Gaussian distribution with the parameter &#x3b7; as a shape parameter approximates a histogram of the whitened spectral sequence.

Parameter determination device, method, program and recording medium for determining a parameter indicating a characteristic of sound signal

An encoding apparatus is an encoding apparatus for encoding a time-series signal for each of predetermined time sections in a frequency domain, wherein a parameter &#x3b7; is a positive number, the parameter &#x3b7; corresponding to a time-series signal is a shape parameter of generalized Gaussian distribution that approximates a histogram of a whitened spectral sequence, which is a sequence obtained by dividing a frequency domain sample sequence corresponding to the time-series signal by a spectral envelope estimated by regarding the &#x3b7;-th power of absolute values of the frequency domain sample sequence as a power spectrum, and any of a plurality of parameters &#x3b7; is selective or the parameter &#x3b7; is variable for each of the predetermined time sections; and the encoding apparatus comprises an encoding portion encoding the time-series signal for each of the predetermined time sections by an encoding process with a configuration identified at least based on the parameter &#x3b7; for each of the predetermined time sections.

Apparatuses and methods for encoding and decoding a time-series sound signal by obtaining a plurality of codes and encoding and decoding distortions corresponding to the codes

Growing community of inventors

Atsugi, Japan

Hirokazu Kameoka