National Institute of Advanced Industrial Science and Technology

Nec Corporation

Yamaha Corporation

Nippon Telegraph and Telephone Corporation

Oki Electric Industry Co., Ltd.

Masataka Goto

Tomoyasu Nakano

Hiroshi Okuno

Takuya Fujishima

Keita Arimoto

Hiromasa Fujihara

Kazuyoshi Yoshii

Jun Ogata

Yasuhiro Sato

Takayuki Goto

Tetsuya Kato

Kenji Sato

Kazuyuki Tate

Tohru Suzuki

Yoichi Ogawa

Katsutoshi Itoyama

Kosetsu Tsukuda

Satoru Fukayama

Takahiro Inoue

Elias Pampalk

Katunobu Itou

Katsutoshi Itoyma

A system for generating topic inference information of lyrics that can provide more useful for topic interpretation of lyrics. A device for learning topic numbers performs an operation of updating and learning topic numbers, which performs an operation of updating topic numbers on all of a plurality of lyrics data of each of a plurality of artists, for a predetermined number of times. The operation of updating topic numbers updates the topic number assigned to a given lyrics data of a given artist using a random number generator having a deviation of appearance probability corresponding to a probability distribution over topic numbers. An outputting device outputs the topic numbers of the plurality of lyrics data for each of the plurality artists, and a probability distribution over words for each of the topic numbers.

System for generating topic inference information of lyrics

A music-synchronized control platform to synchronize music playback or music-synchronized performance or both with music in a plurality of devices. A master control section of at least one music-synchronized control master controls a master music playback player or master music-synchronized performer by converting a timing in the reference clock time corresponding to an in-content playback timing, which is obtained from the status of a virtual music playback player, to a timing in the individual clock time corresponding to the in-content playback timing. The slave control section of each of the plurality of music-synchronized control slaves control a slave music playback player or a slave music-synchronized performer and by converting a timing in the reference clock time corresponding to the in-content playback timing, which is obtained from the status of the virtual music playback player, to a timing in the individual clock time corresponding to the in-content playback timing.

Platform for control in synchronization with music and control method therefor

A system, method, and computer program for estimation of a target value, which can change the aggregation of estimation results based on the degrees of confidence, taking the nature of an input observation signal into consideration. An unknown observation signal is input to a plurality of regression models. A plurality of estimated values are respectively obtained by a plurality of regression models corresponding to the plurality of features of the unknown observation signal. A target value of the unknown observation signal is estimated by aggregation of the target values. The estimating section calculates weights to be added the estimation results output from the regression models, based on the degrees of confidence with respect to the inputs into the regression models. A target value of the unknown observation signal is estimated through the aggregation by calculating a weighted sum of the estimation results output from the regression models.

System, method, and computer program for estimation of target value

A system for multifaceted singing analysis for retrieval of songs or music including singing voices having some relationship in latent semantics with a singing voice included in one particular song or music. A topic analyzing processor uses a topic model to analyze a plurality of vocal symbolic time series obtained for a plurality of musical audio signals. The topic analyzing processor generates a vocal topic distribution for each of the musical audio signals whereby the vocal topic distribution is composed of a plurality of vocal topics each indicating a relationship of one of the musical audio signals with the other musical audio signals. The topic analyzing processor generates a vocal symbol distribution for each of the vocal topics whereby the vocal symbol distribution indicates occurrence probabilities for the vocal symbols. A multifaceted singing analyzing processor performs analysis of singing voices included in musical audio signals, in the multifaceted viewpoint.

System and method for multifaceted singing analysis

A singing synthesis section for generating singing by integrating into one singing a plurality of vocals sung by a singer a plurality of times or vocals of which parts that he/she does not like are sung again. A music audio signal playback section plays back the music audio signal from a signal portion or its immediately preceding signal corresponding to a character in the lyrics when the character displayed on the display screen is selected by a character selecting section. An estimation and analysis data storing section automatically aligns the lyrics with the vocal, decomposes the vocal into three elements, pitch, power, and timber, and stores them. A data selecting section allows the user to select each of the three elements for respective time periods of phonemes. The data editing section modifies the time periods of the three elements in alignment with the modified time periods of the phonemes.

System and method for singing synthesis

For high-accuracy analysis and high-quality synthesis of voice sound (singing and speech), provided herein are a system and a method for estimating from an audio signal spectral envelopes and group delays for sound analysis and synthesis with high accuracy and high temporal resolution. An estimation system of spectral envelopes and group delays includes a fundamental frequency estimation section, an amplitude spectrum acquisition section, a group delay extraction section, a spectral envelope integration section, and a group delay integration section. The spectral envelope integration section sequentially obtains a spectral envelope for sound synthesis by averaging overlapped spectra. The group delay integration section selects from a plurality of group delays a group delay corresponding to the maximum envelope of each frequency component of the spectral envelope and integrates groups delays thus selected to sequentially obtain a group delay for sound synthesis.

Estimation system of spectral envelopes and group delays for sound analysis and synthesis, and audio signal synthesis system

Herein provided is a system for singing synthesis capable of reflecting not only pitch and dynamics changes but also timbre changes of a user's singing. A spectral transform surface generating sectiontemporally concatenates all the spectral transform curves estimated by a second spectral transform curve estimating sectionto define a spectral transform surface. A synthesized audio signal generating sectiongenerates a transform spectral envelope at each instant of time by scaling a reference spectral envelope based on the spectral transform surface. Then, the synthesized audio signal generating sectiongenerates an audio signal of a synthesized singing voice reflecting timbre changes of an input singing voice, based on the transform spectral envelope and a fundamental frequency contained in a reference singing voice source data.

System and method for singing synthesis capable of reflecting voice timbre changes

A system provided herein may perform automatic temporal alignment between music audio signal and lyrics with higher accuracy than ever. A non-fricative section extracting 4 extracts non-fricative sound sections, where no fricative sounds exist, from the music audio signal. An alignment portionincludes a phone modelfor singing voice capable of estimating phonemes corresponding to temporal-alignment features. The alignment portionperforms an alignment operation using as inputs temporal-alignment features obtained from a temporal-alignment feature extracting portion, information on vocal and non-vocal sections obtained from a vocal section estimating portion, and a phoneme network SN on conditions that no phonemes exist at least in non-vocal sections and that no fricative phonemes exist in non-fricative sound sections.

System and method for automatic temporal alignment between music audio signal and lyrics

Disclosed herein is a pitch estimation apparatus and associated methods for estimating a fundamental frequency of an audio signal from a fundamental frequency probability density function by modeling the audio signal as a weighted mixture of a plurality of tone models corresponding respectively to harmonic structures of individual fundamental frequencies, so that the fundamental frequency probability density function of the audio signal is given as a distribution of respective weights of the plurality of the tone models.

Estimating pitch by modeling audio as a weighted mixture of tone models for harmonic structures

An unknown word is additionally registered in a speech recognition dictionary by utilizing a correction result, and a new pronunciation of the word that has been registered in a speech recognition dictionary is additionally registered in the speech recognition dictionary, thereby increasing the accuracy of speech recognition. The start time and finish time of each phoneme unit in speech data corresponding to each phoneme included in a phoneme sequence acquired by a phoneme sequence converting sectionare added to the phoneme sequence. A phoneme sequence extracting sectionextracts from the phoneme sequence a phoneme sequence portion composed of phonemes existing in a segment corresponding to the period from the start time to the finish time of the word segment of the word corrected by a word correcting sectionand the extracted phoneme sequence portion is determined as the pronunciation of the corrected word. An additional registration sectioncombines the corrected word with the pronunciation determined by a pronunciation determining sectionand additionally registers the combination as new word pronunciation data in the speech recognition dictionaryif it is determined that a word obtained after correction has not been registered in the speech recognition dictionary. The additional registration sectionadditionally registers the pronunciation determined by the pronunciation determining sectionas another pronunciation of the corrected word if it is determined that the corrected word has been registered.

Speech recognition system and program therefor

A musical piece recommendation system that allows instantaneous registration of a new user and a new musical piece without retraining in a basic training section. A first incremental training section monitors a rating history storage section, and each time a change is made to a rating history or a new user is added, performs updating of or addition of the topic selection probability for the user for which the change is made to the rating history or for the new user such that the likelihood determined by a basic training section is kept maximized. A second incremental training section monitors an acoustic feature storage section, and each time a new musical piece is added to perform addition to acoustic features, adds the musical piece selection probability related to the added musical piece such that the likelihood determined by the basic training section is kept maximized.

Musical piece recommendation system and method

A music information retrieval system of the present invention can retrieve unknown songs including singing voices having similar voice timbres. Voice timbre features of the songs and identifiers for the respective songs are stored in voice timbre feature storage section. When one of the songs is selected, similarity calculation sectioncalculates voice timbre similarities between the selected song and the respective remaining songs, based on voice timbre features of the selected song and the other songs. Similar song retrieval and display sectiondisplays on a displaya plurality of identifiers for songs which are similar to the selected song in voice timbre. Song data reproduction sectionreproduces song data corresponding to one or more identifiers selected from among the plurality of identifiers displayed on the display

Music information retrieval system

There is provided a singing synthesis parameter data estimation system that automatically estimates singing synthesis parameter data for automatically synthesizing a human-like singing voice from an audio signal of input singing voice. A pitch parameter estimating sectionestimates a pitch parameter, by which the pitch feature of an audio signal of synthesized singing voice is got closer to the pitch feature of the audio signal of input singing voice based on at least both of the pitch feature and lyric data with specified syllable boundaries of the audio signal of input singing voice. A dynamics parameter estimating sectionconverts the dynamics feature of the audio signal of input singing voice to a relative value with respect to the dynamics feature of the audio signal of synthesized singing voice, and estimates a dynamics parameter, by which the dynamics feature of the audio signal of synthesized singing voice is got close to the dynamics feature of the audio signal of input singing voice that has been converted to the relative value.

Singing synthesis parameter data estimation system

An audio signal produced by playing a plurality of musical instruments is separated into sound sources according to respective instrument sounds. Each time a separation process is performed, the updated model parameter estimation/storage sectionestimates parameters respectively contained in updated model parameters such that updated power spectrograms gradually change from a state close to initial power spectrograms to a state close to a plurality of power spectrograms most recently stored in a power spectrogram separation/storage section. Respective sections including the power spectrogram separation/storage sectionand an updated distribution function computation/storage sectionrepeatedly perform process operations until the updated power spectrograms change from the state close to the initial power spectrograms to the state close to the plurality of power spectrograms most recently stored in the power spectrogram separation/storage section. The final updated power spectrograms are close to the power spectrograms of single tones of one musical instrument contained in the input audio signal formed to contain harmonic and inharmonic models.

Sound source separation system, sound source separation method, and computer program for sound source separation

The present invention provides a music artist retrieval system which makes it possible for users to automatically retrieve an unknown music artist similar to the user's favorite artist while actually reproducing and confirming a piece of music of the unknown artist. A music artist similarity map storing section () computes a plurality of similarities for a plurality of music artists and makes a music artist similarity map for the plurality of music artists based on the plurality of similarities, then stores the music artist similarity map. Here, the similarities are computed between one of the plurality of music artists and the other music artists based on features of the respective music artists. A similar artists selecting and displaying section () displays on a display plurality of indications related to one music artist and two or more music artists whose similarities are close to the one music artist, based on the music artist similarity map. A music data playing section () reproduces music data of a music artist related to a selected artist indication when a play command is inputted.

Growing community of inventors

Tsukuba, Japan

Masataka Goto