Method of associating multiple modalities and a multimodal system

Abstract

The multimodal managing system invention comprises: a modality linking data generating unit that generates association data in order to associate a first modality and a second modality of the plurality of modalities; an association correspondence unit that sends content that includes association data, wherein association data is combined with the content to be sent to the first modality, to the first modality; a session information storage unit that stores the correspondence among the connection information of each modality, the link information about the links between the modalities, and the association data generated by the modality linking data generating unit as a session information table; and a modality associating unit that receives link request data sent from the second modality using the association data included in the content that includes association data, and executes a matching process that compares the link request data with the association data stored in the session information storage unit.