Nippon Telegraph and Telephone Corporation

Massachusetts Institute of Technology

Akisato Kimura

Kunio Kashino

Yoshitaka Ushiku

Takahito Kawanishi

James Robert Glass

Takayuki Kurozumi

David F Harwath

Yasunori Ohishi

Hiroshi Murase

A learning device calculates a feature of each data included in a pair of datasets in which two modalities among a plurality of modalities are combined, using a model that receives data on a corresponding modality among the modalities and outputs a feature obtained by mapping the received data into an embedding space. The learning device then selects similar data similar to each target data that is data on a first modality in a first dataset of the datasets, from data on a second modality included in a second dataset of the datasets. The learning device further updates a parameter of the model such that the features of the data in the pair included in the first and the second datasets are similar to one another, and the feature of data paired with the target data is similar to the feature of data paired with the similar data.

Learning device, learning method, and learning program for images and sound which uses a similarity matrix

A learning device calculates an image feature using a model (image encoder) that receives an image and outputs the image feature obtained by mapping the image into a latent space. The learning device calculates an audio feature using a model (audio encoder) that receives a speech in a predetermined language and outputs the audio feature obtained by mapping the speech into the latent space, and that includes a neural network provided with a self-attention mechanism. The learning device updates parameters of the models used by an image feature calculation unit and an audio feature calculation unit such that the image feature of a first image is similar to the audio feature of a speech corresponding to the first image.

Learning device, learning method, learning program, retrieval device, retrieval method, and retrieval program

A repetitive structure extraction device includes an image feature extraction unit which extracts an image feature for each of a plurality of images which are captured at one or a plurality of locations and which are given different capture times, a temporal feature extraction unit which extracts, for each of the plurality of images, a temporal feature according to a predetermined period from a capture time given to the image, and a repetitive structure extraction unit which learns, on the basis of the image feature extracted for each of the plurality of images by the image feature extraction unit and the temporal feature extracted for each of the plurality of images by the temporal feature extraction unit, a repetitive structure which is used to perform interconversion between the temporal feature and a component of the image feature and which is provided according to a correlation of periodic change between the component of the image feature and the temporal feature.

Image retrieval device and method, photograph time estimation device and method, repetitive structure extraction device and method, and program

There are included: an initial sub-signal creation section which creates, from an original signal, sub-signals of shorter length than the original signal; a created sub-signal selection section which, for the sub-signals produced by the initial sub-signal creation section, prunes the created sub-signal candidates to those for which the amount of data is less than for the original signal; a sub-signal re-creation section which determines a created sub-signal which is actually to be used, using the created sub-signal candidates produced by the created sub-signal selection section; a compression mapping determination section which determines, from the sub-signals produced by the sub-signal re-creation section, a mapping for calculation of a compressed signal; and a signal compression section which calculates a compressed signal corresponding to the sub-signals obtained by the sub-signal re-creation section, based upon the mapping obtained by the compression mapping determination section.

Signal compression method, device, program, and recording medium; and signal retrieval method, device, program, and recording medium

A signal detection system of the invention comprises: a target signal feature quantity calculation section which derives a feature quantity series from a target signal; a stored signal feature quantity calculation section which derives a feature quantity series from a stored signal; a target signal histogram calculation section which searches for a histogram of feature quantities in a target signal feature quantity series; a stored signal histogram series calculation section which obtains a histogram series in regard to a stored signal feature quantity series, by calculating a histogram of feature quantities; a stored signal histogram grouping section which groups histogram series, for which a similarity level satisfies a criteria; a stored signal histogram group selection section which selects groups which include an area to be output from a histogram group; a stored signal collation section which performs collation with respect to histograms of a histogram group, and obtains a similarity value; and a collation result output section which outputs the area collated by the similarity value, as a detection result.

Growing community of inventors

Atsugi, Japan

Akisato Kimura