Nippon Telegraph and Telephone Corporation

Massachusetts Institute of Technology

The University of Tokyo

Kunio Kashino

Kaoru Hiramatsu

Takahito Kawanishi

Yasunori Oishi

Akisato Kimura

Hiroshi Murase

Takayuki Kurozumi

Go Irie

Takuhiro Kaneko

Xiaomeng Wu

Hidehisa Nagano

Yoshitaka Ushiku

Shota Ikawa

Gavin Smith

James Robert Glass

Chihiro Watanabe

David F Harwath

Yasunori Ohishi

Kiyoharu Aizawa

Masataka Yamaguchi

Masaru Tsuchida

Krishna Onkar

A credit segment identifying device includes an extracting unit. The extracting units extracts, from a first speech signal, a plurality of first partial speech signals. Each of the plurality of first partial speech signals is a part of the first speech signals and shifted from each other in time direction. An identifying unit identifies a credit segment in the first speech signal by determining whether each of the first partial speech signals includes a credit according to an association between each of second partial signals and the presence/absence of a credit. The each of second partial signals is extracted from a second speech signal.

Sponsorship credit period identification apparatus, sponsorship credit period identification method and program

A search apparatus for searching media data for a target region that matches query data includes a first feature extraction unit configured to extract a first feature vector from the query data using a first trained neural network; a second feature extraction unit configured to obtain a first region from the media data and extract a second feature vector from the first region using a second trained neural network; a localization unit configured to determine a candidate for the target region using a third trained neural network, based on the first feature vector, the second feature vector, and the first region or a location of the first region; and a control unit configured to repeat the operations of the second feature extraction unit and the localization unit until a predetermined condition is satisfied, by using the determined candidate for the target region as the first region to be used by the second feature extraction unit.

Search apparatus, training apparatus, search method, training method, and program

To provide translation techniques for translating a natural language representation in one language into a natural language representation in another language using sound. With Land Lbeing different languages from each other, a translation data generation apparatus includes: a latent variable generation unit that generates, from a natural language representation in the language L, a latent variable corresponding to the natural language representation in the language Lusing a language Lencoder; an index calculation unit that calculates an index for the natural language representation in the language Lfrom the natural language representation in the language L; and a natural language representation generation unit that generates a natural language representation in the language Lcorresponding to the natural language representation in the language Lfrom the latent variable and the index for the natural language representation in the language Lusing a language Ldecoder.

Data generation model learning apparatus, latent variable generation model learning apparatus, translation data generation apparatus, data generation model learning method, latent variable generation model learning method, translation data generation method, and program

An acquiring unit of a depth estimation apparatus acquires an input image. In addition, a depth map generating unit inputs the input image acquired by the acquiring unit into a depth estimator for generating, from an image, a depth map in which a depth of a space that appears on the image is imparted to each pixel of the image, and generates an estimated depth map that represents a depth map corresponding to the input image. The depth estimator is a model having been learned in advance so as to reduce, with respect to each error between a depth of the estimated depth map and a depth of a correct-answer depth map that presents the depth map of a correct answer, a value of a loss function set such that a degree of increase of a loss value with respect to a pixel at which the error is larger than a threshold is smaller than a degree of increase of a loss value with respect to a pixel at which the error is equal to or smaller than the threshold.

Depth estimation device, depth estimation model learning device, depth estimation method, depth estimation model learning method, and depth estimation program

A signal generation device includes a variable generation unit and a signal generation unit. The variable generation unit generates a plurality of latent variables corresponding to a plurality of features of a signal. The signal generation unit inputs, to at least one neural network learned in advance, a latent variable representing attributes obtained by converting a part of the plurality of latent variables by an attribute vector representing attributes of a signal to be generated and the other part of the plurality of latent variables representing an identity and generates the signal to be generated using the at least one neural network.

Signal generation device, signal generation learning device, method, and program

It is possible to analyze what type of input/output mapping is mainly performed in the entire hidden layer of a neural network. A relationship analysis unitcalculates strength of a relationship of each combination of a dimension of the input data and a unit of the neural network and calculates strength of a relationship of each combination of the unit and a dimension of the output data. A role analysis unitcalculates a relationship between a prescribed number of types of roles and the unit and a relationship between the prescribed number of types of roles, the dimension of the input data, and the dimension of the output data on the basis of the strength of the relationship of each combination of the dimension of the input data and the unit of the neural network and the strength of the relationship of each combination of the unit and the dimension of the output data.

Analysis device, method, and program for analyzing neural network structure

To provide techniques for generating, from a sound signal, a natural language representation corresponding to the sound signal while controlling a predetermined index for a natural language representation. A data generation apparatusincludes: a latent variable generation unitthat generates, from a sound signal, a latent variable corresponding to the sound signal using an encoder; and a data generation unitthat generates a natural language representation corresponding to the sound signal from the latent variable and a condition concerning an index for the natural language representation using a decoder.

Data generation model learning apparatus, data generation apparatus, data generation model learning method, data generation method, and program

An image adjustment device includes: an illumination component derivation unit that derives an illumination component of a grayscale image; a reflectance component derivation unit that derives a reflectance component image that is a resulting image in which the illumination component is removed from the grayscale image; a contrast component derivation unit that derives a contrast component based on a contrast value between a pixel of the reflectance component image and a peripheral area of the pixel; a histogram derivation unit that derives a luminance histogram of the grayscale image weighted according to the contrast value for each pixel of the contrast component; a conversion function derivation unit that derives a luminance conversion function for converting a luminance such that a luminance histogram of a converted grayscale image in which the grayscale image is converted by the luminance conversion function and a predetermined histogram are matched with or similar to each other; and a luminance conversion unit that generates the converted grayscale image.

Image adjustment apparatus, image adjustment method, and program

The present invention enables the structure of a neural network to be quantitatively analyzed. An analyzing unit calculates, for each of combinations of a dimension of input data and a cluster, a sum of squared errors between an output of each unit belonging to the cluster when a value of the dimension of the input data is replaced with an average value of the dimension of the input data included in learning data and an output of each unit belonging to the cluster for the input data before replacement as a relationship between the combinations, and calculates, for each of combinations of the cluster and a dimension of output data, a squared error between the value of the dimension of the output data when an output value of each unit belonging to the cluster is replaced with an average output value of each unit of the cluster when the input data included in the learning data was input and the value of the dimension of the output data before replacement as a relationship between the combinations.

Device, method, and program for quantitatively analyzing structure of a neural network

To provide database generation techniques that can accurately and efficiently generate a database useable in text-based sound signal search. A sound signal database generation apparatus includes: a latent variable generation unit that generates, from a sound signal, a latent variable corresponding to the sound signal using a sound signal encoder; a data generation unit that generates a natural language representation corresponding to the sound signal from the latent variable and a condition concerning an index for a natural language representation using a natural language representation decoder; and a sound signal database generation unit that generates a record including the natural language representation corresponding to the sound signal and the sound signal from the natural language representation corresponding to the sound signal and the sound signal, and generates a sound signal database made up of the record.

Sound signal database generation apparatus, sound signal search apparatus, sound signal database generation method, sound signal search method, database generation apparatus, data search apparatus, database generation method, data search method, and program

The present invention enables detection of a sponsorship credit display in a broadcast program with higher precision. A sponsorship credit display detection deviceaccording to the present invention includes: a learning data creation unitthat creates, as learning data, from a broadcast program in which the sponsorship credit display has been detected, a still image where the sponsorship credit display is displayed and a still image where the sponsorship credit display is not displayed; a learning unitthat uses the learning data created by the learning data creation unitto learn a parameter to be applied to a detection model for detecting a sponsorship credit display in a broadcast program; and a sponsorship credit display detection unitthat uses the detection model to which the parameter learned by the learning unithas been applied to detect a sponsorship credit display in a recognition target broadcast program.

Device and method for detecting display of provided credit, and program

A learning device calculates a feature of each data included in a pair of datasets in which two modalities among a plurality of modalities are combined, using a model that receives data on a corresponding modality among the modalities and outputs a feature obtained by mapping the received data into an embedding space. The learning device then selects similar data similar to each target data that is data on a first modality in a first dataset of the datasets, from data on a second modality included in a second dataset of the datasets. The learning device further updates a parameter of the model such that the features of the data in the pair included in the first and the second datasets are similar to one another, and the feature of data paired with the target data is similar to the feature of data paired with the similar data.

Learning device, learning method, and learning program for images and sound which uses a similarity matrix

A learning device calculates an image feature using a model (image encoder) that receives an image and outputs the image feature obtained by mapping the image into a latent space. The learning device calculates an audio feature using a model (audio encoder) that receives a speech in a predetermined language and outputs the audio feature obtained by mapping the speech into the latent space, and that includes a neural network provided with a self-attention mechanism. The learning device updates parameters of the models used by an image feature calculation unit and an audio feature calculation unit such that the image feature of a first image is similar to the audio feature of a speech corresponding to the first image.

Learning device, learning method, learning program, retrieval device, retrieval method, and retrieval program

The purpose of the present invention is to enable learning of a neural network for extracting features of images having high robustness from an undiscriminating image region while minimizing the number of parameters of a pooling layer. A parameter learning unitlearns parameters of each layer in a convolutional neural network configured by including a fully convolutional layer for performing convolution of an input image to output a feature tensor of the input image, a weighting matrix estimation layer for estimating a weighting matrix indicating a weighting of each element of the feature tensor, and a pooling layer for extracting a feature vector of the input image based on the feature tensor and the weighting matrix. The parameter learning unitlearns the parameters such that a loss function value obtained by calculating a loss function expressed by using a distance between a first feature vector of a first image and a second feature vector of a second image, which are relevant images and are obtained by applying the convolutional neural network, becomes smaller.

Image feature learning device, image feature learning method, image feature extraction device, image feature extraction method, and program

To enable extraction of an area related to a sponsor credit from a video including the sponsor credit of a television broadcast or the like without generating learning data for each form of various kinds of sponsor credits. A detection device () according to the present invention includes a detection unit () that associates a still image including a prescribed character or figure from a preliminary video or a still image not including the prescribed character or figure with a sound signal including the prescribed sound acquired from the preliminary video so as to detect a desired scene as an area that includes at least one of the prescribed character or figure and the prescribed sound from the target video.

Growing community of inventors

Tokyo, Japan

Kunio Kashino