Lexical analysis training of convolutional neural network by windows of different lengths with matrix of semantic vectors

Abstract

An information processing apparatus includes a lexical analysis unit that generates a training word string, a group generation unit that generates a plurality of training word groups, a matrix generation unit that generates, for each training word group, a training matrix in which a plurality of words and respective semantic vectors of the words are associated, a classification unit that calculates, for a word of each position of the training word string, a probability of the word corresponding to a specific word, using the training matrices generated by the matrix generation unit and a determination model that uses a convolutional neural network, and an optimization processing unit that updates parameters of the determination model, such that the probability of the word labeled as corresponding to the specific word is high, among the probabilities of the words of the respective positions of the training word string calculated by the classification unit.