International Business Machines Corporation

Samsung Electronics Co., Ltd.

Takayuki Osogami

Tetsuro Morimura

Takayuki Katsuki

Rudy Raymond Harry Putra

Sakyasingha Dasgupta

Raymond Harry Rudy

Shohei Ohsawa

Yachiko Obara

Masatoshi Ishii

Hiroki Yanagisawa

Kohji Hosokawa

Atsuya Okazaki

Junka Okazawa

Risa Kawanaka

Makoto Otsuka

Kun Zhao

Kohichi Kamijoh

Takashi Itoh

Ryo Kawahara

Hiroshi Kajino

Tomoyuki Shirai

Mikael F Onsjö

Toshinari Itoki

Hiroshi Horii

Toshihiro Takahashi

Akira Koseki

Masaki Ono

Shoko Suzuki

Teruo Koyanagi

Toshinari Itoko

Akifumi Wachi

Sei Kato

Ryo Iwaki

Muhammad Anis Uddin Nasir

Sadamori Kojaku

Rikiya Takahaski

A computer-implemented method and an apparatus are provided for learning a first model. The method includes generating a second model based on the first model. The first model is configured to perform a learning process based on sequentially inputting each of a plurality of pieces of input data that include a plurality of input values and that are from a first input data sequence. The second model is configured to learn a first learning target parameter included in the first model based on inputting, in an order differing from an order in the first model, each of a plurality of pieces of input data that include a plurality of input values and are from a second input data sequence. The method further includes performing a learning process using both the first model and the second model. The method also includes storing the first model that has been learned.

Learning apparatus and method for bidirectional learning of predictive model based on data sequence

A computer-implemented method is provided for offline reinforcement learning with a dataset. The method includes training a neural network which inputs a state-action pair and outputs a respective Q function for each of a reward and one or more safety constraints, respectively. The neural network has a linear output layer and remaining non-linear layers being represented by a feature mapping function. The training includes obtaining the feature mapping function by constructing Q-functions based on the dataset according to an offline reinforcement algorithm. The training further includes tuning, using the feature mapping function, a weight between the reward and the one or more safety constraints, wherein during the obtaining and the tuning steps, an estimate of a Q-function is provided by subtracting an uncertainty from an expected value of the Q-function. The uncertainty is a function to map the state-action pair to an error size.

Stepwise uncertainty-aware offline reinforcement learning under constraints

A method for a determinantal Point Process-based prediction includes obtaining, using a hardware processor, a training data set stored on one or more computer readable storage mediums operably coupled to the hardware processor, training an asymmetric kernel of a Determinantal Point Process (DPP) from a training data set by calculating an inverse matrix of a sum of the asymmetric kernel and an identity matrix in a recursive manner to reduce time and computational resources utilized, and determining a prediction model by training the asymmetric kernel as at least part of a prediction model to make a prediction.

Training asymmetric kernels of determinantal point processes

A method is provided for choosing an action of an agent in a first team that competes against a second team. The method includes determining an action, based on first, second and third types of local payoff matrices. The method further includes performing the action. The determining step includes representing, by the first type of local payoff matrices, a payoff to the first team due to a pairwise interaction between agent teammates of the first team. The determining step further includes representing, by the second type of local payoff matrices, a payoff to the first team due to a pairwise interaction between agent opponents from the first team and the second team. The determining step also includes representing, by the third type of local payoff matrices, a payoff to the first team due to a pairwise interaction between agent teammates of the second team.

Risk sensitive approach to strategic decision making with many agents

A computer-implemented method, a computer program product, and a computer processing system are provided for online learning for a Dynamic Boltzmann Machine (DyBM) with hidden units. The method includes imposing, by a processor device, limited connections in the DyBM where (i) a current observation xdepends only on latest hidden units hand all previous observations xand (ii) the latest hidden units hdepend on all the previous observations xwhile being independent of older hidden units h. The method further includes computing, by the processor device, gradients of an objective function. The method also includes optimizing, by the processor device, the objective function in polynomial time using a stochastic Gradient Descent algorithm applied to the gradients.

Online learning for dynamic Boltzmann machines with hidden units

A method, a computer program product, and a system of adversarial semi-supervised one-shot training using a data stream. The method includes receiving a data stream based on an observation, wherein the data stream includes unlabeled data and labeled data. The method also includes training a prediction model with the labeled data using stochastic gradient descent based on a classification loss and an adversarial term and training a representation model with the labeled data and the unlabeled data based on a reconstruction loss and the adversarial term. The adversarial term is a cross-entropy between the middle layer output data from the models. The classification loss is a cross-entropy between the labeled data and an output from the prediction model. The method further includes updating a discriminator with middle layer output data from the prediction model and the representation model and based on a discrimination loss, and discarding the data stream.

Adversarial semi-supervised one-shot learning

An n-steps-ahead value of time-series data is estimated by a prediction model configured to output a sum of discounted m-th order differences of adjacent time steps at each time step, wherein an m-th order difference at a corresponding time step is discounted by using a discount factor such that an m-th order difference discount increases as a time step of the m-th order difference increases.

N-steps-ahead prediction based on discounted sum of m-th order differences

In an approach, a processor obtains a target base strategy for selecting actions of a target agent. A processor obtains an adversarial base strategy for selecting adversarial actions of an adversarial agent. A processor calculates, for each candidate action among a plurality of candidate actions of the target agent, a risk measure of the candidate action based on the adversarial base strategy and a payoff to the target agent in a case where the target agent takes the candidate action and the adversarial agent takes an adversarial action based on the adversarial base strategy. A processor generates a target strategy by adjusting the target base strategy based on the risk measure for each candidate action.

Generating strategy based on risk measures

A neuromorphic chip includes synaptic cells including respective resistive devices, axon lines, dendrite lines and switches. The synaptic cells are connected to the axon lines and dendrite lines to form a crossbar array. The axon lines are configured to receive input data and to supply the input data to the synaptic cells. The dendrite lines are configured to receive output data and to supply the output data via one or more respective output lines. A given one of the switches is configured to connect an input terminal to one or more input lines and to changeably connect its one or more output terminals to a given one or more axon lines.

Neuromorphic chip for updating precise synaptic weight values

A cumulative reward of a target system type is predicted by training a prediction model by performing an iteration for each time step. The iteration includes recursively updating a matrix by using the weighted difference between an eligibility trace of a current time step and an eligibility trace of a previous time step, recursively updating a vector by using a reward of a subsequent time step and the eligibility trace of the current time step, and recursively updating an eligibility trace of a subsequent time step by using a feature vector of the subsequent time step. Each feature vector is an encoded representation of a state of a training system of the target system type at a corresponding time step. The matrix and the vector are output as the prediction model for estimating the cumulative reward of a target time step of a target system of the target system type.

Cumulative reward predictor training

A computer-implemented method is provided for machine prediction. The method includes forming, by a hardware processor, a Convolutional Dynamic Boltzmann Machine (C-DyBM) by extending a non-convolutional DyBM with a convolutional operation. The method further includes generating, by the hardware processor using the convolution operation of the C-DyBM, a prediction of a future event at time t from a past patch of time-series of observations. The method also includes performing, by the hardware processor, a physical action responsive to the prediction of the future event at time t.

Convolutional dynamic Boltzmann Machine for temporal event sequence

An apparatus for implementing a computing system to predict preferences includes at least one processor device operatively coupled to a memory. The at least one processor device is configured to calculate a parameter relating to a density of a prior distribution at each sample of a set of samples associated with the prior distribution. The at least one parameter including a distance from each sample to at least one neighboring sample. The at least one processor device is further configured to estimate, for the plurality of samples, at least one differential entropy of at least one posterior distribution associated with at least one observation based on the parameter relating to the density of the prior distribution at each sample and the likelihood of observation for each sample. The estimation is performed without sampling the at least one posterior distribution to reduce consumption of resources of the computing system.

Computational estimation of a characteristic of a posterior distribution

Systems and methods for modelling time-series data includes testing a testing model with a plurality of hyper-forgetting rates to select a best performing hyper-forgetting rate. A model optimization is tested using the best performing hyper-forgetting rate with the testing model to test combinations of hyper-parameters to select a best performing combination of hyper-parameters. An error of the model is determined using the model optimization. Model parameters are recursively updated according to the least squares regression by determining a pseudo-inverse of a Hessian of the least squares regression at a current time stamp according to a projection of the time-series data at the current time stamp and the pseudo-inverse of the Hessian at a previous time-stamp to determine an optimum model parameter. A next step behavior of the time-series data is predicted using the optimum model parameter. The next step behavior is stored in a database for access by a user.

Behavior prediction with dynamic adaptation to environmental conditions

An information processing apparatus includes a history acquisition section configured to acquire history data including a history indicating that a plurality of selection subjects have selected selection objects; a learning processing section configured to allow a choice model to learn a preference of each selection subject for a feature and an environmental dependence of selection of each selection object in each selection environment using the history data, where the choice model uses a feature value possessed by each selection object, the preference of each selection subject for the feature, and the environmental dependence indicative of ease of selection of each selection object in each of a plurality of selection environments to calculate a selectability with which each of the plurality of selection subjects selects each selection object; and an output section configured to output results of learning by the learning processing section.

Predicting a consumer selection preference based on estimated preference and environmental dependence

Methods and systems for selecting and performing group actions include selecting parameters for an approximated action-value function, which determines a reward value associated with a particular group action taken from a particular state, using a determinant of a parameter matrix for the action-value function. A group action is selected using the approximated action-value function and the selected parameters. Agents are triggered to perform respective tasks in the group action.

Growing community of inventors

Tokyo, Japan

Takayuki Osogami