International Business Machines Corporation

Asim Munawar

Ryuki Tachibana

Tadanobu Inoue

Subhajit Chaudhury

Daiki Kimura

Michiaki Tatsubori

Phongtharin Vinayavekhin

Giovanni De De Magistris

Guillaume Jean Victor Marie Le Moing

Don Joven R Agravante

Jayakorn Vongkulbhisal

Don Joven Ravoy Agravante

Clement D M Creusot

Shuichi Shimizu

Tatsuya Ishikawa

Sakyasingha Dasgupta

Tu-Hoa Pham

Masataro Asai

Tristan Matthieu Stampfler

Roland Everett Fall, Iii

Jason Marc Plawinski

According to an aspect of the present invention, a computer-implemented method is provided for reinforcement learning. The method includes reading, by a processor device, an action manifold which is described as a n-polytope, at least one physical action limit, and at least one safety constraint. The method further includes updating, by the processor device, the action manifold based on the at least one physical action limit and the at least one safety constraint. The method also includes performing, by the processor device, the reinforcement learning by selecting a constrained action from among a set of constrained actions in the action manifold.

Safe and fast exploration for reinforcement learning using constrained action manifolds

Aspects of the present invention disclose a method for a distance-based vector classification in anomaly detection. The method includes one or more processors identifying one or more audio communications from a first user to a second user, wherein the one or more audio communications is transmitted utilizing a first computing device. The method further includes determining an objective of the first user based at least in part on the audio communication of the first user. The method further includes determining a set of conditions corresponding to the one or more audio communications and the objective, wherein the set of conditions indicate a vulnerability of personal data of the first user. The method further includes prohibiting the first computing device from transmitting audio data that includes the personal data of the first user.

Anomaly detection by ranking from algorithm

A computer-implemented method is provided for training a multi-source sound localization model using labeled simulation data and unlabeled real data. The method includes inputting the labeled simulation data and the unlabeled real data respectively into a multi-source sound localization model of a neural network to obtain a localization heatmap from an output layer of the multi-source sound localization model for each of the labeled simulation data and the unlabeled real data. The method further includes inputting the localization heatmap for each of the labeled simulation data and the unlabeled real data into an output discriminator. The method also includes training the output discriminator so that the output discriminator assigns a domain class label to distinguish simulation data from real data. The method additionally includes training, by a hardware process, the multi-source sound localization model by a first adversarial loss for the output discriminator with an original localization model loss.

Sim-to-real learning of 2D multiple sound source localization

In an approach, a processor trains a model, via a reinforcement learning process, to produce a first action function for relating states of a natural language based response environment to actions applicable to the natural language based response environment. A processor retrains the model, via the reinforcement learning process, to produce a second action function, including iterations of: applying the first action function to a current state representation of the natural language based response environment to obtain a ground-truth action representation, emphasizing a word of the current state representation based on relevancy to the ground-truth action representation to obtain a modified state representation, applying a model to the modified state representation to obtain an untrained action representation, and submitting the untrained action representation to a natural language based response environment to obtain a subsequent state representation, where the subsequent state representation becomes the current state representation for a subsequent iteration.

Text-based response environment action selection

A method is provided for detecting a higher-level action from one or more trajectories of real states. The trajectories are based on an experts' action demonstration. The method trains predictors to predict future states. Each predictor has a different duration of the higher-level action to be detected. The method predicts, using the predictors, the future states using past ones of the real states in the one or more trajectories as inputs for the predictors. The method determines if a match exists between any of the future states relative to a real future state with a corresponding same duration from the one or more trajectories. The method outputs a pair that includes the matching one of the future states as a prediction input and the real future state with the corresponding same duration from the one or more trajectories as the higher-level action corresponding thereto, responsive to the match existing.

Discovering higher-level actions from expert's action demonstration

Methods and systems for localizing a sound source include determining a spatial transformation between a position of a reference microphone array and a position of a displaced microphone array. A sound is measured at the reference microphone array and at the displaced microphone array. A source of the sound is localized using a neural network that includes respective paths for the reference microphone array and the displaced microphone array. The neural network further includes a transformation layer that represents the spatial transformation.

Two-dimensional sound localization with transformation layer

A computer-implemented method is provided for modified Lexicographic Reinforcement Learning. The computer implemented method includes obtaining, by a hardware processor, a sequence of tasks. Each of the tasks corresponds to, and has a one-to-one correspondence with, a respective award from among set of rewards. The method further includes performing, by the hardware processor for each of the tasks, reinforcement learning and deep learning for both of (i) one or more policies and (ii) one or more value functions, with a plurality of sets of samples. A plurality of solutions in a form of the one or more policies and the one or more value functions are parametrized by a single neural network with a selector which selects an input of the single neural network from among the plurality of sets of samples.

Lexicographic deep reinforcement learning using state constraints and conditional policies

In an approach for propagating labels of objects in an image, a processor receives the image. A processor performs a normalization of the image. A processor runs the image through a pre-trained object detector. A processor receives a set of detected objects from the pre-trained object detector. A processor determines a width dimension and a height dimension of a bounding box for each detected object of the set of detected objects. A processor propagates a label for each instance of each detected object in the image with the respective bounding box using prior geometric knowledge of bounding box placement. A processor inverses the normalization of the labeled image. A processor outputs the labeled image.

Accelerated object labeling using prior geometric knowledge

A computer-implemented method is provided for training a classification model. The method includes preparing, by a processor, positive and negative class data. The method further includes iteratively training the classification model, by the processor, using the positive class data and the negative class data such that the positive class data is reconstructed and the negative class data is prevented from being constructed, by the classification model. In response to a selection of a non-integer value as a number of negative learning iterations to be performed to train the classification model, a particular set of the negative class data that is reconstructed best by the classification model from among all of the negative class data is selected to be used for negative learning by the classification model. The training based on the positive class data is performed once before the negative learning iterations and once after each negative learning iteration.

Adaptive control of negative learning for limited reconstruction capability auto encoder

Edge-Loss-based image construction is enabled by a method including generating a reconstructed image from a first edge image with a generator, extracting a second edge image from the reconstructed image with an edge extractor, smoothing the first edge image and the second edge image, discriminating between the reconstructed image and an original image corresponding to the first edge image with a discriminator to obtain an adversarial loss, and training the generator by using an edge loss and the adversarial loss, the edge loss being calculated from the smoothed first edge image and the smoothed second edge image.

Image reconstruction based on edge loss

A computer-implemented method, computer program product, and system are provided for learning mapping information between different modalities of data. The method includes mapping, by a processor, high-dimensional modalities of data into a low-dimensional manifold to obtain therefor respective low-dimensional embeddings through at least a part of a first network. The method further includes projecting, by the processor, each of the respective low-dimensional embeddings to a common latent space to obtain therefor a respective one of separate latent space distributions in the common latent space through at least a part of a second network. The method also includes optimizing, by the processor, parameters of each of the networks by minimizing a distance between the separate latent space distributions in the common latent space using a variational lower bound. The method additionally includes outputting, by the processor, the parameters as the mapping information.

Generating cross-domain data using variational mapping between embedding spaces

A computer-implemented method is provided for multi-source sound localization. The method includes extracting, by a hardware processor, spectral features from respective pluralities of microphones comprised in each of two or more microphone arrays. The method further includes forming, by the hardware processor, respective sets of pairs of the spectral features from the respective pluralities of microphones within each of the two or more microphone arrays by rearranging and duplicating the spectral features from the respective pluralities of microphones included in each of the two or more microphone arrays. The method also includes inputting, by the hardware processor, the respective sets of pairs of the spectral features into a neural network to encode the spectral features into deep features and decode the deep features to output from the neural network at least one location representation of one or more sound sources.

Feature processing for multi-array sound applications with deep learning and limited data

A computer-implemented method executed by a robotic system for performing a positional search process in an assembly task is presented. The method includes decomposing, by the robotic system, a perturbation motion into a plurality of actions, the perturbation motion being a motion for an assembly position searched by the robotic system, each action of the plurality of actions related to a specific direction. The method further includes performing reinforcement learning by selecting an action among decomposed actions and assembly movement actions at each step of the positional search process based on corresponding force-torque data received from at least one sensor associated with the robotic system. The method also includes outputting a best action at each step for completion of the assembly task as a result of the reinforcement learning.

Decomposed perturbation approach using memory based learning for compliant assembly tasks

A method for learning a classification model by a computer system is disclosed. One or more positive class data and one or more negative class data are prepared. The classification model is trained based on the positive class data to adjust one or more parameters of the classification model so that the positive class data is reconstructed by the classification model. The classification model is trained based on the negative class data to adjust the one or more parameters so that the negative class data is prevented from being reconstructed by the classification model. For the negative class data, changes in the one or more parameters with gradient of an objective function may be calculated using an unsupervised learning algorithm. The one or more parameters may be updated based on the changes in an opposite manner to the training based on the positive class.

Growing community of inventors

Ichikawa, Japan

Asim Munawar