International Business Machines Corporation

Djallel Bouneffouf

Bei Chen

Yasaman Khazaeni

Mayank Agarwal

Sohini Upadhyay

Elizabeth Daly

Akihiro Kishimoto

Radu Marinescu

Oznur Alkan

Toshiyuki Hama

Hsiang Han Hsu

Martin Wistuba

Parikshit Ram

Mikhail Yurochkin

Ambrish Rwat

A novel formulation called the Context-Attentive Bandit with Observations (CABO) is described, where only a limited number of features can be accessed by the learner. The present invention is applicable to many problems including problems arising in clinical settings and dialog systems where it is not possible to reveal the whole feature set. The present invention adapts the standard contextual bandit algorithm known as Thompson Sampling with a novel algorithm, we call Context-Attentive Thompson Sampling with Observations (CATSO). Experimental results are included to demonstrate its effectiveness including a regret analysis and an empirical evaluation demonstrating advantages of the disclosed novel approach on several real-life datasets.

Feedback driven decision support in partially observable settings

A computer implemented method of modifying molecular structures constrained by a budget is provided. The computer implemented method includes receiving from a user a subset of molecules, where each molecule is represented as a generation path, and receiving from the user an allotted budget for modifying a selection of molecules from the subset of molecules. The computer implemented method further includes testing a first molecule, and reducing the allotted budget based on the resources expended to test the first molecule. The computer implemented method further includes testing a second molecule, and reducing the allotted budget based on the resources expended to test the second molecule. The computer implemented method further includes determining a remaining amount of the allotted budget, and testing additional molecules from the subset of molecules until the allotted budget is exhausted. The computer implemented method further includes presenting the tested molecules to the user.

Optimization of multiple molecules

An online machine learning model such as an autonomous agent predicts an action. A processor associated with, or running, the online machine learning model observes an environment for an interval of time for a real reward associated with the action. Responsive to determining that the real reward is not received within the interval of time, the processor determines based on a criterion whether to allocate an immediate reward received within the interval of time to the online machine learning model, where the immediate reward is an approximation of the real reward. Responsive to determining that the immediate reward is to be allocated, the processor allocates the immediate reward to the online machine learning model. The online machine learning model further learns or retrains itself based on the immediate reward.

Online machine learning with immediate rewards when real rewards are delayed

Systems, computer-implemented methods, and computer program products to facilitate automated generation of a machine learning pipeline based on a pipeline grammar are provided. According to an embodiment, a system can comprise a memory that stores computer executable components and a processor that executes the computer executable components stored in the memory. The computer executable components can comprise a pipeline structure generator component that generates a machine learning pipeline structure based on a pipeline grammar. The computer executable components can further comprise a pipeline optimizer component that selects one or more machine learning modules that achieve a defined objective to instantiate a machine learning pipeline based on the machine learning pipeline structure.

Automated generation of a machine learning pipeline

A feature vector characterizing a system to be analyzed via online partially rewarded machine learning is obtained. Based on the feature vector, a decision is made, via the machine learning, using an online policy. The system is observed for environmental feedback. In at least a first instance, wherein the observing indicates that the environmental feedback is available, the environmental feedback is obtained. In at least a second instance, wherein the observing indicates that the environmental feedback is missing, the environmental feedback is imputed via an online imputation method. the online policy is updated based on results of the obtained environmental feedback and the online imputation method. A decision is output based on the updated online policy.

Online partially rewarded learning

Growing community of inventors

Wappingers Falls, NY, United States of America

Djallel Bouneffouf