International Business Machines Corporation

Gerald James Tesauro

Jeffrey Owen Kephart

Hendrik F Hamann

Theodore Gerard Van Kessel

Jonathan Lenchner

Levente Klein

Siyuan Lu

Fernando Jimenez Marianno

Rajarshi Das

Kilian Quirin Weinberger

Youngdeok Hwang

Deepak Srinivas Turaga

David Michael Chess

Ian N Whalley

Alla Segal

Horst Cornelius Samulowitz

Tamir Klinger

Richard B Segal

James Edwin Hanson

Chandrasekhara K Reddy

Gregory Bret Sorkin

Ashish Sabharwal

Steven Richard White

Rajashi Das

Shawn Patrick Mullen

Mark N Wegman

Irina Rish

George Francis Ramsay, Iii

Neal Richard Marion

James Stanley Tesauro

Alina Beygelzimer

Janusz Marecki

William Edward Walsh

Nicholas K Jong

A parameter-based multi-model blending method and system are described. The method includes selecting a parameter of interest among parameters estimated by each of a set of individual models, running the set of individual models with a range of inputs to obtain a range of estimates of the parameters from each of the set of individual models, and identifying, for each of the set of individual models, critical parameters among the parameters estimated, the critical parameters exhibiting a specified correlation with an error in estimation of the parameter of interest. For each subspace of combinations of the critical parameters, obtaining a parameter-based blended model is based on blending the set of individual models in accordance with the subspace of the critical parameters, the subspace defining a sub-range for each of the critical parameters.

Parameter-dependent model-blending with multi-expert based machine learning and proxy sites

A system, method, and computer program product for automatically selecting from a plurality of analytic algorithms a best performing analytic algorithm to apply to a dataset is provided. The automatically selecting from the plurality of analytic algorithms the best performing analytic algorithm to apply to the dataset enables a training a plurality of analytic algorithms on a plurality of subsets of the dataset. Then, a corresponding prediction accuracy trend is estimated across the subsets for each of the plurality of analytic algorithms to produce a plurality of accuracy trends. Next, the best performing analytic algorithm is selected and outputted from the plurality of analytic algorithms based on the corresponding prediction accuracy trend with a highest value from the plurality of accuracy trends.

Automated intelligent data navigation and prediction tool

The present invention is a method and an apparatus for reward-based learning of policies for managing or controlling a system or plant. In one embodiment, a method for reward-based learning includes receiving a set of one or more exemplars, where at least two of the exemplars comprise a (state, action) pair for a system, and at least one of the exemplars includes an immediate reward responsive to a (state, action) pair. A distance metric and a distance-based function approximator estimating long-range expected value are then initialized, where the distance metric computes a distance between two (state, action) pairs, and the distance metric and function approximator are adjusted such that a Bellman error measure of the function approximator on the set of exemplars is minimized. A management policy is then derived based on the trained distance metric and function approximator.

Method and apparatus for improved reward-based learning using adaptive distance metrics

A method, system and computer program product for choosing actions in a state of a planning problem. The system simulates one or more sequences of actions, state transitions and rewards starting from the current state of the planning problem. During the simulation of performing a given action in a given state, a data record is maintained of observed contextual state information, and observed cumulative reward resulting from the action. The system performs a regression fit on the data records, enabling estimation of expected reward as a function of contextual state. The estimations of expected rewards are used to guide the choice of actions during the simulations. Upon completion of all simulations, the top-level action which obtained highest mean reward during the simulations is recommended to be executed in the current state of the planning problem.

Monte-Carlo planning using contextual information

Methods and systems are provided for autonomic control and optimization of computing systems. A plurality of component models for one or more components in an autonomic computing system are maintained in a system level database. These component models are obtained from a source external to the management server including the components associated with the models. Component models are added or removed from the database or updated as need. A system level management server in communication with the database utilizes the component models maintained in the system level database and generic component models as needed to compute an optimum state of the autonomic computing system. The autonomic computing system is managed in accordance with the computed optimum state.

Autonomic computing system with model transfer

A system, method and computer program product for planning actions in a repeated Stackelberg Game, played for a fixed number of rounds, where the payoffs or preferences of the follower are initially unknown to the leader, and a prior probability distribution over follower types is available. In repeated Bayesian Stackelberg games, the objective is to maximize the leader's cumulative expected payoff over the rounds of the game. The optimal plans in such games make intelligent tradeoffs between actions that reveal information regarding the unknown follower preferences, and actions that aim for high immediate payoff. The method solves for such optimal plans according to a Monte Carlo Tree Search method wherein simulation trials draw instances of followers from said prior probability distribution. Some embodiments additionally implement a method for pruning dominated leader strategies.

Optimal policy determination using repeated stackelberg games with unknown player preferences

In one embodiment, the present invention is a method for allocation of finite computational resources amongst multiple entities, wherein the method is structured to optimize the business value of an enterprise providing computational services. One embodiment of the inventive method involves establishing, for each entity, a service level utility indicative of how much business value is obtained for a given level of computational system performance. The service-level utility for each entity is transformed into a corresponding resource-level utility indicative of how much business value may be obtained for a given set or amount of resources allocated to the entity. The resource-level utilities for each entity are aggregated, and new resource allocations are determined and executed based upon the resource-level utility information. The invention is thereby capable of making rapid allocation decisions, according to time-varying need or value of the resources by each of the entities.

Method and apparatus for utility-based dynamic resource allocation in a distributed computing system

The present invention is a method and an apparatus for reward-based learning of management policies. In one embodiment, a method for reward-based learning includes receiving a set of one or more exemplars, where at least two of the exemplars comprise a (state, action) pair for a system, and at least one of the exemplars includes an immediate reward responsive to a (state, action) pair. A distance measure between pairs of exemplars is used to compute a Non-Linear Dimensionality Reduction (NLDR) mapping of (state, action) pairs into a lower-dimensional representation, thereby producing embedded exemplars, wherein one or more parameters of the NLDR are tuned to minimize a cross-validation Bellman error on a holdout set taken from the set of one or more exemplars. The mapping is then applied to the set of exemplars, and reward-based learning is applied to the embedded exemplars to obtain a learned management policy.

Method and apparatus for improved reward-based learning using nonlinear dimensionality reduction

In one embodiment, the present invention is a method for reward-based learning of improved systems management policies. One embodiment of the inventive method involves obtaining a decision-making entity and a reward mechanism. The decision-making entity manages a plurality of application environments supported by a data processing system, where each application environment operates on data input to the data processing system. The reward mechanism generates numerical measures of value responsive to actions performed in states of the application environments. The decision-making entity and the reward mechanism are applied to the application environments, and results achieved through this application are processed in accordance with reward-based learning to derive a policy. The reward mechanism and the policy are then applied to the application environments, and the results of this application are processed in accordance with reward-based learning to derive a new policy.

Method and apparatus for reward-based learning of improved policies for management of a plurality of application environments supported by a data processing system

Methods, systems, and products are provided for peer-to-peer computer software installation. Embodiments include receiving, by an observing install agent running on an observing host from a test install agent running on a test host, performance information describing the performance of software installed on the test host; determining, by the observing install agent, whether the performance information meets performance criteria for the observing host; and if the performance information meets the performance criteria for the observing host, installing the software on the observing host. In some embodiments, determining, by the observing install agent, whether the performance information meets performance criteria for the observing host is carried out by determining, whether the performance information meets performance criteria for the observing host in dependence upon a rule.

Autonomic peer-to-peer computer software installation

Active sample collaborative prediction method, system and program storage device are provided. A method in one aspect may include determining approximation X for matrix Y using collaborative prediction, said matrix Y being sparse initially and representing pairwise measurement values; selecting one or more unobserved entries from said matrix Y representing active samples using said approximation X and an active sample heuristic; obtaining values associated with said unobserved entries; inserting said values to said matrix Y; and repeating the steps of determining, selecting, obtaining and inserting until a predetermined condition is satisfied.

Active sampling collaborative prediction method for end-to-end performance prediction

Growing community of inventors

Croton-on-Hudson, NY, United States of America

Gerald James Tesauro