International Business Machines Corporation

Alexey Tsitkin

Segev Eliezer Wasserkrug

Alexander Zadorojniy

Sergey Zeltyn

Michael Sambur

A method for determining a variable near-optimal policy for a problem formulated as Markov Decision Process, the problem comprising at least one limited action entry, the limited action entry being an entry of an action of a finite set of actions limited in the number of times its value may be changed, the method comprising using at least one hardware processor for: receiving data elements with respect to the problem, the data elements comprising: (a) a finite set of states, (b) the finite set of actions, (c) a transition probabilities matrix determining transition probabilities between states of the finite set of states, once actions of the set of actions are performed; (d) an immediate cost function, wherein the value of the immediate cost function is determined for a pair of a state of the finite set of states and an action of the finite set of actions, and (e) a discount factor; updating one or more data elements of the received data elements relating to the at least one limited action entry, wherein the one or more data elements are selected from the group consisting of: the transition probabilities matrix, the immediate cost function and the discount factor, and wherein the updating is triggered by a change of a value of a limited action entry of the at least one limited action entry; and following the updating of the one or more data elements, calculating a current near-optimal policy for the problem based on the updated one or more data elements.

Determining variable optimal policy for an MDP paradigm

A computerized method for scalable management optimization of pressurized water distribution networks, comprises receiving a network model representing a pressurized water distribution physical network having water flow variables. The network model is simulated by solving non-linear mathematical equations representing the behavior of the water flow variables. One or more result sets are fed to the non-linear mathematical equations. The non-linear mathematical equations are linearized. Network model optimization requirements are received from a user. The linearized mathematical equations are optimized according to the received network model optimization requirements. A local search starting from the at least one optimized solution is performed using the non-linearized mathematical equations, thereby generating a solution.

Growing community of inventors

Petach Tikva, Israel

Alexey Tsitkin