Fujitsu Corporation

Okinawa Institute of Science and Technology School Corporation

The University of Tokyo

Fujitsu Limited Kawasaki, Japan

Hidenao Iwane

Hitoshi Yanami

Hirokazu Anai

Tomotake Sasaki

Yoshihiro Okawa

Eiji Uchibe

Kenji Doya

Shinji Hara

Suguru Fujita

Tsuneo Nakata

Junichi Shigezumi

Junji Kaneko

A policy improvement method of improving a policy of reinforcement learning based on a state value function is performed by a computer. The method causes a computer to execute a process including: calculating an input to a control target based on the policy and a predetermined exploration method of exploring for an input to the control target in the reinforcement learning; and updating a parameter of the policy based on a result of applying the calculated input to the control target, using the input to the control target and a generalized inverse matrix regarding a state of the control target.

Policy improvement method, policy improvement program storage medium, and policy improvement device

A non-transitory, computer-readable recording medium stores therein a reinforcement learning program that uses a value function and causes a computer to execute a process comprising: estimating first coefficients of the value function represented in a quadratic form of inputs at times in the past than a present time and outputs at the present time and the times in the past, the first coefficients being estimated based on inputs at the times in the past, the outputs at the present time and the times in the past, and costs or rewards that corresponds to the inputs at the times in the past; and determining second coefficients that defines a control law, based on the value function that uses the estimated first coefficients and determining input values at times after estimation of the first coefficients.

Recording medium, reinforcement learning method, and reinforcement learning apparatus

A computer-implemented reinforcement learning method includes determining, based on a target probability of satisfaction of a constraint condition related to a state of a control object and a specific time within which a controller causes the state of the control object not satisfying the constraint condition to be the state of the control object satisfying the constraint condition, a parameter of a reinforcement learner that causes, in a specific probability, the state of the control object to satisfy the constraint condition at a first timing following a second timing at which the state of control object satisfies the constraint condition; and determining a control input to the control object by either the reinforcement learner or the controller, based on whether the state of the control object satisfies the constraint condition at a specific timing.

Reinforcement learning method and reinforcement learning system

A non-transitory, computer-readable recording medium stores a program of reinforcement learning by a state-value function. The program causes a computer to execute a process including calculating a temporal difference (TD) error based on an estimated state-value function, the TD error being calculated by giving a perturbation to each component of a feedback coefficient matrix that provides a policy; calculating based on the TD error and the perturbation, an estimated gradient function matrix acquired by estimating a gradient function matrix of the state-value function with respect to the feedback coefficient matrix for a state of a controlled object, when state variation of the controlled object in the reinforcement learning is described by a linear difference equation and an immediate cost or an immediate reward of the controlled object is described in a quadratic form of the state and an input; and updating the feedback coefficient matrix using the estimated gradient function matrix.

Apparatus, method and recording medium for controlling system using temporal difference error

A reinforcement learning method executed by a computer includes calculating a degree of risk for a state of a controlled object at a current time point with respect to a constraint condition related to the state of the controlled object, the degree of risk being calculated based on a predicted value of the state of the controlled object at a future time point, the predicted value being obtained from model information defining a relationship between the state of the controlled object and a control input to the controlled object; and determining the control input to the controlled object at the current time point, from a range defined according to the calculated degree of risk so that the range becomes narrower as the calculated degree of risk increases.

Reinforcement learning method, recording medium, and reinforcement learning system

A reinforcement learning device includes a processor that determines a first action on a control target by using a basic controller that defines an action on the control target depending on a state of the control target. The processor performs a first reinforcement learning within a first action range around the first action in order to acquire a first policy for determining an action on the control target depending on a state of the control target. The first action range is smaller than a limit action range for the control target. The processor determines a second action on the control target by using the first policy. The processor updates the first policy to a second policy by performing a second reinforcement learning within a second action range around the second action. The second action range is smaller than the limit action range.

Reinforcement learning method and device

A control server according to an embodiment sorts a plurality of notebook PCs into a plurality of groups so that the total value of the remaining energy is a value similar to the total value of the remaining energy of the rechargeable batteries of a plurality of notebook PCs included in a different group. The control server according to the embodiment performs local search individually on the sorted groups, and generates a control plan for the individual notebook PCs.

Control method, control server, and computer-readable recording medium

When model expressions of objective functions are generated at vertexes of a quadrilateral on a plane concerning P and N channels of transistors in SRAM, the initial number of times of simulation is allocated to each objective function at each designated vertex according to weight values set based on relationships presumed among the objective functions at each designated vertex. For each objective function at each designated vertex, first simulation is executed the allocated number of times. Furthermore, a model expression is generated from the first simulation result, and an evaluation indicator of an approximation accuracy of the model expression is calculated. Then, for each model expression, it is determined whether the corresponding model expression has influence on the yield, and based on the evaluation indicator of the corresponding model expression and presence or absence of the influence, it is determined whether additional simulation is required for the corresponding objective function.

Model expression generation method and apparatus

This method includes: generating a constraint equation from data of an approximate expression of a cost function representing a relationship between a plurality of design parameters and a cost, data of a route in a cost space and data of a search range in a design parameter space; obtaining a logical expression of a solution for the constraint equation from a quantifier elimination processing unit that carries out a processing according to a quantifier elimination method; substituting coordinates of each of a plurality of points within the search range in the design parameter space into the logical expression of the solution to determine, for each of the plurality of points, true or false of the logical expression of the solution; and displaying the design parameter space in which a display object including a first point for which true is determined is disposed at the first point.

Display processing technique of design parameter space

A design support apparatus for determining a plurality of objective functions for modeling an object having a plurality of elements, each of the elements providing variable geometrical parameters, the design support apparatus includes a memory for storing the variable geometrical parameters and a processor for executing a process including: determining boundary information associated with specified geometrical parameters of the elements which indicate a state of contact between the elements, dividing the variable geometrical parameters into a plurality of groups on the basis of the boundary information, and determining the plurality of objective functions for each of the groups by using the variable geometrical parameters.

Support apparatus using objective function

A design support apparatus includes: a logical expression substitution unit to substitute a part of the logical expression, which includes a function expression of the design variables and a quantifier attached to the design variable, with a substitution variable; a quantifier elimination unit to generate a relational expression including the substitution variable and design variables without the quantifier by eliminating the design variable to which the quantifier is attached from the logical expression; a sampling point generation unit to generate a plurality of sampling points corresponding to the design variables and the substitution variable included in the relational expression; a possible range computation unit to compute, for each of the sampling points, a possible range that the relational expression may take, by calculating values of remaining design variables included in the relational expression based on the relational expression; and a possible range display unit to display the possible range.

Support apparatus and method for simplifying design parameters during a simulation process

A first model expression having a first order and representing a relationship between evaluation indicators and the design parameters and a second model expression having a second order higher than the first order and representing a relationship between them are generated. Then, according to a quantifier elimination method, values of the design parameters, which realize an optimum solution of the first model expression, are calculated. And, a design parameter whose value is identical to an upper limit value or a lower limit value is identified, and the second model expression is transformed by substituting the upper limit value or the lower limit value, as a value of the identified design parameter, for the second model expression. Finally, according to the quantifier elimination method, values of the design parameters in the transformed second model expression, which realize an optimum solution of the transformed second model expression, is calculated.

Optimization processing method and apparatus

An embodiment relates to a multi-objective optimization design supporting technique to reduces the computational complexity of QE/CAD. When the input logical expression generated by a logical-expression-with-qualifier generation unit is satisfied in regard to the sample point included in a certain piece of cell information for each value of the same design parameter, a first cell processing unit does not evaluate the input logical expression on the cell information including other sample points having a value equal to or smaller than the value of a predetermined design variable (for example, a design variable indicating a yield) corresponding to the sample point above, but selects it as the cell information for an output of a logical expression without a qualifier.

Growing community of inventors

Kawasaki, Japan

Hidenao Iwane