Honeywell G.m.b.h.

Honeywell International Inc.

Philip D Loewen

Johan Backstrom

Michael Forbes

Guy A Dumont

Qiugang Lu

R Bhushan Gopaluni

Gregory Edward Stewart

Bhushan Gopaluni

Nathan Lawrence

Bhusban Gopaluni

Lee D Rippon

A method and system for reinforcement learning can include an actor-critic framework comprising an actor and a critic, the actor comprising an actor network and the critic comprising a critic network; and a controller comprising a neural network embedded in the actor-critic framework and which can be tuned according to reinforcement learning based tuning including anti-windup tuning.

Method and system for directly tuning PID parameters using a simplified actor-critic approach to reinforcement learning

A method includes obtaining operating data associated with operation of a cross-directional industrial process controlled by at least one model-based process controller. The method also includes, during a training period, performing closed-loop model identification with a first portion of the operating data to identify multiple sets of first spatial and temporal models. The method further includes identifying clusters associated with parameter values of the first spatial and temporal models. The method also includes, during a testing period, performing closed-loop model identification with a second portion of the operating data to identify second spatial and temporal models. The method further includes determining whether at least one parameter value of at least one of the second spatial and temporal models falls outside at least one of the clusters. In addition, the method includes, in response to such a determination, detecting that a mismatch exists between actual and modeled behaviors of the industrial process.

Model-plant mismatch detection with support vector machine for cross-directional process behavior monitoring

A method and system for reinforcement learning can involve applying a finite-difference approach to a controller, and tuning the controller in response to applying the finite-difference approach by taking a state as an entirety of a closed-loop step response. The disclosed finite-different approach is based on a random search to tuning the controller, which operates on the entire closed-loop step-response of the system and iteratively improves the gains towards a desired closed-loop response. This allows for prescribing stability requirement into the reward function without any modeling procedures.

Application of simple random search approach for reinforcement learning to controller tuning parameters

A method includes repeatedly identifying one or more values for one or more model parameters of at least one model associated with a process. The one or more values for the one or more model parameters are identified using data associated with the process. The method also includes clustering the values of the one or more model parameters into one or more clusters. The method further includes identifying one or more additional values for the one or more model parameters using additional data associated with the process. In addition, the method includes detecting a mismatch between the at least one model and the process in response to determining that at least some of the one or more additional values fall outside of the one or more clusters. The values could be clustered using a support vector machine.

Model-plant mismatch detection using model parameter data clustering for paper machines or other systems

A method includes obtaining closed-loop data associated with operation of an industrial process controller, where the industrial process controller is configured to control at least part of an industrial process using at least one model. The method also includes generating at least one noise model associated with the industrial process controller using at least some of the closed-loop data. The method further includes filtering the closed-loop data based on the at least one noise model. In addition, the method includes generating one or more model parameters for the industrial process controller using the filtered closed-loop data.

Closed-loop model parameter identification techniques for industrial model-based process controllers

Sheetmaking cross-directional (CD) control requires a sophisticated model-based controller whose operation requires an accurate model of process behavior, but due to the complexity of the process, identifying these process models is challenging. Current techniques rely on open-loop process experimentation. Using non-causal scalar transfer functions for the steady-state CD process model and controller model avoid the problem of large dimensions associated with the CD process. These non-causal transfer functions can be represented by causal transfer functions that are equivalent to the non-causal ones in the sense of the output spectrum. A closed-loop optimal input design framework is proposed based on these causal equivalent models. CD actuators have responses in both sides along the cross direction which can be viewed as a non-causal behavior. Techniques to perform the non-causal modeling are demonstrated and developed in a closed-loop optimal input design framework based on non-causal modeling of the closed-loop CD process.

Growing community of inventors

Vancouver, Canada

Philip D Loewen