启发式动态规划在污水处理过程控制中的应用

针对溶解氧及硝态氮浓度的跟踪控制问题,提出了一种基于回声状态网络的启发式动态规划控制方法,该方法首先对当前策略进行评价,然后根据评价结果对当前策略进行调整,这个过程交替进行,直至发现最优的控制策略.评价函数及控制策略的逼近均采用回声状态网络实现.为保证控制器的可用性,对控制器学习过程的参数选择范围进行了分析.污水处理过程的控制实验表明,该方法能够显著提高系统控制的平稳性及控制精度.     

Optimal trajectory transfer in the presence of large controlled gain variations

A common problem encountered in the design of an optimal linear time invariant trajectory transfer system is the maintenance of the optimal control policy in the presence of large parameter variations. In this work, the maintenance of the control policy in systems having uncertain actuator (or controller) gains is developed. Necessary and sufficient conditions are derived using linear programming techniques. The resulting analysis computes the maximal dynamic range over which a nominally optimal control policy, less a scale factor, is algebraically invariant. The scaling of the original unperturbed control policy is accomplished through the application of a derived constant coefficient linear transform. The computed maximal dynamic range of admissible parameter variations can be used as an acceptance or rejection criteria for mass produced optimized controllers. Finally, a simple example is presented which demonstrates the mechanics of the developed methodology.     

Optimal and suboptimal control policies for a solar collector system

An optimal control problem to maximize the net energy gathered by a flat-plate solar collector system by controlling the collector fluid flow rate is investigated. The problem is formulated in terms of a distributed parameter system and solved using the method of characteristics. It is shown that if the pump of the collector loop is such that its pumping power is greater than a linear function of the fluid velocity, then the optimal control policy is one in which the fluid flow is instantly switched between zero and maximum rates. Necessary conditions that determine the optimal switching times are derived. Because the resultant switching function of the optimal policy is shown to be decomposable into two parts, one that depends on the state of the system and another that requires a priori knowledge of the solar intensity over the entire period of operation, a suboptimal control policy that can be implemented by an on/off feedback controller with hysteresis is proposed. When this suboptimal policy is compared with the optimal policy, it is shown that on a clear day with sufficient solar insolation to dictate a two-switch optimal policy, the two policies are identical. Under other weather conditions, the feedback suboptimal controller will keep the pump off for a slightly shorter period of time than the time dictated by the optimal control.     

Enhanced design of cascade control systems for unstable processes with time delay

In this paper, optimal H₂ internal model controller (IMC) is designed for control of unstable cascade processes with time delays. The proposed control structure consists of two controllers in which inner loop controller (secondary controller) is designed using IMC principles. The primary controller (master controller) is designed as a proportional-integral-derivative (PID) in series with a lead-lag filter based on IMC scheme using optimal H₂ minimisation. Selection of tuning parameter is important in any IMC based design and in the present work, maximum sensitivity is used for systematic selection of the primary loop tuning parameter. Simulation studies have been carried out on various unstable cascade processes. The present method provides significant improvement when compared to the recently reported methods in the literature particularly for disturbance rejection. The present method also provides robust closed loop performances for large uncertainties in the process parameters. Quantitative comparison has been carried out by considering integral of absolute error (IAE) and total variation (TV) as performance indices.     

An asymptotically optimal learning controller for finite Markov chains with unknown transition probabilities

A learning controller is presented for a Markovian decision problem in which the transition probabilities are unknown. This controller, which is designed to be asymptotically optimal with consideration of a conflict between estimation and control, uses a performance criterion incorporating a tradeoff between them explicitly for determination of a control policy. It is shown that this controller achieves asymptotic optimality in the sense that the relative frequency of applying the optimal policy converges to unity.     

Run-to-run control methods based on the DHOBE algorithm

Since process models are typically not known exactly in real problems, it is important to estimate the process parameters before one applies the optimal control to a process. In this paper, the Dasgupta-Huang optimal bounding ellipsoid (DHOBE) algorithm is employed to estimate process parameters in semiconductor process run-to-run (RtR) control. At each iteration, the DHOBE algorithm returns an outer bounding ellipsoid of the likely process parameter set. If the vector center of the ellipsoid is taken as the estimate of the process parameter vector, then a model-reference controller results; if the vector within the ellipsoid that produces the worst expected cost is taken as the process parameter estimate, then a worst-case controller results. These two methods are compared with other RtR control schemes: the exponentially weighted moving average (EWMA) method and the optimizing adaptive quality controller (OAQC). Simulation results show that the performance of the model-reference RtR controller based on the DHOBE algorithm is comparable to or better than that of the other two RtR controllers in some specific examples of semiconductor processes.     

An Improved Parameterized Controller Reduction Technique via New Frequency Weighted Model Reduction Formulation

In this paper, an improved parameterized controller reduction technique via a new frequency weighted model reduction formulation is developed for the feedback control of MIMO discrete time systems particularly for non‐unity feedback control system configurations which have the controller located in the feedback path. New frequency weights which are a function of a free parameter matrix are derived based on a set of equivalent block diagrams and this leads to a generalized double sided frequency weighted model reduction formulation. Solving this generalized double sided frequency weighted model reduction problem for various values of the free parameter results in obtaining controllers which correspond to each value of the free parameter. It is shown that the proposed formulation has a useful characteristic such that selecting a controller which corresponds to a large value of the free parameter results in obtaining an optimal reduced order controller and using this optimal reduced order controller in a closed loop system results in significant reduction in the infinity norm of the approximation error between the original closed loop system and the closed loop system which uses an optimal reduced order controller (when compared to existing frequency weighted model reduction methods).     

基于粒子群优化算法的PID控制器参数整定

PID控制器的性能完全依赖于其参数的整定和优化,但参数的整定及在线自适应调整对常规的PID控制器是难以解决的问题。根据粒子群算法具有对整个参数空间进行高效并行搜索的特点,提出了一种基于粒子群优化算法整定PID控制器参数的设计方法,并定义了一种新的性能指标函数来评价PID控制器的性能。现以二阶的船舶控制装置为研究对象,运用粒子群优化方法对PID控制器参数进行了寻优研究。仿真结果表明,该方法比一般PID参数整定方法具有更好的控制性能指标,有着一定的工程应用价值。     

Discrete-time jump LQG problem

The stochastic optimal control problem is considered for a class of noisy discrete-time linear systems with markovian jumping parameters and quadratic cost. It is assumed that the controller has complete access to the jumping parameters. Using a dynamic programming technique, an optimal control policy is derived, together with an explicit expression for the optimal cost. Besides the intrinsic theoretical interest in its own right, the results derived here may have an important bearing on the study of the partially observable situation (no access to the jumping parameter), inter alia.     

自由漂浮空间机器人末端轨迹优化自适应控制

惯性参数不确定情况下的自由漂浮空间机器人(FFSR)轨迹跟踪控制是当前FFSR自主控制研究的重点与难点之一.针对该问题,提出一种FFSR末端轨迹优化自适应跟踪控制方法.该方法首先基于离散状态依赖黎卡提方程(DSDRE),设计两级DSDRE优化跟踪控制器,然后在控制器输出基础上,通过求解有约束条件下的非线性优化问题实现FFSR惯性参数的辨识,进而根据辨识结果调整控制器相关参数,实现FFSR末端轨迹的优化自适应跟踪控制.最后,采用平面两连杆FFSR模型进行仿真,验证了所提出方法的有效性.     

基于参数探索的期望最大化策略搜索

针对随机探索易于导致梯度估计方差过大的问题,提出一种基于参数探索的期望最大化(Expectation-maximization,EM)策略搜索方法.首先,将策略定义为控制器参数的一个概率分布.然后,根据定义的概率分布直接在控制器参数空间进行多次采样以收集样本.在每一幕样本的收集过程中,由于选择的动作均是确定的,因此可以减小采样带来的方差,从而减小梯度估计方差.最后,基于收集到的样本,通过最大化期望回报函数的下界来迭代地更新策略参数.为减少采样耗时和降低采样成本,此处利用重要采样技术以重复使用策略更新过程中收集的样本.两个连续空间控制问题的仿真结果表明,与基于动作随机探索的策略搜索强化学习方法相比,本文所提方法不仅学到的策略最优,而且加快了算法收敛速度,具有较好的学习性能.     

Reinforcement learning of dynamic behavior by using recurrent neural networks

Reinforcement learning is a learning scheme for finding the optimal policy to control a system, based on a scalar signal representing a reward or a punishment. If the observation of the system by the controller is sufficiently rich to represent the internal state of the system, the controller can achieve the optimal policy simply by learning reactive behavior. However, if the state of the controlled system cannot be assessed completely using current sensory observations, the controller must learn a dynamic behavior to achieve the optimal policy. In this paper, we propose a dynamic controller scheme which utilizes memory to uncover hidden states by using information about past system outputs, and makes control decisions using memory. This scheme integrates Q-learning, as proposed by Watkins, and recurrent neural networks of several types. It performs favorably in simulations which involve a task with hidden states. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

ILC strategy for progress improvement of economic performance in industrial model predictive control systems

A novel approach to progress improvement of the economic performance in model predictive control (MPC) systems is developed. The conventional LQG based economic performance design provides an estimation which cannot be done by the controller while the proposed approach can develop the design performance achievable by the controller. Its optimal performance is achieved by solving economic performance design (EPD) problem and optimizing the MPC performance iteratively in contrast to the original EPD which has nonlinear LQG curve relationship. Based on the current operating data from MPC, EPD is transformed into a linear programming problem. With the iterative learning control (ILC) strategy, EPD is solved at each trial to update the tuning parameter and the designed condition; then MPC is conducted in the condition guided by EPD. The ILC strategy is proposed to adjust the tuning parameter based on the sensitivity analysis. The convergence of EPD by the proposed ILC has also been proved. The strategy can be applied to industry processes to keep enhancing the performance and to obtain the achievable optimal EPD. The performance of the proposed method is illustrated via an SISO numerical system as well as an MIMO industry process.     

Identification and Control of Multivariable Stochastic Feedback Systems

Abstract

This paper considers the problem of simultaneous identification and control of stochastic processes characterized by linear dynamic models with unknown systems parameter coefficients. Stochastic approximation is used to derive consistent identification algorithms for the case in which arbitrary feedback controls are present. These identification methods can also be used for determining the order of the system, if the latter is unknown, as well as the exact canonical structure for the multivariable case.

An approximation to the optimal control solution is obtained by explicitly separating the functions of identification and control, and asymptotic convergence to a stochastic optimal controller is attained without on-line structural modification.     

模态跳变概率可控的Markov跳变线性系统的优化

研究模态跳变概率可控的Markov跣变线性二次模型的最优控制问题,考虑两类模态跳变控制策略:开环模态控制和闭环模态控制,应用策略迭代和性能势的概念,给出了最优的闭环模态控制优于最优的开环模态控制的充分条件,以指导最优控制器的设计,在已知最优的开环模态控制策略的基础上,应用策略迭代给出了构造闭环模态控制策略的方法,以进一步改善系统的性能.     

多电机驱动系统的保性能鲁棒Funnel控制

本文针对含参数不确定性的多电机驱动系统,提出一种基于最优保性能鲁棒的Funnel控制方法实现系统的规定跟踪性能.该控制方法通过构造Funnel函数对误差系统进行变换,并设计自适应反步控制器保证变换后系统的稳定性即可使跟踪误差的瞬态和稳态响应均被限制在给定的Funnel边界内.然而由于系统中存在的参数不确定性会影响系统的规定控制性能,本文在Funnel控制基础上又设计了最优保性能鲁棒控制器.它是通过将参数不确定性系统的保性能鲁棒控制问题转化为标称系统的最优控制问题,并求解新的黎卡提方程而得到的.因此所设计的控制器不但消除了参数不确定性对系统的影响并且能够使系统的性能指标达到一确定的上界.最后,对四电机驱动系统进行了仿真和实验验证,说明所提出控制方法的有效性.     

LQG control of networked control systems with access constraints and delays

We explore the LQG control of a networked control system (NCS) in which a linear plant is controlled remotely over a network or other shared communication medium. The medium provides a limited number of simultaneous connections, so that only a subset of the plant's sensors and actuators may communicate with the controller at any one time, subject to known transmission delays. Instead of insisting on jointly optimal control and medium access policies, we reduce the infinity of possible access sequences down to those which preserve the stabilisability and detectability of the underlying plant, and are periodic. Our choice of communication and NCS model effect a kind of ‘decoupling’ of the LQG problem, in the sense that the medium access policy can be selected independently of the controller. This guarantees the existence of a stabilising LQG controller which is optimal for the communication policy of choice, and which is then combined with a delay compensator. We include simulations that illustrate our approach.     

Stabilization of control loops consisting of FOPDT process and parameter-dependent PID controller

In this paper, problem of stability analysis of the control loops consisting of first-order plus dead time (FOPDT) processes and proportional-integrative-derivative (PID) controllers is studied, where the controller coefficients are functions of one or more independent parameters. An effective procedure is presented to determine a stability region in the independent parameters space. This method does not require complex numerical calculations such as solving nonlinear equations. It is based on usage of a two-valued indicator function and by using that, a stability region is easily determined. In order to clarify that, why the stability region needs to be specified in the “independent parameters space” an optimal method is given to design the PID controller for the FOPDT processes, as an instance. In this optimal method the controller coefficients are obtained as the functions of a free parameter, where this parameter needs to be chosen by the designer such that it should be near to the maximum operating frequency of the system, besides on the other hand the closed-loop system to be stable. In the end, two illustrative examples are given in order to show the usefulness and effectiveness of the proposed method, and to compare the obtained stability regions with the whole stability regions.     

Analytical decoupling control strategy using a unity feedback control structure for MIMO processes with time delays

An analytical decoupling control method is proposed for multiple-input–multiple-output (MIMO) processes with multiple time delays. The desired diagonal system transfer matrix is proposed first in terms of the H₂ optimal performance specification, resulting in the ideal desired decoupling controller matrix derived within the framework of a unity feedback control structure. It is demonstrated that dead-time compensators must be enclosed in the decoupling controller matrix to realize absolute decoupling for MIMO processes with multiple time delays. To alleviate the difficulties associated with the implementation, the ideal desired decoupling controller matrix is transformed into a practical form using an analytical approximation approach. Correspondingly, the stability of the resultant control system is assessed, together with its robust stability in the presence of process uncertainties. An on-line tuning rule for the single adjustable parameter of each column controllers in the decoupling controller matrix is given to cope with the process unmodeled dynamics. Finally, illustrative examples are given to show the superiority of the proposed method over the recently improved decoupling control methods.     

Optimal tracking control of a class of nonlinear discrete-time switched systems using adaptive dynamic programming

In this paper, an infinite-horizon optimal tracking control scheme is proposed for a class of nonlinear discrete-time switched systems. First, via system transformation, the optimal tracking problem is converted into designing an optimal regulator for the tracking error dynamics. And then, with convergence analysis in terms of value function and control policy, the iterative adaptive dynamic programming (ADP) algorithm is introduced to obtain the infinite-horizon optimal tracking controller which makes the value function close to its optimal value function. Next, two neural networks are used as parametric structures to implement the ADP algorithm, which aim at approximating the value function and the control policy, respectively. Finally, a simulation example is included to complement the theoretical discussions.