An adaptive automaton controller for discrete-time markov processes

The computation of an optimal feedback controller characteristic for a non-linear stochastic system may be facilitated by the use of a stochastic automaton as a system model. A problem of particular interest is that of a long duration stationary Markov process in which the state is observable but the process dynamics and disturbance characteristics are initially unknown. The determination of a suitable control algorithm, in the form of an adaptive automation in the feedback loop, is considered in this paper for such a process.

Since the algorithm is to be used on-line to perform simultaneously the functions of estimation and control, it must constitute an efficient convergent multi-stage dual control strategy. It is shown that an existing method for dual control of a repetitive single-stage stochastic process may be extended to apply to the present case. A method is introduced of calculating successive policy estimates recursively, so that the task of updating the estimated optimal feedback policy at each stage of the process is rendered feasible. The application of the automaton controller is illustrated by the simulated adaptive control of a non-linear conditionally stable heat treatment process disturbed by multiplicative noise.     

Optimal fixed rules and simple feedback laws in the design of economic policy

The constancy parameter of fixed-rate-of-growth rules and the parameters of simple feedback laws, linking policy instruments to specified endogenous variables, can be determined optimally within a policy optimization framework. An approach for doing this is described along with results using the National Institute of Economic and Social Research nonlinear econometric model of the U.K. The framework is also extended to rational expectations. The effect of an optimal fixed-rate-of-growth monetarist rule is illustrated in an example incorporating the credible announcement of monetary targets. This is evaluated against a strategy derived without the assumption of announcement effects. The use of indicators is also discussed to provide an empirical example for parametrized optimal feedback laws. Exchange rate indicators are derived and compared with open-loop policies. Robust control extensions are suggested and Monte Carlo simulations are used to test the behaviour of the policies in the presence of stochastic disturbances.     

A stochastic control model for optimal timing of climate policies

A stochastic control model is proposed as a paradigm for the design of optimal timing of greenhouse gas (GHG) emission abatement. The resolution of uncertainty concerning climate sensitivity and the technological breakthrough providing access to a carbon-free production economy are modeled as controlled stochastic jump processes. The optimal policy is characterized using the dynamic programming solution to a piecewise deterministic optimal control problem. A numerical illustration is developed with a set of parameters calibrated on recently proposed models for integrated assessment of climate policies. The results are interpreted and the insights they provide on the timing issue of climate policy are discussed.     

Adaptive dual control of discrete-time distributed-parameter stochastic systems

The problem of adaptive dual control of discrete-time distributed-parameter stochastic systems is examined. It is shown that there exists an important difference between feedback and closed-loop policies of control for this type of system as for the lumped parameter case. This difference is based on the adaptivity feature of the control. Namely, when the control policy affects both the state and its uncertainty (dual effect) it possesses the so-called feature of active adaptivity and can only be a characteristic of a closed-loop policy, whereas a feedback policy can only be passively adaptive. These results can be used to develop a control algorithm for non-linear problems for which the realization of optimal control laws involves control strategies with both learning and control features.     

Stochastic differential games and inverse optimal control and stopper policies

In this paper, we consider a two-player stochastic differential game problem over an infinite time horizon where the players invoke controller and stopper strategies on a nonlinear stochastic differential game problem driven by Brownian motion. The optimal strategies for the two players are given explicitly by exploiting connections between stochastic Lyapunov stability theory and stochastic Hamilton–Jacobi–Isaacs theory. In particular, we show that asymptotic stability in probability of the differential game problem is guaranteed by means of a Lyapunov function which can clearly be seen to be the solution to the steady-state form of the stochastic Hamilton–Jacobi–Isaacs equation, and hence, guaranteeing both stochastic stability and optimality of the closed-loop control and stopper policies. In addition, we develop optimal feedback controller and stopper policies for affine nonlinear systems using an inverse optimality framework tailored to the stochastic differential game problem. These results are then used to provide extensions of the linear feedback controller and stopper policies obtained in the literature to nonlinear feedback controllers and stoppers that minimise and maximise general polynomial and multilinear performance criteria.     

Proof of quasi-adaptivity for the m-measurement feedback class of stochastic control policies

It is shown that them-measurement feedback (mM) policy for nonlinear stochastic control, performs as well or better than the open-loop optimal policy and hence becomes one of only a few suboptimal policies presently known to be quasi-adaptive in the sense of Witsenhausen.     

Optimal control of two interacting service stations

Optimal controls described by switching curves in the two-dimensional state space are shown to exist for the optimal control of a Markov network with two service stations and linear cost. The controls govern routing and service priorities. Finite horizon and long run average cost problems are considered and value iteration is a key tool. Nonconvex value functions are shown to exist for slightly more general networks. Nonconvex value functions are also shown to arise for a simple single station control problem in which the instantaneous cost is convex but not monotone. Nevertheless, optimality of threshold policies is established for the single station problem. The proof is based on a novel use of stochastic coupling and policy iteration.     

Optimal tracker for unreliable manufacturing systems

This paper deals with the inventory-production control problem where the produced items are assumed to deteriorate at a rate that depends on the demand rate of the production system. The state of this production system is assumed to be described by a continuous-time Markov process taking values in a finite discrete space. The inventory production control problem is formulated as a stochastic optimal control problem. The optimal policy that solves the optimal control problem is obtained in terms of a set of coupled Riccati equations. The guaranteed cost problem is also treated. A numerical example is provided to show the usefulness of the proposed model.     

Policy iteration based feedback control

It is well known that stochastic control systems can be viewed as Markov decision processes (MDPs) with continuous state spaces. In this paper, we propose to apply the policy iteration approach in MDPs to the optimal control problem of stochastic systems. We first provide an optimality equation based on performance potentials and develop a policy iteration procedure. Then we apply policy iteration to the jump linear quadratic problem and obtain the coupled Riccati equations for their optimal solutions. The approach is applicable to linear as well as nonlinear systems and can be implemented on-line on real world systems without identifying all the system structure and parameters.     

A linear programming approach for multivariable feedback control with inequality constraints

A multivariable control scheme which explicitly includes inequality constraints on the state, control and output variables is developed using a linear programming (LP) formulation. A suboptimal feedback control policy results since the LP calculations are repeated at each sampling instant. A number of important properties related to the LP control problem are obtained from a theoretical analysis. Simulation results illustrate that the on-line computational requirements are modest and that the new LP approach compares favourably with existing methods such as optimal feedback control     

Predictive stochastic feedforward-feedback control of a heat exchanger-stirred tank system

An optimal stochastic feedforward-feedback control scheme is implemented on a heat exchanger-stirred tank system using an on-line minicomputer. Because variations in the measured disturbance variable affect the process output (controlled variable) before the compensating action can become effective, the feedforward action must be predictive in nature. Statistical time series models are used to model both the measured and unmeasured disturbances in the system. These stochastic disturbance models and the transfer-function models for the process are identified, fitted and checked using statistical model-building procedures on a set of data collected on-line using the minicomputer. The predictive feedforward-feedback controller is derived from these models. The performance of this control scheme is compared with that of an optimal feedback scheme and the actual on-line performance is shown to conform well to the theoretical predictions     

Joint production and supply control in three levels flexible manufacturing systems

This paper considers a stochastic optimal control problem of a three stages Flexible Manufacturing System. The supplier (i.e., upstream FMS) and the transformation stage (i.e., FMS) are both subject to random events. Our objective is to find a feedback control policy for the supply and production activities that minimizes the incurred cost. It is shown that the considered joint production and supply control problem is difficult to tackle using a dissociated analytical approach. A simulation based approach is thus proposed to achieve a close approximation of the optimal policy. The advantages of the approach include possible extensions after numerical characterization of the optimal control policy.     

Fixed‐structure robust controller design for chatter mitigation in high‐speed milling

Chatter is an instability phenomenon in high‐speed milling that limits machining productivity by the induction of tool vibrations, inferior machining accuracy, noise, and wear of machine components. In this paper, a fixed‐structure active chatter control design methodology is proposed, which enables dedicated shaping of the chatter stability boundary such that working points of higher machining productivity become feasible while avoiding chatter. The control design problem is cast into a nonsmooth optimization problem, which is solved using bundle methods. Using this approach, fixed‐structure dynamic (delayed) output feedback controllers can be synthesized. Distinct benefits of this approach are the a priori fixing of the controller order, the limitation of the control action, and the fact that no finite‐dimensional model approximations and online chatter estimation techniques are required. All these benefits are important in milling practice. Representative examples illustrate the power of the proposed methodology in terms of increasing the chatter‐free depth of cut, thereby enabling significant increases in machining productivity. Copyright © 2014 John Wiley & Sons, Ltd.     

Artificial intelligence approaches to determination of CNC machining parameters in manufacturing: a review

In Computer Numerical Control (CNC) machining, determining optimum or appropriate cutting parameters can minimize machining errors such as tool breakage, tool deflection and tool wear, thus yielding a high productivity or minimum cost. There have been a number of attempts to determine the machining parameters through off-line adjustment or on-line adaptive control. These attempts use many different kinds of techniques: CAD-based approaches, Operations Research approaches, and Artificial Intelligence (AI) approaches. After describing an overview of these approaches, we will focus on reviewing AI-based techniques for providing a better understanding of these techniques in machining control. AI-based methods fall into three categories: knowledge-based expert systems approach, neural networks approach and probabilistic inference approach. In particular, recent research interests mainly tend to develop on-line or real-time expert systems for adapting machining parameters. The use of AI techniques would be valuable for the purpose.     

Optimal manufacturing flow controllers: zero-inventory policies andcontrol switching sets

A continuous-flow control model of a single workstation with multiple failure modes and part types is considered. Although the general model is intractable, properties of the optimal control policy are identified that can be used to help formulate heuristic policies. Control switching sets are described and shown to have a threshold form. For a single machine with two part-types, conditions are found under which no inventory is held, analogous to the single-part-type result of Bielecki and Kumar (1988)     

Method of stochastic sensitivity synthesis in a stabilisation problem for nonlinear discrete systems with incomplete information

A problem of the synthesis of the assigned probabilistic distribution near the equilibrium of stochastic nonlinear discrete-time system with incomplete state information is considered. We construct a static feedback regulator which provides a required stochastic sensitivity of this equilibrium. It is shown that this problem is reduced to the solution of some quadratic matrix equations. The solvability of these quadratic equations is analysed, and attainability sets are described. In the two-dimensional case, two variants of the control input are discussed and compared. These general results are applied to the suppression of large-amplitude oscillations around the equilibria of the stochastically forced Henon model with noisy observations. We show how suggested control technique, by minimising the stochastic sensitivity, allows us to suppress chaos and provide a structural stabilisation.     

Identification and Control of Multivariable Stochastic Feedback Systems

Abstract

This paper considers the problem of simultaneous identification and control of stochastic processes characterized by linear dynamic models with unknown systems parameter coefficients. Stochastic approximation is used to derive consistent identification algorithms for the case in which arbitrary feedback controls are present. These identification methods can also be used for determining the order of the system, if the latter is unknown, as well as the exact canonical structure for the multivariable case.

An approximation to the optimal control solution is obtained by explicitly separating the functions of identification and control, and asymptotic convergence to a stochastic optimal controller is attained without on-line structural modification.     

Modeling and evaluating design alternatives for an on-lineinstrumentation system: a case study

This paper demonstrates the use of a model-based evaluation approach for instrumentation systems (ISs). The overall objective of this study is to provide early feedback to tool developers regarding IS overhead and performance; such feedback helps developers make appropriate design decisions about alternative system configurations and task scheduling policies. We consider three types of system architectures: network of workstations (NOW), symmetric multiprocessors (SMP), and massively parallel processing (MPP) systems. We develop a Resource OCCupancy (ROCC) model for an on-line IS for an existing tool and parameterize it for an IBM SP-2 platform. This model is simulated to answer several “what if” questions regarding two policies to schedule instrumentation data forwarding: collect-and-forward (CF) and batch-and-forward (BF). In addition, this study investigates two alternatives for forwarding the instrumentation data: direct and binary tree forwarding for an MPP system. Simulation results indicate that the BF policy can significantly reduce the overhead and that the tree forwarding configuration exhibits desirable scalability characteristics for MPP systems. Initial measurement-based testing results indicate more than 60 percent reduction in the direct IS overhead when the BF policy was added to Paradyn parallel performance measurement tool     

进化神经网络在机床工具损耗预测中的应用

设计了一个基于实数编码的改进进化算法优化神经网络的连接权和网络结构.该算法 可以根据种群停止进化代数自适应调节变异率、根据个体适应度调节变异量.加工实验表明采用 进化神经网络可以较准确预测出电火花铣削加工工具损耗,所提出的进化算法是有效的,预测结 果较标准BP神经网络高.该预测模型为电火花铣削加工工具在线自动补偿打下基础.