Stochastic optimal control via Bellman''s principle

This paper presents a strategy for finding optimal controls of non-linear systems subject to random excitations. The method is capable to generate global control solutions when state and control constraints are present. The solution is global in the sense that controls for all initial conditions in a region of the state space are obtained. The approach is based on Bellman's principle of optimality, the cumulant neglect closure method and the short-time Gaussian approximation. Problems with state-dependent diffusion terms, non-closeable hierarchies of moment equations for the states and singular state boundary condition are considered in the examples. The uncontrolled and controlled system responses are evaluated by creating a Markov chain with a control dependent transition probability matrix via the generalized cell mapping method. In all numerical examples, excellent controlled performances were obtained.     

Skillful control under uncertainty via direct reinforcement learning

Complexity and uncertainty in modern robots and other autonomous systems make it difficult to design controllers for such systems that can achieve desired levels of precision and robustness. Therefore learning methods are being incorporated into controllers for such systems, thereby providing the adaptibility necessary to meet the performance demands of the task. We argue that for learning tasks arising frequently in control applications, the most useful methods in practice probably are those we call direct associative reinforcement learning methods. We describe direct reinforcement learning methods and also illustrate with an example the utility of these methods for learning skilled robot control under uncertainty.     

事件触发下马尔可夫跳跃神经网络的随机同步

本文研究了事件触发机制下带有随机噪声的马尔可夫跳跃神经网络的随机同步问题.为了更有效地降低数据传输量和节约网络资源,本文采用了一种事件触发控制.当传输误差和状态误差满足触发条件时,数据才能够被传输,使得主从系统可以在有限的资源和带宽下实现同步.通过构建新的Lyapunov泛函,以及使用广义Dynkin公式和不等式分析方...     

Stochastic minimum-energy control

We give the solution to the minimum-energy control problem for linear stochastic systems. The problem is as follows: given an exactly controllable system, find the control process with the minimum expected energy that transfers the system from a given initial state to a desired final state. The solution is found in terms of a certain forward–backward stochastic differential equation of Hamiltonian type.     

Stochastic bang-bang control

In this correspondence, a class of nonlinear stochastic systems with controls using only partial information about the current state is considered. It is shown that the time optimal control law is of bang-bang type and that it can be computed by solving a two-point boundary value problem (TPBVP) involving systems of nonlinear integro-partial differential equations.     

Stochastic mean square exponential synchronisation of time-varying complex dynamical networks via pinning control

This paper investigates the problem of stochastic mean square exponential synchronisation of complex dynamical networks with time-varying delay via pinning control. By applying the Lyapunov method and stochastic analysis, criteria on mean square exponential synchronisation are established under linear feedback pinning control and adaptive feedback pinning control, which depend on the time-varying delay and stochastic perturbation. These results complement and improve the previously known results. Two numerical examples are given to illustrate the effectiveness and correctness of the derived theoretical results.     

A simulational comparison of intelligent control algorithms on a direct drive manipulator

This paper investigates the control of nonlinear systems by neural networks and fuzzy logic. As the control methods, Gaussian neuro-fuzzy variable structure (GNFVS), feedback error learning architecture (FELA) and direct inverse modeling architecture (DIMA) are studied, and their performances are comparatively evaluated on a two degrees of freedom direct drive robotic manipulator with respect to trajectory tracking performance, computational complexity, design complexity, RMS errors, necessary training time in learning phase and payload variations.     

Stochastic models of mob control

This paper explores the following model of agents’ threshold behavior. Making binary decisions (choosing between “activity” and “passivity”), agents consider the choice of other members in a group. We formulate and solve an associated control problem, i.e., the random choice problem for the initial states of some agents in order to vary the number of agents preferring “passivity” in an equilibrium.     

Stochastic theory of inventory control

The main results of the probabilistic analysis of the stationary modes in the singleitem and multi-item probabilistic models of optimal control were surveyed. Principles of construction of multidimensional recurrent schemes of calculation of complex probabilistic distributions by their generating functions were presented and illustrated by an example. Some of the results were published for the first time.     

Design of sliding surfaces in variable structure control via a direct pole assignment scheme

This paper presents a systematic approach to determine the switching surface in an equivalent pole placement control applied to multivariable systems. This scheme can be applied to both minimum-phase and non-minimum-phase systems. The closed-loop eigenvalues of the control system in the sliding mode can be arbitrarily assigned with considerable freedom. An example is carried out to show the effectiveness of the proposed method.     

Study on the exponential path tracking control of robot manipulators via direct adaptive methods

Exponential path tracking control represents an important issue pertaining to the transient performance of robot control systems. In this paper, the so-called Exp-transformation is applied to obtain transformed robot dynamics models which are used to derive several adaptive control algorithms that achieve exponential path tracking. In contrast to the existing composite adaptive control method; where both the tracking error and the prediction error are used and persistent excitation (p.e.) is required, the proposed strategy requires only the tracking error. This makes the control structure simpler and easier to implement. The main contribution of this paper is the development of practical control strategies for which the p.e. requirement is completely removed (as opposed to relaxing it to semi-p.e. as was done in a recent work). The fundamental idea introduced for exponential stability analysis is conceptually simple and global results are obtained.     

Stochastic servomechanism design via optimal filter theory

The feedback realization of the continuous optimal filter is applied to familar problems of the design of servomechanisms with stochastic inputs and disturbances. This approach leads immediately to optimal feedback configurations and, interestingly, to integral plus derivative control for the wind torque disturbed antenna servo problem. As in optimal filter theory, time-varying gains are generated for the finite time or "lock on" problem. Since digital computer programs are already available for computing these gains the design of complex intercoupled servos is feasible. The application to "minimum bandwidth" servo design is indicated.     

Optimal control of switched systems via non-linear optimization based on direct differentiations of value functions

This paper presents an approach for solving optimal control problems of switched systems. In general, in such problems one needs to find both optimal continuous inputs and optimal switching sequences, since the system dynamics vary before and after every switching instant. After formulating a general optimal control problem, we propose a two stage optimization methodology. Since many practical problems only concern optimization where the number of switchings and the sequence of active subsystems are given, we concentrate on such problems and propose a method which uses nonlinear optimization and is based on direct differentiations of value functions. The method is then applied to general switched linear quadratic (GSLQ) problems. Examples illustrate the results.     

Stochastic optimal control under Poisson-distributed observations

Optimal control problems for linear, stochastic continuous-time systems are considered, in which the time domain is decomposed into a finite set of N disjoint random intervals of the form [t_i, t _i+1), in which a complete state observation is taken at each instant t_i, 0⩽i⩽N-1. Two optimal control problems termed, respectively, the (piecewise) time-invariant control and time-variant control are considered in this framework. Concerning the observation point process, we first consider the general situation in which the increment intervals are i.i.d.r.v.s with unspecified probabilistic distributions. The (piecewise) time-invariant solution is thoroughly developed in this general case, and computations are illustrated using Erlang as the observations interarrival distribution. Next, the problem is specialized so increments are exponentially distributed, and the particular optimal control structure that results from this assumption is presented. Finally, and still under the Poisson assumption and for the time-variant case, we show that the control problem is closely related to linear quadratic Gaussian regulation with an exponentially discounted cost. The optimal control is made again of a sequence of piecewise open-loop controls corresponding, in this case, to linear feedback of the state predictor based on the most recent information on each interval. The feedback gains are time-varying matrices obtained from a sequence of algebraic Riccati equations, which are also computed off-line     

Stochastic control of an epidemic process

An optimal control problem for a susceptible → infective → removal type epidemic process is formulated and then it is shown how a control procedure can be obtained so that the expected final size of the epidemic can be adaptively minimized.     

Stochastic optimal control of networked control systems with control packet dropouts

This paper considers the stochastic optimal control problem for networked control systems(NCSs)with control packet dropouts.The proportional plus up to the third-order derivative(PD3)compensation strategy is adopted to compensate for control packet dropouts at the actuator by using the past control packets stored in the buffer.Based on the strategy,a new NCS structure model with packet dropouts is provided,where the packet dropout is assumed to obey the Bernoulli random binary distribution.In terms of the given model,the stochastic optimal control law is proposed. Numerical examples illustrate the effectiveness of the results.     

Stochastic control with uncertain macroeconomic parameters

An econometric model of the U.S. economy with random coefficients is solved as a stochastic control problem by augmenting the state vector with random coefficients, linearizing the resulting problem, and obtaining a feedback rule for the augmented problem. The performance of this augment closed loop rule is then compared to regular closed rule with Monte Carlo methods on one-random parameter and three-random parameter examples.