A comparison of open-loop and closed-loop strategies in colloidal self-assembly

Ordered colloidal crystals possess unique photonic properties for a wide range of applications in engineering, material science, communications, and medicine. However, ordered structures are hard to achieve even in small systems, due to the formation of defects during the process. In this paper, we focus on the control of a micron-sized SiO₂ colloidal self-assembly process for defect-free crystal in an externally applied electric field batch system. Five control strategies including time-constant and time-varying, heuristic and model-based open-loop and closed-loop policies are investigated, to understand their control mechanisms. The model-based policies are designed using a reduced-order Markov state model, built with samples from Brownian dynamics simulations, and are calculated with dynamic programming. The performance of all the five strategies are evaluated on both the Brownian dynamics simulation and the experiments. Time-varying control strategies can improve the yield of grain-boundary-free crystals over that of their time-constant counterparts. Moreover, using feedback as “endpoint detection” to terminate the process can also shorten the process time, compared to open-loop strategies.     

Solution time reduction techniques of a stochastic dynamic programming approach for hazardous material route selection problem

We use a stochastic dynamic programming (SDP) approach to solve the problem of determining the optimal routing policies in a stochastic dynamic network. Due to its long time for solving SDP, we propose three techniques for pruning stochastic dynamic networks to expedite the process of obtaining optimal routing policies. The techniques include: (1) use of static upper/lower bounds, (2) pre-processing the stochastic dynamic networks by using the start time and origin location of the vehicle, and (3) a mix of pre-processing and upper/lower bounds. Our experiments show that while finding optimal routing policies in stochastic dynamic networks, the last two of the three strategies have a significant computational advantage over conventional SDP. Our main observation from these experiments was that the computational advantage of the pruning strategies that depend on the start time of the vehicle varies according to the time input to the problem. We present the results of this variation in the experiments section. We recommend that while comparing the computational performances of time-dependent techniques, it is very important to test the performance of such strategies at various time inputs.     

A stochastic control approach to Slotted-ALOHA random access protocol

ALOHA random access protocols are distributed protocols based on transmission probabilities, that is, each node decides upon packet transmissions according to a transmission probability value. In the literature, ALOHA protocols are analysed by giving necessary and sufficient conditions for the stability of the queues of the node buffers under a control vector (whose elements are the transmission probabilities assigned to the nodes), given an arrival rate vector (whose elements represent the rates of the packets arriving in the node buffers). The innovation of this work is that, given an arrival rate vector, it computes the optimal control vector by defining and solving a stochastic control problem aimed at maximising the overall transmission efficiency, while keeping a grade of fairness among the nodes. Furthermore, a more general case in which the arrival rate vector changes in time is considered. The increased efficiency of the proposed solution with respect to the standard ALOHA approach is evaluated by means of numerical simulations.     

Economic lot sizing in a production system with random demand

An extended economic production quantity model that copes with random demand is developed in this paper. A unique feature of the proposed study is the consideration of transient shortage during the production stage, which has not been explicitly analysed in existing literature. The considered costs include set-up cost for the batch production, inventory carrying cost during the production and depletion stages in one replenishment cycle, and shortage cost when demand cannot be satisfied from the shop floor immediately. Based on renewal reward process, a per-unit-time expected cost model is developed and analysed. Under some mild condition, it can be shown that the approximate cost function is convex. Computational experiments have demonstrated that the average reduction in total cost is significant when the proposed lot sizing policy is compared with those with deterministic demand.     

Stability and output feedback stabilization for systems with Markovian switching and impulse effects

We investigate the exponential stability in the mean square sense for the systems with Markovian switching and impulse effects. Based on the statistic property of the Markov process, a stability criterion is established. Then, by the parameterizations via a family of auxiliary matrices, the dynamical output feedback controller can be solved via an LMI approach, which makes the closed-loop system exponentially stable. A numerical example is given to demonstrate the method.     

Symmetric approximate linear programming for factored MDPs with application to constrained problems

A weakness of classical Markov decision processes (MDPs) is that they scale very poorly due to the flat state-space representation. Factored MDPs address this representational problem by exploiting problem structure to specify the transition and reward functions of an MDP in a compact manner. However, in general, solutions to factored MDPs do not retain the structure and compactness of the problem representation, forcing approximate solutions, with approximate linear programming (ALP) emerging as a promising MDP-approximation technique. To date, most ALP work has focused on the primal-LP formulation, while the dual LP, which forms the basis for solving constrained Markov problems, has received much less attention. We show that a straightforward linear approximation of the dual optimization variables is problematic, because some of the required computations cannot be carried out efficiently. Nonetheless, we develop a composite approach that symmetrically approximates the primal and dual optimization variables (effectively approximating both the objective function and the feasible region of the LP), leading to a formulation that is computationally feasible and suitable for solving constrained MDPs. We empirically show that this new ALP formulation also performs well on unconstrained problems.      

A new LP formulation of the admission control problem modelled as an MDP under average reward criterion

The admission control problem can be modelled as a Markov decision process (MDP) under the average cost criterion and formulated as a linear programming (LP) problem. The LP formulation is attractive in the present and future communication networks, which support an increasing number of classes of service, since it can be used to explicitly control class-level requirements, such as class blocking probabilities. On the other hand, the LP formulation suffers from scalability problems as the number C of classes increases. This article proposes a new LP formulation, which, even if it does not introduce any approximation, is much more scalable: the problem size reduction with respect to the standard LP formulation is O((C?+?1)²/2 ^C ). Theoretical and numerical simulation results prove the effectiveness of the proposed approach.     

Dynamic vehicle allocation control for automated material handling system in semiconductor manufacturing

The current study examines the dynamic vehicle allocation problems of the automated material handling system (AMHS) in semiconductor manufacturing. With the uncertainty involved in wafer lot movement, dynamically allocating vehicles to each intrabay is very difficult. The cycle time and overall tool productivity of the wafer lots are affected when a vehicle takes too long to arrive. In the current study, a Markov decision model is developed to study the vehicle allocation control problem in the AMHS. The objective is to minimize the sum of the expected long-run average transport job waiting cost. An interesting exhaustive structure in the optimal vehicle allocation control is found in accordance with the Markov decision model. Based on this exhaustive structure, an efficient algorithm is then developed to solve the vehicle allocation control problem numerically. The performance of the proposed method is verified by a simulation study. Compared with other methods, the proposed method can significantly reduce the waiting cost of wafer lots for AMHS vehicle transportation.     

On the existence of stationary optimal policies for partially observed MDPs under the long-run average cost criterion

This paper studies the problem of the existence of stationary optimal policies for finite state controlled Markov chains, with compact action space and imperfect observations, under the long-run average cost criterion. It presents sufficient conditions for existence of solutions to the associated dynamic programming equation, that strengthen past results. There is a detailed discussion comparing the different assumptions commonly found in the literature.     

带有干扰的挠性卫星非线性姿态输出反馈控制

针对挠性卫星在飞行过程中存在参数不确定性、干扰（常值扰动和正弦扰动）及挠性附件的振动控制问题,提出了一类基于输出反馈控制系统的鲁棒设计方法,该设计仅利用姿态四元数输出信息,而无需角速度、挠性变形位移及其速率测量信息;同时,在控制中又引入积分环节用于减小常值干扰引起的稳态误差,并且控制器参数的选者并不依赖于系统参数,基于Lyapunov理论证明了所设计的控制器保证了姿态的稳定和模态振动的衰减;最后,将该方法应用于挠性卫星的姿态机动控制,仿真结果表明该控制器不仅对参数不确定性具有很好的鲁棒性,而且能够有效消除常值干扰和正弦干扰的影响,在完成姿态机动控制的同时,能够抑制挠性附件的结构振动,具有良好的过渡过程品质．     

Piecewise output feedback control for affine systems with disturbances based on linear temporal logic specifications

In the paper,we investigate the problem of finding a piecewise output feedback control law for an uncertain affine system such that the resulting closed-loop output satisfies a desired linear temporal logic (LTL) specification.A two-level hierarchical approach is proposed to solve the problem in a triangularized output space.In the lower level,we explore whether there exists a robust output feedback control law to make the output starting in a simplex either remains in it or leaves via a specific facet.In the higher level,for the triangularization,we construct the transition system according to the reachability relationship obtained in the lower level and search for feasible paths that meet the LTL specification.The control approach is then applied to solve a motion planning problem.     

Output feedback variable structure control for linear systems with uncertainties and disturbances

This paper proposes a dynamic output feedback variable structure controller for linear MIMO systems with mismatched and matched norm-bounded uncertainties and matched nonlinear disturbances. The proposed controller consists of nonlinear and linear parts, similar to the unit vector type of controller. The nonlinear part of the new controller takes care only of matched uncertainties and disturbances and, on the other hand, the linear part with full dynamics completely handles mismatched uncertainties. Designing such a linear part leads to use a Lyapunov function associated with the full states, which achieves the global stability against the mismatched uncertainties. The resulting criteria are, furthermore, converted into solvable ones, by using the so-called cone complementary linearization algorithm for bi-convex problems.     

随机拓扑结构下多智能体系统的平均一致性*

利用马尔可夫收敛准则、图的Laplace矩阵谱特性和欧氏度量的极值,对一类具有随机拓扑结构的离散时间多智能体系统平均一致性问题进行了深入讨论。引入完好概率矩阵的概念,建立随机拓扑结构下离散时间系统的一致性算法,应用马尔可夫过程收敛相关结论及伴随算子,从欧氏度量极值的角度证明了系统可达到渐近平均一致,并得出了所需满足的条件,该条件放宽了对系统连通性的要求。最后,采用六个智能体组成的多智能体系统进行计算机仿真,对理论的正确性进行了验证。     

Optimum random and age replacement policies for customer-demand multi-state system reliability under imperfect maintenance

This paper proposes the generalised random and age replacement policies for a multi-state system composed of multi-state elements. The degradation of the multi-state element is assumed to follow the non-homogeneous continuous time Markov process which is a continuous time and discrete state process. A recursive approach is presented to efficiently compute the time-dependent state probability distribution of the multi-state element. The state and performance distribution of the entire multi-state system is evaluated via the combination of the stochastic process and the Lz-transform method. The concept of customer-centred reliability measure is developed based on the system performance and the customer demand. We develop the random and age replacement policies for an aging multi-state system subject to imperfect maintenance in a failure (or unacceptable) state. For each policy, the optimum replacement schedule which minimises the mean cost rate is derived analytically and discussed numerically.     

Decision-theoretic planning with generalized first-order decision diagrams

Many tasks in AI require representation and manipulation of complex functions. First-Order Decision Diagrams (FODD) are a compact knowledge representation expressing functions over relational structures. They represent numerical functions that, when constrained to the Boolean range, use only existential quantification. Previous work has developed a set of operations for composition and for removing redundancies in FODDs, thus keeping them compact, and showed how to successfully employ FODDs for solving large-scale stochastic planning problems through the formalism of relational Markov decision processes (RMDP). In this paper, we introduce several new ideas enhancing the applicability of FODDs. More specifically, we first introduce Generalized FODDs (GFODD) and composition operations for them, generalizing FODDs to arbitrary quantification. Second, we develop a novel approach for reducing (G)FODDs using model checking. This yields – for the first time – a reduction that maximally reduces the diagram for the FODD case and provides a sound reduction procedure for GFODDs. Finally we show how GFODDs can be used in principle to solve RMDPs with arbitrary quantification, and develop a complete solution for the case where the reward function is specified using an arbitrary number of existential quantifiers followed by an arbitrary number of universal quantifiers.     

Stochastic system controller synthesis for reachability specifications encoded by random sets

We consider a reach–avoid specification for a stochastic hybrid dynamical system defined as reaching a goal set at some finite time, while avoiding an unsafe set at all previous times. In contrast with earlier works which consider the target and avoid sets as deterministic, we consider these sets to be probabilistic. An optimal control policy is derived which maximizes the reach–avoid probability. Special structure on the stochastic sets is exploited to make the computation tractable for large space dimensions.     

Dynamic output feedback robust MPC with convex optimisation for system with polytopic uncertainty

This paper proposes a dynamic output feedback robust model predictive control for system with polytopic model uncertainty. Earlier researches on this topic utilise iterative methods to solve the non-convex optimisation problem which are computationally demanding. In order to reduce the computational burden, we explore a new approach in this paper, where, by utilising some proper matrix transformations, a computationally more efficient but conservative convex optimisation problem is formulated which can be solved in terms of linear matrix inequalities. Furthermore, we try to reduce the conservativeness by introducing a nonsingular matrix as a degree of freedom. The recursive feasibility and the convergence of the augmented state to the equilibrium point are guaranteed. The effectiveness of the proposed approach is illustrated by two numerical examples.     

Static output feedback sliding mode control for time‐varying delay systems with time‐delayed nonlinear disturbances

In this paper, a robust stabilization problem for a class of linear time‐varying delay systems with disturbances is studied using sliding mode techniques. Both matched and mismatched disturbances, involving time‐varying delay, are considered. The disturbances are nonlinear and have nonlinear bounds which are employed for the control design. A sliding surface is designed and the stability of the corresponding sliding motion is analysed based on the Razumikhin Theorem. Then a static output feedback sliding mode control with time delay is synthesized to drive the system to the sliding surface in finite time. Conservatism is reduced by using features of sliding mode control and systems structure. Simulation results show the effectiveness of the proposed approach. Copyright © 2009 John Wiley & Sons, Ltd.     

Adaptive controller design and disturbance attenuation for SISO linear systems with noisy output measurements and partly measured disturbances

In this article, we present robust adaptive controller design for SISO linear systems with noisy output measurements and partly measured disturbances. Using the worst-case analysis approach, we formulate the robust adaptive control problem as a non-linear H ^∞-optimal control problem under imperfect state measurements and solve it using game theory. The design paradigm is the same as (Pan, Z. and Ba?ar, T., 1998, Adaptive Controller Design for Tracking and Disturbance Attenuation for SISO Linear Systems with Noisy Output Measurements, CSL report, Urbana, IL: University of Illinois at Urbana-Champaign) with the only difference being the treatment of the measured disturbances. The same result as (Pan and Ba?ar l998) is achieved. In addition, when the relative degrees from the measured disturbances to the output are no less than that from the control input, the controller designed achieves the zero disturbance attenuation level with respect to the measured disturbance inputs. The asymptotic tracking objective is achieved even if the measured disturbance is only uniformly bounded, without requiring it to be of finite energy. This strong robustness property is then illustrated by numerical examples.     

模态跳变概率可控的Markov跳变线性系统的优化

研究模态跳变概率可控的Markov跣变线性二次模型的最优控制问题,考虑两类模态跳变控制策略:开环模态控制和闭环模态控制,应用策略迭代和性能势的概念,给出了最优的闭环模态控制优于最优的开环模态控制的充分条件,以指导最优控制器的设计,在已知最优的开环模态控制策略的基础上,应用策略迭代给出了构造闭环模态控制策略的方法,以进一步改善系统的性能.