转移概率未知下具有双Markov 链的网络控制系统控制器设计

研究一类基于Markov模型的网络控制系统的稳定性和镇定控制器设计问题.针对网络控制系统中受控对象模型的随机切换和通信过程中的丢包问题,利用具有两个独立Markov链的离散时间Markov跳跃系统进行建模.在该Markov跳跃系统模态转移概率矩阵部分元素未知的情况下,充分考虑转移概率的约束条件,给出系统可镇定的充要条件和状态反馈控制器的设计方法.最后通过仿真示例验证了所提出方法的有效性.     

具有脉冲效应的切换随机系统的几乎必然稳定性

针对一类在切换时刻具有脉冲行为的Markov切换非线性随机系统,首先,应用切换的Lyapunov函数方法研究系统的稳定性,给出系统几乎必然稳定的充分条件,该条件不依赖于系统的矩稳定性;然后,进一步对线性系统的稳定化问题进行分析与设计,对随机子系统的控制结构同时出现在方程的位移部分与扩散部分,给出相应的状态反馈增益矩阵的求解方法;最后,数值算例说明了所设计方法的有效性.     

线性混合系统的可观性分析

线性混合系统的控制输入(包括离散控制和连续控制)会影响混合系统的状态可观性．混合系统的可观性包括初始的离散状态和连续状态，以及离散状态的切换时间．对此．分析了系统的控制输入对线性混合系统状态(主要是离散状态)的影响，并论述这将有助于改善某些线性混合系统状态的可观性．通过分析给出了在这种情形下比较宽松的线性混合系统状态可观性条件及其秩检验条件，并给出说明性示例．     

基于混合随机Petri网的一类混合系统的模型及稳定性

混合动态系统是包含离散事件系统（ＤＥＳ）和连续变量系统（ＣＶＳ）的复杂系统。讨论了一类每个离散状态包含一个连续动态系统,且离散事件具有Ｍａｒｋｏｖ链性质的随机混合系统,在提出利用混合随机Ｐｅｔｒｉ网的建模方法的基础上,给出混合均方稳定的概念、稳定性和可镇定条件,最后以一个简单的例子做子说明和仿真。     

随机平稳策略下半Markov决策过程的仿真优化算法

基于性能势理论和等价Markov过程方法,研究了一类半Markov决策过程(SMDP)在参数化随机平稳策略下的仿真优化算法,并简要分析了算法的收敛性．通过SMDP的等价Markov过程,定义了一个一致化Markov链,然后根据该一致化Markov链的单个样本轨道来估计SMDP的平均代价性能指标关于策略参数的梯度,以寻找最优(或次优)策略．文中给出的算法是利用神经元网络来逼近参数化随机平稳策略,以节省计算机内存,避免了“维数灾”问题,适合于解决大状态空间系统的性能优化问题．最后给出了一个仿真实例来说明算法的应用．     

一类分层非结构化P2P系统的随机切换模型

对于一类利用集中式构架和分布式构架各自优点的分层非结构化P2P系统,通过定义一种Markov切换空间模型来描述其动态分组切换行为.在Markov决策过程理论的基础上,给出了关于性能指标的策略迭代和在线策略迭代算法,并通过实例仿真说明该方法的优越性.     

脉冲随机混合系统的稳定性与鲁棒稳定性分析

针对一类在切换时刻具有脉冲行为的Markov切换随机系统,首先,利用多Lyapunov函数的方法研究系统的稳定性,得到系统依概率稳定的充分条件,该条件以线性矩阵不等式(LMI)的形式给出;然后,进一步对系统的稳定化以及鲁棒稳定性问题进行分析与设计,给出相应的状态反馈增益矩阵和脉冲增益矩阵的求解方法;最后,数值算例说明了所设计方法的有效性.     

基于系统调用和齐次Markov链模型的程序行为异常检测

异常检测是目前入侵检测领域研究的热点内容.提出一种新的基于系统调用和Markov链模型的程序行为异常检测方法,该方法利用一阶齐次Markov链对主机系统中特权程序的正常行为进行建模,将Markov链的状态同特权程序运行时所产生的系统调用联系在一起,并引入一个附加状态;Markov链参数的计算中采用了各态历经性假设;在检测阶段,基于状态序列的出现概率对特权程序当前行为的异常程度进行分析,并根据Markov链状态的实际含义和程序行为的特点,提供了两种可选的判决方案.同现有的基于隐Markov模型和基于人工免疫原理的检测方法相比,提出的方法兼顾了计算成本和检测准确度,特别适用于在线检测.该方法已应用于实际入侵检测系统,并表现出良好的检测性能.     

网络化时滞控制系统的有限时间稳定性分析

针对网络控制系统中存在于传感器-控制器-执行器间的双时延问题,提出了一种基于Markov模型的状态反馈控制策略.与传统应用Markov随机过程的方式相比,该策略采用两个Markov链描述每一个时延,通过状态反馈把该随机系统描述为具有四个随机参量的离散Markov跳变系统.利用Lyapunov有限时间稳定性理论分析得到该系统稳定的充分条件,并利用线性矩阵不等式(LMI)得到了可行的反馈矩阵.数值仿真结果进一步证明了该策略的有效性.     

人工蜂群算法的收敛性分析

利用随机过程理论,对人工蜂群算法收敛性进行理论分析,给出人工蜂群算法的一些数学定义和蜜源位置的一步转移概率,建立人工蜂群算法的Markov链模型,分析此Markov链的一些性质,论证了人工蜂群状态序列是有限齐次Markov链,且状态空间是不可约的。结合随机搜索算法的全局收敛准则,证明了人工蜂群算法能够满足随机搜索算法全局收敛的两个假设,保证算法的全局收敛。     

Partially‐observable stochastic hybrid systems (poshss) state estimation and optimal control

This paper discusses the state estimation and optimal control problem of a class of partially‐observable stochastic hybrid systems (POSHS). The POSHS has interacting continuous and discrete dynamics with uncertainties. The continuous dynamics are given by a Markov‐jump linear system and the discrete dynamics are defined by a Markov chain whose transition probabilities are dependent on the continuous state via guard conditions. The only information available to the controller are noisy measurements of the continuous state. To solve the optimal control problem, a separable control scheme is applied: the controller estimates the continuous and discrete states of the POSHS using noisy measurements and computes the optimal control input from the state estimates. Since computing both optimal state estimates and optimal control inputs are intractable, this paper proposes computationally efficient algorithms to solve this problem numerically. The proposed hybrid estimation algorithm is able to handle state‐dependent Markov transitions and compute Gaussian‐ mixture distributions as the state estimates. With the computed state estimates, a reinforcement learning algorithm defined on a function space is proposed. This approach is based on Monte Carlo sampling and integration on a function space containing all the probability distributions of the hybrid state estimates. Finally, the proposed algorithm is tested via numerical simulations.     

Stability analysis and stabilization of networked linear systems with random packet losses

This paper is concerned with the stability analysis and stabilization of networked discrete-time and sampled-data linear systems with random packet losses. Asymptotic stability, mean-square stability, and stochastic stability are considered. For networked discrete-time linear systems, the packet loss period is assumed to be a finite-state Markov chain. We establish that the mean-square stability of a related discrete-time system which evolves in random time implies the mean-square stability of the system in deterministic time by using the equivalence of stability properties of Markovian jump linear systems in random time. We also establish the equivalence of asymptotic stability for the systems in deterministic discrete time and in random time. For networked sampled-data systems, a binary Markov chain is used to characterize the packet loss phenomenon of the network. In this case, the packet loss period between two transmission instants is driven by an identically independently distributed sequence assuming any positive values. Two approaches, namely the Markov jump linear system approach and randomly sampled system approach, are introduced. Based on the stability results derived, we present methods for stabilization of networked sampled-data systems in terms of matrix inequalities. Numerical examples are given to illustrate the design methods of stabilizing controllers.     

基于切换频度的马尔科夫网络控制系统均方指数镇定

针对一类马尔科夫网络控制系统(Networked control system, NCS),研究了其均方指数镇定问题. 首先将网络控制系统建模为离散时间切换系统,子系统间的切换过程由一个转移概率矩阵已知的马尔科夫链描述, 并给出了子系统间切换频度的范围;进而基于随机过程理论和切换系统稳定性理论, 利用状态反馈实现了网络控制系统均方指数镇定,状态反馈控制律可通过求解一组线性矩阵不等式获得. 最后通过数值仿真例子验证了本文方法的有效性.     

一类线性离散时间结构随机跳变系统的逼近滤波算法

提出了具有条件马尔可夫跳变结构的离散时间随机系统的条件滤波方法, 应用随机变结构系统的性质对滤波算法进行简化处理, 并将后验概率密度函数用条件高斯函数来逼近, 得到具有条件马尔可夫结构离散随机系统的逼近最优滤波算法, 最后给出滤波算法的计算步骤并仿真验证了算法的正确性.     

协同业务过程的随机行为分析方法

参与组织随机行为是评价业务协同有效实施的一个关键因素。结合进程代数和马尔科夫链,提出了一种协同业务过程的随机行为分析方法。首先,使用有限状态自动机建模每个参与组织的业务过程,通过引入异步消息通信关系定义协同业务过程。其次,提出将参与组织业务过程转换成通信顺序进程（Communication Sequential Process,CSP）方法,进而将每个参与组织业务过程对应CSP进程并发组合得到协同业务过程对应的CSP进程,并根据CSP操作语义构建协同业务过程的状态迁移模型。最后,引入状态迁移模型正则化概念,从理论上证明正则化的状态迁移模型与一个齐次马尔科夫链相对应,进而根据平衡方程求得状态迁移系统中每个状态的稳定概率,以此为基础实现参与组织随机行为分析。通过对电设备采购过程建模与随机行为分析阐述该方法的可行性和有效性。     

Control of a hybrid stochastic system

We consider in this paper a continuous-time stochastic hybrid control system with a finite time horizon. The objective is to minimize a linear function of the expected state trajectory. The state evolves according to a linear dynamics. However, the parameters of the state evolution equation may change at discrete times according to a controlled Markov chain which has finite state and action spaces. We use a procedure similar in form to the maximum principle; this determines a control strategy which is asymptotically optimal as the number of transitions during the finite time horizon grows to infinity.     

基于Markov延迟特性的闭环网络控制系统研究

针对控制网络中固有的随机传输延迟 ,提出了一种新颖的控制模式 ,实现了对存在多步随机传输延迟的网络控制系统的数学建模 .基于该模型 ,利用Markov链理论 ,得到了满足给定性能指标的随机最优控制律 ,同时给出了求取相应的Markov链状态转移矩阵的方法 .文末通过实验研究 ,验证了所提理论的正确性和有效性     

Minimax control of switching systems under sampling

We consider a general class of systems subject to two types of uncertainty: A continuous deterministic uncertainty that affects the system dynamics, and a discrete stochastic uncertainty that leads to jumps in the system structure at random times, with the latter described by a continuous-time finite state Markov chain. When only sampled values of the system state is available to the controller, along with perfect measurements on the state of the Markov chain, we obtain a characterization of minimax controllers, which involves the solutions of two finite sets of coupled PDEs, and a finite dimensional compensator. For the linear-quadratic case, a complete characterization is given in terms of coupled generalized Riccati equations, which also provides the solution to a particular H^∞ optimal control problem with randomly switching system structure and sampled state measurements.     

Stabilization of Hybrid Systems by Feedback Control Based on Discrete‐Time State and Mode Observations

Recently, a kind of feedback control based on discrete‐time state observations was proposed to stabilize continuous‐time hybrid stochastic systems in the mean‐square sense. We find that the feedback control there still depends on the continuous‐time observations of the mode. However, it usually costs to identify the current mode of the system in practice. So we can further improve the control to reduce the control cost by identifying the mode at discrete times when we make observations for the state. In this paper, we aim to design such a type of feedback control based on the discrete‐time observations of both state and mode to stabilize the given hybrid stochastic differential equations (SDEs) in the sense of mean‐square exponential stability. Moreover, a numerical example is given to illustrate our results.