Almost Sure Stabilization for Feedback Controls of Regime-Switching Linear Systems With a Hidden Markov Chain

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a hidden Markov chain. In the previous investigation on this class of problems, averaging criteria were used, which provides only the system behavior in some expectation sense. A closer scrutiny of the system behavior necessarily requires the consideration of sample path properties. Different from previous work on stabilization of adaptive controlled systems with a hidden Markov chain, where average criteria were considered, this work focuses on the almost sure stabilization or sample path stabilization of the underlying processes. Under simple conditions, it is shown that as long as the feedback controls have linear growth in the continuous component, the resulting process is regular. Moreover, by appropriate choice of the Lyapunov functions, it is shown that the adaptive system is stabilizable almost surely. As a by-product, it is also established that the controlled process is positive recurrent.     

基于小波分析和HMM的语音识别模型建立与仿真

利用隐马尔可夫模型HMM优异的时序建模能力及小波变换可以对信号进行多尺度分析并有效提取信号的局部信息的特点,建立了混合语音识别模型.在语音信号的识别过程中考虑到了信号的非平稳性,采用并行识别的方法分别获取分类信息,根据混合模型的识别算法做出识别决策,减小了系统对环境的依赖性,提高了其自适应能力.仿真实验结果表明,混合模型识别结果比单一HMM模型或小波模型识别结果更佳,提高了整体的识别速度和识别率.     

Partially‐observable stochastic hybrid systems (poshss) state estimation and optimal control

This paper discusses the state estimation and optimal control problem of a class of partially‐observable stochastic hybrid systems (POSHS). The POSHS has interacting continuous and discrete dynamics with uncertainties. The continuous dynamics are given by a Markov‐jump linear system and the discrete dynamics are defined by a Markov chain whose transition probabilities are dependent on the continuous state via guard conditions. The only information available to the controller are noisy measurements of the continuous state. To solve the optimal control problem, a separable control scheme is applied: the controller estimates the continuous and discrete states of the POSHS using noisy measurements and computes the optimal control input from the state estimates. Since computing both optimal state estimates and optimal control inputs are intractable, this paper proposes computationally efficient algorithms to solve this problem numerically. The proposed hybrid estimation algorithm is able to handle state‐dependent Markov transitions and compute Gaussian‐ mixture distributions as the state estimates. With the computed state estimates, a reinforcement learning algorithm defined on a function space is proposed. This approach is based on Monte Carlo sampling and integration on a function space containing all the probability distributions of the hybrid state estimates. Finally, the proposed algorithm is tested via numerical simulations.     

Asymptotically optimal controls of hybrid linear quadratic regulators in discrete time

This work develops asymptotically optimal controls for a class of discrete-time hybrid systems involving singularly perturbed Markov chains having weak and strong interactions. The state space of the underlying Markov chain is decomposed into a number of recurrent classes and a group of transient states. Using a hierarchical control approach, by aggregating the states in each recurrent class into a single state, a continuous-time quadratic limit control problem in which the resulting limit Markov chain has much smaller state space is derived. Using the optimal control for the limit problem, a control for the original problem is constructed, which is shown to be nearly optimal. Finally, a numerical example is given to demonstrate the effectiveness of the approximation scheme.     

A Proposed Hybrid Recurrent Neural Control System for Two Co-operating Robots

This paper discusses a model refernce adaptive (MRAC) position/force controller using proposed neural networks for two co-operating planar robots. The proposed neural network is a recurrent hybrid network. The recurrent networks have feedback connections and thus an inherent memory for dynamics, which makes them suitable for representing dynamic systems. A feature of the networks adopted is their hybrid hidden layer, which includes both linear and nonlinear neurons. On the other hand, the results of the case of a single robot under position control alone are presented for comparison. The results presented show the superior ability of the proposed neural network based model reference adaptive control scheme at adapting to changes in the dynamics parameters of robots.     

基于扩展状态观测器的随机隐Markov正跳变系统有限时间异步控制

针对一类含不匹配扰动的随机隐Markov跳变系统, 本文研究了基于扩展状态观测器(ESO)的有限时间异步 控制问题. 首先, 引入一组扩展变量将隐Markov跳变系统转换成一组新的随机扩展系统, 补偿不匹配扰动对系统控 制输出的影响. 基于Lyapunov–Krasovskii泛函方法, 给出使得基于ESO的闭环随机隐Markov增广跳变系统是正系 统, 且有限时间有界的充分条件. 进而得到直接求解观测器增益和控制器增益的线性矩阵不等式. 最后, 通过仿真结 果验证了本文所设计的异步状态反馈控制器和观测器的有效性和可行性.     

Control of discrete-time HMM partially observed under fractional Gaussian noises

A discrete-time control problem of a finite-state hidden Markov chain partially observed in a fractional Gaussian process is discussed using filtering. The control problem is then recast as a separated problem with information variables given by the unnormalized conditional probabilities of the whole path of the hidden Markov chain. A dynamic programming result and a minimum principle are obtained.     

Adaptive control of linear Markov jump systems

Stabilization of linear Markov jump systems via adaptive control is considered in this paper. The switching law is assumed to be unobservable Markov process. A sufficient condition is obtained for the stochastic stabilizability based on common quadratic Lyapunov functions (QLFs). The constructive proof provides a method to construct a sampling adaptive stabilizer. An example is used to describe the design of adaptive control, which stabilizes the system.     

Intelligent adaptive model-based control of robotic dynamic systems with a hybrid fuzzy-neural approach

We describe in this paper a new method for adaptive model-based control of robotic dynamic systems using a new hybrid fuzzy-neural approach. Intelligent control of robotic systems is a difficult problem because the dynamics of these systems is highly nonlinear. We describe an intelligent system for controlling robot manipulators to illustrate our fuzzy-neural hybrid approach for adaptive control. We use a new fuzzy inference system for reasoning with multiple differential equations for model selection based on the relevant parameters for the problem. In this case, the fractal dimension of a time series of measured values of the variables is used as a selection parameter. We use neural networks for identification and control of robotic dynamic systems. We also compare our hybrid fuzzy-neural approach with conventional fuzzy control to show the advantages of the proposed method for control.     

Bayesian posterior mean estimates for Poisson hidden Markov models

This paper focuses on the Bayesian posterior mean estimates (or Bayes’ estimate) of the parameter set of Poisson hidden Markov models in which the observation sequence is generated by a Poisson distribution whose parameter depends on the underlining discrete-time time-homogeneous Markov chain. Although the most commonly used procedures for obtaining parameter estimates for hidden Markov models are versions of the expectation maximization and Markov chain Monte Carlo approaches, this paper exhibits an algorithm for calculating the exact posterior mean estimates which, although still cumbersome, has polynomial rather than exponential complexity, and is a feasible alternative for use with small scale models and data sets. This paper also shows simulation results, comparing the posterior mean estimates obtained by this algorithm and the maximum likelihood estimates obtained by expectation maximization approach.     

On nearly optimal controls of hybrid LQG problems

The paper is concerned with hybrid control of a class of linear quadratic Gaussian (LQG) systems modulated by a finite-state Markov chain. It develops approximation schemes for systems involving singularly perturbed Markov chains with weak and strong interactions, which are useful for natural time-scale separation and for large-scale Markovian systems. The presentation encompasses three cases: the recurrent Markov chains, inclusion of transient states, and inclusion of absorbing states. Near optimality of the approximation schemes is obtained, and a numerical example is presented. Computational results indicate our approximation schemes perform well     

基于特征模型自适应方法的一类最小相位非线性系统稳定性分析

在特征建模理论中,由全系数自适应控制器组成的闭环系统是一个非常复杂的混合系统,采用传统自适应框架难以进行分析,因此,稳定性分析一直是该领域的一个难点.本文以一类最小相位、相对阶为2的单输入单输出(SISO)高阶非线性系统为例,通过一种新的特征建模方法,把高阶混合系统变换为一个含有稳定未建模误差的、参数有界慢时变的采样间接自适应控制问题,并利用基于欧拉近似离散化模型的采样系统稳定性分析方法进行了系统分析.该方法可进一步推广到任意相对阶的SISO或多输入多输出(MIMO)系统甚至无限维最小相位系统中去.     

一类具有Markov跳跃参数的不确定混合线性时滞系统鲁棒稳定性研究

讨论了一类具有Markov跳跃参数的不确定混合线性时滞系统的鲁棒稳定性问题．分别给出了非匹配条件下不确定部分范数上界已知时使混合线性系统以概率1渐近稳定的充分条件,和匹配条件下不确定部分范数上界未知时同样可以实现混合系统以概率1渐近稳定的鲁棒自适应控制设计方案．文章研究结果表明,此控制方案对混合线性时滞系统的不确定部分是有效的．     

Masquerade detection using profile hidden Markov models

In this paper, we consider the problem of masquerade detection, based on user-issued UNIX commands. We present a novel detection technique based on profile hidden Markov models (PHMMs). For comparison purposes, we implement an existing modeling technique based on hidden Markov models (HMMs). We compare these approaches and show that, in general, our PHMM technique is competitive with HMMs. However, the standard test data set lacks positional information. We conjecture that such positional information would give our PHMM a significant advantage over HMM-based detection. To lend credence to this conjecture, we generate a simulated data set that includes positional information. Based on this simulated data, experimental results show that our PHMM-based approach outperforms other techniques when limited training data is available.     

分散混杂自适应模糊H∞控制器及其在自动化公路系统中的应用

本文针对一类SISO不确定非线性大系统,提出了一种混杂间接和直接自适应分散模糊H∞控制器.通过组合模糊系统和H∞跟踪技术开发的分散自适应模糊控制算法避免了控制设计中含有的符号函数.两种自适应模糊控制器的组合消除了它们各自均不能够同时融合被控对象知识与控制知识的局限.闭环大系统被证明是稳定的,且具有H∞跟踪性能.该算法应用于自动化公路系统中车辆的纵向跟随控制,仿真结果表明混杂自适应模糊H∞控制系统的跟踪性能更好而相应的控制幅值却更小.     

Markov切换系统的随机稳定性分析

根据混合系统离散状态的动态行为和Markov链的状态也是离散的特点,提出了一类离散状态的动态行为是Markov链的混合系统。与传统的混合系统相比,这类系统能够刻画出混合系统离散动态行为的随机性,可以用来描述系统受到外界环境因素制约和内部突发事件等随机因素影响而发生变化的动态行为。根据动态系统的稳定性定义以及随机过程理论,给出了Markov线性切换系统的随机稳定性定义,并且分析了Markov线性切换系统的随机稳定性问题,给出了判定随机稳定性的充分必要条件。     

无线多跳网络结合功率控制的物理载波侦听分析模型

载波侦听阈值的选取对无线多跳网络MAC层的协议性能有着重要影响.已有研究中忽略了确认报文、累积干扰在不同的载波侦听阈值下对系统性能的影响,并在分析中夸大了隐藏节点所导致的信道冲突.针对上述问题,提出了一种结合功率控制的物理载波侦听分析模型.该模型以网络整体性能为优化目标,对全网的累积干扰与空间复用度进行分析,给出节点的平均信道容量.此外,对信道状态建立马尔可夫链模型,提出了4类载波侦听范围内发生的信道冲突以及两类全网累积干扰所引发的信道冲突.分析上述各类冲突对信道利用率的影响,并结合节点的平均信道容量给出优化的载波侦听阈值以及传输功率.与已有研究相比,模型中明确分析确认报文、累积干扰以及隐藏节点对信道冲突的影响.分析结果表明,如果不考虑上诉因素将无法获得最优的载波侦听范围,并且会导致网络性能的下降.     

基于马氏决策过程模型的动态系统学习控制:研究前沿与展望

基于马氏决策过程(Markov decision process, MDP)的动态系统学习控制是近年来一个涉及机器学习、控制理论和运筹学等多个学科的交叉研究方向, 其主要目标是实现系统在模型复杂或者不确定等条件下基于数据驱动的多阶段优化控制. 本文对基于MDP的动态系统学习控制理论、算法与应用的发展前沿进行综述,重点讨论增强学习(Reinforcement learning, RL)与近似动态规划(Approximate dynamic programming, ADP)理论与方法的研究进展,其中包括时域差值学习理论、求解连续状态与行为空间MDP的值函数逼近方法、 直接策略搜索与近似策略迭代、自适应评价设计算法等,最后对相关研究领域的应用及发展趋势进行分析和探讨.     

时延变化的滞后系统的HMRAC的设计

首次把混合自适应算法应用到时延变化的滞后系统中,实现了对时延变化的滞后系统的混合自适应控制,增强了该系统的鲁棒性.解决了以往由于时延变化而引起的滞后系统不稳定问题.     

Identification of switched Markov autoregressive eXogenous systems with hidden switching state

Identification of the Switched Markov Autoregressive eXogenous (ARX) systems is considered in this paper. With a Markov chain model governing the evolution of the hidden switching state, a Switched Markov ARX System (SMARX) is formulated and a solution strategy is proposed. The Expectation–Maximization (EM) algorithm is employed in the identification of the SMARX systems in which both a Hidden Markov Model (HMM) for the discrete-valued switching dynamics and local ARX models for continuous dynamics are estimated. Through the comparison between the proposed method and previous switched ARX system identification methods, it is shown that by modeling both the switching and continuous dynamics, the accuracy of the identification results can, to various extent, be improved.