认知用户基于预测的动态频谱接入算法

频谱接入技术的关键是解决认知用户如何选择合适的空闲信道以及认知用户间如何实现频谱共享.在公共控制信道较难获得的情况下,基于部分可观测Markov决策过程(POMDP)的频谱预测算法,可以显著地提高系统的吞吐量;由于认知用户之间缺少接入信息的交换,使得多个认知用户在同一时隙接入同一信道的冲突概率较大.针对多用户竞争冲突问题,通过引入竞争反馈的偏差因子,并利用均衡的混合信道选择策略对认知无线网络动态频谱接入过程进行研究.通过大量仿真对认知用户的吞吐量和频谱利用率以及碰撞率进行分析,研究表明,均衡的混合信道选择策略可以有效地提高系统的吞吐量及频谱利用率.     

基于内部信念状态POMDP模型在用户兴趣获取中的应用

随着智能人机界面的发展 ,用户兴趣获取的研究被引起广泛的重视 ,并有大量的学习模型 .本文根据有模型的POMDP结构 ,提出了基于内部信念状态的 POMDP结构 ,进行用户兴趣获取 ,使得系统既能够结合领域专家的知识 ,在模型中应用部分的用户兴趣结构 .又采用类似有模型的 POMDP结构 ,从而能够直接或者通过简单的转化 ,使用已有的大量成熟算法 ,规划用户兴趣获取的行为 .这里 ,本文研究了模型中参数的设置对用户兴趣获取的影响 .     

口语对话系统的POMDP模型及求解

许多口语对话系统已进入实用阶段，但一直没有很好的对话管理模型，把对话管理看做随机优化问题，用马尔科夫决策过程（MDP）来建模是最近出现的方向，但是对话状态的不确定性使MDP不能很好地反映对话模型，提出了一种新的基于部分可观察MDP（POMDP）的口语对话系统模型，用部分可观察特性来处理不确定问题，由于精确求解算法的局限性，考察了许多启发式近似算法在该模型中的话用性，并改进了部分算法，如对于格点近似算法，提出了两种基于模拟点的格点选择方法。     

基于策略迭代和值迭代的POMDP算法

部分可观察Markov决策过程是通过引入信念状态空间将非Markov链问题转化为Markov链问题来求解,其描述真实世界的特性使它成为研究随机决策过程的重要分支.介绍了部分可观察Markov决策过程的基本原理和决策过程,提出一种基于策略迭代和值迭代的部分可观察Markov决策算法,该算法利用线性规划和动态规划的思想,解决当信念状态空间较大时出现的"维数灾"问题,得到Markov决策的逼近最优解.实验数据表明该算法是可行的和有效的.     

POMDP-APF：一种基于POMDP模型的APF无人机路径规划策略

针对无人机在路径规划过程中会遇到静态或者动态的障碍物,从而导致路径规划失败的问题,提出一种基于部分可观测马尔可夫决策过程（partially observable markov decision process,POMDP）模型的人工势场（artificial potential field,APF）无人机路径规划策略（POMDP-APF）。首先使用传感器获得的障碍物信息结合POMDP模型预测障碍物的未来位置,为无人机的路径规划做准备;其次,提出一种新的基于障碍物的正方体外接球的模型,保障无人机在路径规划过程中的安全性;最后,结合改进的APF算法实现无人机的路径规划。仿真结果表明,POMDP-APF策略在无人机实时路径规划中具有良好的可行性和有效性,使无人机能够有效避开障碍物,同时路径长度以及耗费时间更短。     

Optimal spectrum allocation of primary users in light-handed cognitive radio networks

In traditional cognitive radio (CR) network,secondary users (SUs) are always assumed to obey the rule of “introducing no interference to the primary users (PUs) ”.However,this assumption may be not rea...     

一种动态不确定环境中机器人路径规划方法

针对动态不确定环境下的机器人路径规划问题,将部分可观察马尔可夫决策过程（POMDP）与人工势场法（APF）的优点相结合,提出一种新的机器人路径规划方法。该方法充分考虑了实际环境中信息的部分可观测性,并且利用APF无需大量计算的优点指导POMDP算法的奖赏值设定,以提高POMDP算法的决策效率。仿真实验表明,所提出的算法拥有较高的搜索效率,能够快速地到达目标点。     

预测行人运动的服务机器人POMDP 导航

为提高室内动态环境下服务机器人对行人的自然避让能力,对人的运动轨迹模式进行建模,在此基础 上引入了将行人运动长、短期预测结合起来的方法．为适应传感器噪声及网络延迟等因素所造成的感知—控制回路 中的多源不确定性,将人与机器人的相对位置关系建模为部分可观的马尔可夫状态．采用部分可观的马尔可夫决策 过程（POMDP）进行多源不确定性下的概率决策,协调控制机器人全局路径规划、反应式运动及速度控制等行为模 块．实验结果验证,它能够实现提前避碰的安全导航,因避免反复的曲折与徘徊运动而提高了机器人导航效率．     

Partially Observable Markov Decision Process Approximations for Adaptive Sensing

Adaptive sensing involves actively managing sensor resources to achieve a sensing task, such as object detection, classification, and tracking, and represents a promising direction for new applications of discrete event system methods. We describe an approach to adaptive sensing based on approximately solving a partially observable Markov decision process (POMDP) formulation of the problem. Such approximations are necessary because of the very large state space involved in practical adaptive sensing problems, precluding exact computation of optimal solutions. We review the theory of POMDPs and show how the theory applies to adaptive sensing problems. We then describe a variety of approximation methods, with examples to illustrate their application in adaptive sensing. The examples also demonstrate the gains that are possible from nonmyopic methods relative to myopic methods, and highlight some insights into the dependence of such gains on the sensing resources and environment.

Integrated maintenance and operations decision making with imperfect degradation state observations

In highly flexible and integrated manufacturing systems, such as semiconductor fabs, strong interactions between the equipment condition, operations executed on the various machines and the outgoing product quality necessitate integrated decision making in the domains of maintenance scheduling and production operations. Furthermore, in highly complex manufacturing equipment, the underlying condition is not directly observable and can only be inferred probabilistically from the available sensor readings. In order to deal with interactions between maintenance and production operations in Flexible Manufacturing Systems (FMSs) in which equipment conditions are not perfectly observable, we propose in this paper a decision-making method based on a Partially Observable Markov Decision Processes (POMDP's), yielding an integrated policy in the realms of maintenance scheduling and production sequencing. Optimization was pursued using a metaheuristic method that used the results of discrete-event simulations of the underlying manufacturing system. The new approach is demonstrated in simulations of a generic semiconductor manufacturing cluster tool. The results showed that, regardless of uncertainties in the knowledge of actual equipment conditions, jointly making maintenance and production sequencing decisions consistently outperforms the current practice of making these decisions separately.