一种具有弹性服务质量要求的多业务最优接纳控制策略

随着无线网络所支持的业务种类的增加和具有弹性服务质量要求的业务的大量出现，与服务质量保证密切相关的呼叫接纳控制问题成了近年来无线网络研究的热点之一。本文研究了基于准马尔可夫决策过程方法的多业务最优呼叫接纳控制问题。根据业务的特点，首先引入了带宽分配满意度函数和收益率函数，在此基础上，提出了基于带宽分配满意度的最优带宽分配算法和基于准马尔可夫决策过程方法的最优呼叫接纳控制策略。计算结果表明，本文方案能够在对各类业务的呼叫阻塞率进行适当权衡的前提下，进一步提高网络的期望收益率和期望带宽利用率，同时满足了各类业务的最低服务质量要求。     

Quadratic control of stochastic hybrid systems with renewal transitions

We study the quadratic control of a class of stochastic hybrid systems with linear continuous dynamics for which the lengths of time that the system stays in each mode are independent random variables with given probability distribution functions. We derive a condition for finding the optimal feedback policy that minimizes a discounted infinite horizon cost. We show that the optimal cost is the solution to a set of differential equations with unknown boundary conditions. Furthermore, we provide a recursive algorithm for computing the optimal cost and the optimal feedback policy. The applicability of our result is illustrated through a numerical example, motivated by stochastic gene regulation in biology.     

Towards subject independent continuous sign language recognition: A segment and merge approach

This paper presents a segment-based probabilistic approach to robustly recognize continuous sign language sentences. The recognition strategy is based on a two-layer conditional random field (CRF) model, where the lower layer processes the component channels and provides outputs to the upper layer for sign recognition. The continuously signed sentences are first segmented, and the sub-segments are labeled SIGN or ME (movement epenthesis) by a Bayesian network (BN) which fuses the outputs of independent CRF and support vector machine (SVM) classifiers. The sub-segments labeled as ME are discarded and the remaining SIGN sub-segments are merged and recognized by the two-layer CRF classifier; for this we have proposed a new algorithm based on the semi-Markov CRF decoding scheme. With eight signers, we obtained a recall rate of 95.7% and a precision of 96.6% for unseen samples from seen signers, and a recall rate of 86.6% and a precision of 89.9% for unseen signers.     

Reinforcement learning on explicitly specified time scales

In recent years hierarchical concepts of temporal abstraction have been integrated in the reinforcement learning framework to improve scalability. However, existing approaches are limited to domains where a decomposition into subtasks is known a priori. In this article we propose the concept of explicitly selecting time scale related abstract actions if no subgoal related abstract actions are available. This concept is realised with multi-step actions on different time scales that are combined in one single action set. We exploit the special structure of the action set in the MSA-Q-learning algorithm. This approach is suited for learning optimal policies in unstructured domains where a decomposition into subtasks is not known in advance or does not exist at all. By learning different explicitly specified time scales simultaneously, we achieve a considerable improvement of learning speed, which we demonstrate on several benchmark problems.     

Iterative convergence of passage-time densities in semi-Markov performance models

Passage-time densities are important for the detailed performance analysis of distributed computer and communicating systems. We provide a proof and demonstration of a practical iterative algorithm for extracting complete passage-time densities from expressive semi-Markov systems. We end by showing its application to a distributed web-server cluster model of 15.9 million states.     

求解α本征值的中子噪声分析随机模拟方法研究

瞬发中子常数α本征值对核装置的设计和安全分析具有重要意义。为解决α本征值蒙特卡罗计算时存在的问题,提高计算精度,本文结合随机模拟特点,借鉴反应堆噪声分析实验测量的原理,提出了基于噪声分析模拟的α本征值随机计算方法。首先,采用基于广义半马尔科夫过程的离散事件蒙特卡罗模拟方法来模拟中子随机场的演化过程,得到了探测器处中子噪声序列。其次,利用参数自适应计算的中子噪声分析Rossi-α方法、方差-平均值比法、零概率法以及互相关分析法,对中子噪声序列分析,得到了α值。几种方法计算得到的α值相近,最大相对偏差为7.9%,表明求解α本征值的中子噪声分析随机模拟方法是可行的。     

Hierarchical power management of a system with autonomously power-managed components using reinforcement learning

This paper presents a hierarchical dynamic power management (DPM) framework based on reinforcement learning (RL) technique, which aims at power savings in a computer system with multiple I/O devices running a number of heterogeneous applications. The proposed framework interacts with the CPU scheduler to perform effective application-level scheduling, thereby enabling further power savings. Moreover, it considers non-stationary workloads and differentiates between the service request generation rates of various software application. The online adaptive DPM technique consists of two layers: component-level local power manager and system-level global power manager. The component-level PM policy is pre-specified and fixed whereas the system-level PM employs temporal difference learning on semi-Markov decision process as the model-free RL technique, and it is specifically optimized for a heterogeneous application pool. Experiments show that the proposed approach considerably enhances power savings while maintaining good performance levels. In comparison with other reference systems, the proposed RL-based DPM approach, further enhances power savings, performs well under various workloads, can simultaneously consider power and performance, and achieves wide and deep power-performance tradeoff curves. Experiments conducted with multiple service providers confirm that up to 63% maximum energy saving per service provider can be achieved.     

基于UGF和Semi-Markov方法的反应堆泵机组多状态可靠性分析

将通用发生函数(UGF)与半马尔可夫过程(Semi-Markov Process)相结合,对反应堆泵机组进行多状态可靠性分析。给出多状态系统可靠性分析的UGF算法模型,推导多状态设备性能状态的Semi-Markov过程概率表达式,定义设备、系统性能值的定量描述方法。以性能参数是否满足需求值做为系统成功与失效的判据,对比分析反应堆泵机组在需求性能条件下的多状态可用度与2状态可用度结果,并给出系统在任务周期内的平均性能值。结果表明,该方法能够定量分析部分失效对系统可靠性的影响,降低传统的2态可靠性分析方法产生的不必要的保守程度。     

Time-redundant system reliability under randomly constrained time resources

The paper considers the time-redundant system where the system total task is a sequence of n phases and the total task must be executed during constrained time. For every phase, there is its own server, which executes the phase task during randomly distributed time. The server is not perfectly reliable and two types of failure (“open” and “closed”) are possible. Redundant servers may be used in any phase. The time–probability characteristics are introduced for any task, based on which the system reliability is treated as a probability that the system total task will be correctly completed during a corresponding time resource, which also may be randomly distributed. The adequate model is presented and a semi-Markov process is used as a mathematical technique. The closed-form solution was derived based on an acyclic Semi-Markov process. The numerical example of the elaborated approach is presented.