Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

In this paper we discuss an online algorithm based on policy iteration for learning the continuous-time (CT) optimal control solution with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real-time the solution to the optimal control design HJ equation. This method finds in real-time suitable approximations of both the optimal cost and the optimal control policy, while also guaranteeing closed-loop stability. We present an online adaptive algorithm implemented as an actor/critic structure which involves simultaneous continuous-time adaptation of both actor and critic neural networks. We call this ‘synchronous’ policy iteration. A persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for both critic and actor networks, with extra nonstandard terms in the actor tuning law being required to guarantee closed-loop dynamical stability. The convergence to the optimal controller is proven, and the stability of the system is also guaranteed. Simulation examples show the effectiveness of the new algorithm.     

Vulnerability analysis of networks to detect multiphase attacks using the actor-based language Rebeca

Increasing use of networks and their complexity make the task of security analysis more and more complicated. Accordingly, automatic verification approaches have received more attention recently. In this paper, we investigate applying of an actor-based language based on reactive objects for analyzing a network environment communicating via Transport Protocol Layer (TCP). The formal foundation of the language and available tools for model checking provide us with formal verification support. Having the model of a typical network including client and server, we show how an attacker may combine simple attacks to construct a complex multiphase attack. We use Rebeca language to model the network of hosts and its model checker to find counter-examples as violations of security of the system. Some simple attacks have been modeled in previous works in this area, here we detect these simple attacks in our model and then verify the model to find more complex attacks which may include simpler attacks as their steps. We choose Rebeca because of its powerful yet simple actor-based paradigm in modeling concurrent and distributed systems. As the real network environment is asynchronous and event-based, Rebeca can be utilized to specify and verify the asynchronous systems, including network protocols.     

Use Case的分析设计和实现

本文介绍了分析,设计和实现ｕｓｅ ｃａｓｅ的过程,重点阐述ｕｓｅ ｃａｓｅ的描述,细化和实现的有关细节 。     

校园网软件开发的系统分析

成功地开发出一套软件应用系统,其难点往往不在于复杂的编码,而在于编码前所做的一系列工作,即从用户的需求中找出系统的需求,并把这种需求变成系统编程人员和用户都易于理解的统一的信息。采用目前比较流行的面向对象的UML工具,对一个具体的实例进行系统分析,希望通过实例分析可以较快地理解和掌握使用UML建立系统分析的方法。     

基于文本挖掘获取相关角色授权问题研究

如何满足人们合理的个性化需求,实现信息资源的按需分配为目的,是数字化校园建设的核心。在数字校园中,不同群体提供的信息服务是不同的。因此,存在着对信息服务对象的甄别的问题。本文从一个特殊的群体--行政管理群体出发,结合现有的电子政务系统,对电子文本进行挖掘获取相关角色及授权,实现角色及授权管理的时效性、真实性,杜绝了人为因素的影响。     

Semi-Markov adaptive critic heuristics with application to airline revenue management

The adaptive critic heuristic has been a popular algorithm in reinforcement learning(RL) and approximate dynamic programming(ADP) alike.It is one of the first RL and ADP algorithms.RL and ADP algorithms are particularly useful for solving Markov decision processes(MDPs) that suffer from the curses of dimensionality and modeling.Many real-world problems,however,tend to be semi-Markov decision processes(SMDPs) in which the time spent in each transition of the underlying Markov chains is itself a random variable.Unfortunately for the average reward case,unlike the discounted reward case,the MDP does not have an easy extension to the SMDP.Examples of SMDPs can be found in the area of supply chain management,maintenance management,and airline revenue management.In this paper,we propose an adaptive critic heuristic for the SMDP under the long-run average reward criterion.We present the convergence analysis of the algorithm which shows that under certain mild conditions,which can be ensured within a simulator,the algorithm converges to an optimal solution with probability 1.We test the algorithm extensively on a problem of airline revenue management in which the manager has to set prices for airline tickets over the booking horizon.The problem has a large scale,suffering from the curse of dimensionality,and hence it is difficult to solve it via classical methods of dynamic programming.Our numerical results are encouraging and show that the algorithm outperforms an existing heuristic used widely in the airline industry.     

基于随机对策的团队CGA学习

分布式虚拟环境中,个体CGA通过学习来适应环境和团队中其他CGA,从而增强团队求解问题的能力,提高团队的适应性和鲁棒性。当基于随机对策的团队CGA学习出现多个平衡解时,必须解决平衡的选择问题。提出了一种团队CGA学习方法TCCLA,该方法把团队CGA的学习分为两个层次:管理成员的学习和非管理成员的学习,团队中所有成员根据偏好选择最优行为,非管理成员在管理成员的引导下学习到最优平衡,解决了平衡的选择问题,改进了IPL算法,实验表明TCCLA算法的高效性。     

基于角色的Web Services访问控制系统

首先对Web Services的安全技术和访问控制的研究情况进行简单回顾,介绍XACML(eXtensibleAcces s Control Markup language)中的常用术语,然后介绍了一个Web Services的面向服务的基于角色的访问控制系统和安全结构模型,从服务方面和属性方面加强对服务的访问控制.在该体系结构模型中,使用SOAPProxy加强对Web Services的访问控制.通过设计安全cookies和安全SOAP消息,在Web Services上实现基于角色的访问控制.最后给出此系统有待解决的问题.     

Cinematic success, aesthetics, and economics: An exploratory recursive model.

Although the reputation of creative artists is based largely on the merit of their work, the latter can sometimes be assessed in several different ways that may not necessarily agree. This lack of evaluative consensus is perhaps most apparent in cinematic success; this can be judged by film critics (initial and final), movie awards (picture, dramatic, visual, technical, and music), and box office performance (including both first weekend and later gross). Previous research not only shows that these success criteria may not always agree, but also that the criteria may have distinct aesthetic and economic antecedents. However, because the success criteria emerge at distinct points across time, a recursive model can be developed that describes the relationships among the criteria as well as their differential dependence on the predictive factors most frequently identified in the literature. The model was constructed using a sample of 1006 English-language, live-action, feature-length narrative films released between 2000 and 2006. The resulting equations indicate the complexity of cinematic success. Nonetheless, overriding this complexity is the fundamental contrast between film as art and film as entertainment. (PsycINFO Database Record (c) 2010 APA, all rights reserved)     

对以典型推动创先争优活动深入开展的思考

注重典型引路,是创先争优活动的内在要求,也是推动工作的重要手段.我们在推动创先争优活动中,选树典型要突出时代性、广泛性、实效性,使先进典型具有丰富的内涵和强大的生命力.要紧密围绕推动科学发展、促进社会和谐、服务人民群众、加强基层组织等方面培树典型,用推动科学发展的成效来检验典型选树的实效.