Q-learning算法优化的SVDPP推荐算法

为进一步改善个性化推荐系统的推荐效果,通过使用强化学习方法对SVDPP算法进行优化,提出一种新的协同过滤推荐算法。考虑用户评分的时间效应,将推荐问题转化为马尔科夫决策过程。在此基础上,利用Q-learning算法构建融合时间戳信息的用户评分优化模型,同时通过预测评分取整填充和优化边界补全方法预测缺失值,以解决数据稀疏性问题。实验结果显示,该算法的均方根误差较SVDPP算法降低了0.005 6,表明融合时间戳并采用强化学习方法进行推荐性能优化是可行的。     

基于深度强化学习的深圳市急救车调度算法

在院前急救领域中,急救反应时间是指患者拨打急救电话后,急救车到达现场的时间。传统急救车调度算法未全面考虑急救环境的动态性和复杂性因素,导致模型优化的急救反应时间与实际情况存在偏差。将急救车调度问题建模成马尔科夫决策过程,构建基于深度强化学习的急救车调度算法。以多层感知机作为评分网络结构,通过将急救站的动态信息映射为各个急救站的得分,确定急救车被调往各急救站的概率。同时,结合急救车调度的动态决策特点,利用强化学习中演员-评论家框架下的近端策略优化算法改进评分网络参数。在深圳市急救中心真实急救数据集上的实验结果表明,相比Fixed、DSM、MEXCLP等算法,该算法在每个急救事件中的急救反应时间平均缩短约80 s,并且在10 min内急救车的平均到达比例为36.5%,能够实时地将急救车调度到合适的急救站。     

车载Ad Hoc网络中基于移动网关的数据传输

由于车载Ad hoc网络拓扑的动态变化及车载节点的快速移动,应用现有传输方法在其上进行Internet接入点向移动车辆(Infrastructure-to-Vehicle,I2V)数据传输时成功率较低,而且传输延迟高、延迟抖动大.为了解决这一问题,文中利用公交车路线固定、运行特征可预测、节点及线路分布稠密等特性,将公交车作为移动网关(Mobile Gateway,MG),提出了一种新的基于MG转发的I2V数据传输方法(Mobile Gateway based Forwarding,MGF).文中首先将公路网模型化为状态-空间图,再运用马尔可夫决策方法建立了一种基于MG转发的I2V数据传输优化模型,然后通过对模型求解得出I2V数据传输的最优转发决策,最优转发决策指的就是每个状态下对应的最优动作序列,最后在目的车辆行驶轨迹上选取满足传输成功率阈值,并使I2V传输延迟最小的路口节点作为数据包与目的车辆的最优汇聚节点,即目标节点.应用MGF方法,MG节点将以最优概率转发序列向目标节点转发数据包.文中利用模拟平台对MGF方法的传输性能进行了评估,结果表明该方法在满足传输成功率阈值前提下,能够获得最小传输延迟期望.理论分析同样也证明了该方法的有效性.     

Non-myopic vehicle and route selection in dynamic DARP with travel time and workload objectives

In dial-a-ride problems, a fleet of n vehicles is routed to transport people between pick-up and delivery locations. We consider an elementary version of the problem where trip requests arrive in time and require an immediate vehicle assignment (which triggers an appropriate route update of the selected vehicle). In this context, a relatively general objective can be stated as a weighted sum of the system's effort and the customers' inconvenience. However, optimizing almost any objective in this immensely complex stochastic system is prohibitively difficult. Thus the earlier work has largely resorted to heuristic cost functions that arise, e.g., from the corresponding static systems. By using the framework of Markov decision processes and the classical M/M/1 queue as a highly abstract model for a single vehicle, we explain why certain intuitive cost functions indeed give satisfactory results in the dynamic system, and also give an explicit interpretation of different components appearing in a general cost function. The resulting family of heuristic control policies is demonstrated to offer a desired type of performance thus justifying the assumed analogy between a multi-queue and dial-a-ride systems.     

~(99)Tc~m-MDP、~(99)Tc~m(V)-DMSA和~(99)Tc4~m-Citrate显像对骨组织良恶性疾病鉴别诊断的对比研究

为评价99Tcm(V)二巯基丁二酸钠(DMSA)显像和99Tcm枸缘酸(Citrate)显像在骨转移癌和骨及骨关节炎症诊断中的意义,对骨转移癌患者和骨及骨关节炎症患者各18例分别进行99Tcm亚甲基二膦酸(MDP)、99Tcm(V)DMSA和99TcmCitrate全身显像,并比较了它们的显像结果。18例经病理学、CT或MRI证实有骨转移癌的患者,99TcmMDP显像共检出64个病灶,99Tcm(V)DMSA显像显示在与99TcmMDP显像相同部位同检出49个病灶,而99TcmCitrate显像仅检出1个病灶。18例经细菌学、CT或MRI证实的骨及骨关节炎症患者,99TcmMDP显像共检出22个病灶,99Tcm(V)DMSA显像显示在与99Tcm MDP显像相同部位同检出17个病灶,99TcmCitrate显像检出16个病灶。本组病例99Tcm(V)DMSA显像诊断骨转移癌的灵敏度为76.56%,特异性为22.73%; 99TcmCitrate显像诊断骨转移癌的灵敏度仅为1.56%,特异性为27.27%。99Tcm(V)DMSA显像诊断骨及骨关节炎症的灵敏度为77.27%,特异性为23.44%;99TcmCitrate显像诊断骨及骨关节炎症的灵敏度为72.73%,特异性为98.44%。以上结果表明,99Tcm(V)DMSA显像在诊断骨转移癌和骨组织炎症时应该慎重,因为它不能区分骨组织的良恶性病变性质,其对骨组织的良恶性病变性质的鉴别诊断应排除外骨组织炎症、骨折等骨组织良性病变的干扰。而99TcmCitrate     

一种多MODEM卡的设计方案

该文从MODEM的结构出发,分析了各厂家MODEM芯片组的特点,以Conexant(Rockwell)公司生产的L27型微控制器作为核心控制芯片,调制解调部分采用该公司的RC56D系列MODEM芯片组,以复用的方式研究出了一种四路多MODEM卡的设计方案。     

非时齐无界报酬马氏决策规划

论文讨论折扣因子依赖于(i,a)的非时齐马氏决策规划,提出了两组报酬无界的假设,证明了时齐正、负动态规划中的结论在这里都成立。最后讨论了此模型的最优策略性质。     

Lookahead actions in dispatching to parallel queues

Applying the first policy iteration (FPI) to any static dispatching (task assignment) policy yields a new improved dynamic policy that takes into account the particular cost structure and the expected future arrivals. However, it is generally hard to go beyond that due to the complex state space and the resulting difficulty in computing the value function for a dynamic policy. For example, applying FPI to identical FCFS servers with Bernoulli split gives the Least-Work-Left (LWL) policy, for which no closed-form value function is known. In fact, LWL with identical servers is equivalent to an M/G/k queue, the performance measures of which have remained as open problems. The situation gets even more complicated with heterogeneous servers. In this paper, we take an intermediate approach and consider lookahead actions that concern not only the current job but also the job arriving next, after which a basic (static) policy is assumed to take over. This is important as the benefits from some decisions can only be reaped with appropriate subsequent actions. The lookahead enables sound estimates also for marginal admission costs, e.g., with respect to LWL. The superior performance of the new near-optimal dispatching policies is demonstrated numerically.     

Synthesis and Application of Methyl N,O-Hydroxylamine Muramyl Peptides

The innate immune system's interaction with bacterial cells plays a pivotal role in a variety of human diseases. Carbohydrate units derived from a component of bacterial cell wall, peptidoglycan (PG), are known to stimulate an immune response. Nonetheless, access to modified late-stage peptidoglycan intermediates is limited due to their synthetic complexity. A method to rapidly functionalize PG fragments is needed to better understand the natural host–PG interactions. Here methyl N,O-hydroxylamine linkers are incorporated onto a synthetic PG derivative, muramyl dipeptide (MDP). The modification of MDP maintained the ability to stimulate a nuclear factor kappa-light-chain-enhancer of activated B cells (NF-κB) immune response dependent on the expression of nucleotide-binding oligomerization domain-containing protein 2 (Nod2). Intrigued by this modification's maintenance of biological activity, several applications were explored. Methyl N,O-hydroxylamine MDP was amendable to N-hydroxylsuccinimide (NHS) chemistry for bioconjugation to fluorophores as well as a self-assembled monolayer for Nod2 surface plasmon resonance analysis. Finally, linker incorporation was applicable to larger PG fragments, both enzymatically generated from Escherichia coli or chemically synthesized. This methodology provides rapid access to PG probes in one step and allows for the installation of a variety of chemical handles to advance the molecular understanding of PG and the innate immune system.     

Web service composition on IoT reliability test based on cross entropy

Web service has developed the managed IoT application to let connected devices easily and securely interact with cloud applications and other devices. As an important factor for web service, the reliability of web services refers to the probability of web service running success. For modeling web service composition, we should abstract the process of web service composition. Due to the diversity and complexity of web service composition, it is unlikely to do exhaustive testing. In order to improve the quality of web service composition test cases and find out which path leads to the greatest probability of service combination failure, heuristic test case generation method is adopted to obtain the optimal test path. First, the web service composition test is abstracted into the MDP model. The QoS of the web service composition is taken as the software test optimization goal, and the cross-entropy strategy is used to optimize the test case. The experimental results show that the test profile given by the cross-strategy is better than the random test strategy. Detect and exclude the same number of software defects. Cross-entropy strategy can significantly reduce the number of test cases, reduce test costs, and improve defect detection efficiency.