基于机器学习的认知无线网络优化策略

5G的发展带来了终端设备爆炸式增长的现象,使得频谱资源紧缺的问题越加严峻,认知无线网(cognitive radio,CR)的提出,被认为是提高频谱利用率的有效途径。认知无线网,融合了当代无线电通信技术、计算机技术、微电子学技术、软件无线电技术和现代信号处理技术等多学科之长,通过感知周围的电磁环境、学习及理解等方式,自主为用户寻找到当前空闲的频谱,完成信息交互过程。针对频谱资源紧张的现状,为改善频谱分配,首先介绍了有关认知无线网络的概念及其特点,重点介绍了机器学习中遗传算法,强化学习和隐马尔可夫模型在认知无线网络中的应用,并展望了其在认知无线网络中的发展前景。机器学习算法的引入,实现了高效的频谱资源管理,有效地解决了无线频谱资源紧张的问题。     

提高强化学习速度的方法研究

强化学习一词出自于行为心理学,这门学科把学习看作为反复试验的过程,以便把环境的状态映射为动作。强化学习的这种特性必然增加智能系统的困难性,学习时间增长。强化学习学习速度较慢的原因是没有明确的监督信号。因此,强化学习系统在与环境交互时不得不采取反复试验的方法依靠外部评价信号来调整自己的行为。智能系统必然经过很长的学习过程。如何提高强化学习速度是一个最重要的研究问题。该文从几个方面来讨论提高强化学习速度的方法。     

一种基于Q学习的有限理性博弈模型及其应用

传统博弈理论模型建立在人的完全理性基础之上,难以切合实际。有限理性博弈则能够很好地描述实际问题。有限理性的博弈者参与到不完全信息博弈中,对博弈的规则、结构以及对手等博弈信息有一个逐渐适应和了解的过程,因此博弈应是动态进化的模型。针对这一问题,提出了一种基于Q学习算法的不完全信息博弈模型,根据Littman的最大最小原则建立了多指标体系下的策略选择概率分布;构建了Q学习与博弈融合的数学模型,使用Q学习机制来实现博弈模型的动态进化;最后将模型应用于两人追逐的仿真实验,结果表明所提出的模型能够很好地再现追逐情景。     

基于Q-学习算法的交通控制与诱导协同模式的在线选择

采用Q-学习算法实现了交通控制与诱导协同模式的在线选择。首先,采用Q-学习算法训练多智能体,根据多智能体内部的推理得到不同交通状态下的最优协同模式,最终实现交通控制与交通诱导协同模式的在线选择与转换。仿真结果表明,本文提出的基于Q-学习算法的协同模式选择方法在一般交通拥挤状态下具有较好的协同控制效果,对比离线式模式选择方法更能适应交通状态的不断变化,从而达到有效避免严重交通拥堵、改善路网性能的目的。     

A reinforcement learning approach for developing routing policies in multi-agent production scheduling

Most recent research studies on agent-based production scheduling have focused on developing negotiation schema for agent cooperation. However, successful implementation of agent-based approaches not only relies on the cooperation among the agents, but the individual agent’s intelligence for making good decisions. Learning is one mechanism that could provide the ability for an agent to increase its intelligence while in operation. This paper presents a study examining the implementation of the Q-learning algorithm, one of the most widely used reinforcement learning approaches, for use by job agents when making routing decisions in a job shop environment. A factorial experiment design for studying the settings used to apply Q-learning to the job routing problem is carried out. This study not only investigates the effects of this Q-learning application but also provides recommendations for factor settings and useful guidelines for future applications of Q-learning to agent-based production scheduling.     

When to apply the fifth commandment: the effects of parenting on genetic and learning agents

This article explores hybrid agents that use a variety of techniques to improve their performance in an environment over time. We considered, specifically, genetic-learning-parenting hybrid agents, which used a combination of a genetic algorithm, a learning algorithm (in our case, reinforcement learning), and a parenting algorithm, to modify their activity. We experimentally examined what constitutes the best combination of weights over these three algorithms, as a function of the environment's rate of change. For stationary environments, a genetic-parenting combination proved best, with genetics being given the most weight. For environments with low rates of change, genetic-learning-parenting hybrids were best, with learning having the most weight, and parenting having at least as much weight as genetics. For environments with high rates of change, pure learning agents proved best. A pure parenting algorithm operated extremely poorly in all settings.     

基于CMAC的Q算法在机器人足球中的应用

RoboCup是全球影响力最大的机器人足球比赛，是机器人学和人工智能及其应用的标准研究问题之一。仿真组在RoboCup中是重要的一部分。由于仿真组的比赛环境非常复杂，采用手工编码实现的Agent的高层决策无法考虑到比赛的所有情况，缺乏灵活性，并且需要花大量的时间对手工编码中的参数进行调整，结果也不是很理想。所以本文采用机器学习来实现Agent的决策。本文运用了一种基于CMAC的Q学习方法，把该方法应用在禁区内进行2VSl进攻的策略学习训练实例中，实验结果表明了本方法的可行性和有效性。     

Context-aware resource management in multi-inhabitant smart homes: A framework based on Nash -learning

A smart home aims at building intelligent automation with a goal to provide its inhabitants with maximum possible comfort, minimum resource consumption and thus reduced cost of home maintenance. ‘Context Awareness’ is perhaps the most salient feature of such an intelligent environment. An inhabitant’s mobility and activities play a significant role in defining his/her contexts in and around the home. Although there exists an optimal algorithm for location and activity tracking of a single inhabitant, the correlation and dependence between multiple inhabitants’ contexts within the same environment make the location and activity tracking more challenging. In this paper, we first prove that the optimal location prediction across multiple inhabitants in smart homes is an NP-hard problem. Next, to capture the correlation and interactions between different inhabitants’ movements (and hence activities), we develop a novel framework based on a game theoretic, Nash H-learning approach that attempts to minimize the joint location uncertainty of inhabitants. Our framework achieves a Nash equilibrium such that no inhabitant is given preference over others. This results in more accurate prediction of contexts and more adaptive control of automated devices, thus leading to a mobility-aware resource (say, energy) management scheme in multi-inhabitant smart homes. Experimental results demonstrate that the proposed framework is capable of adaptively controlling a smart environment, significantly reduces energy consumption and enhances the comfort of the inhabitants.     

Q-Learning Based Routing Protocol for Congestion Avoidance

The end-to-end delay in a wired network is strongly dependent on congestion on intermediate nodes. Among lots of feasible approaches to avoid congestion efficiently, congestion-aware routing protocols tend to search for an uncongested path toward the destination through rule-based approaches in reactive/incident-driven and distributed methods. However, these previous approaches have a problem accommodating the changing network environments in autonomous and self-adaptive operations dynamically. To overcome this drawback, we present a new congestion-aware routing protocol based on a Q-learning algorithm in software-defined networks where logically centralized network operation enables intelligent control and management of network resources. In a proposed routing protocol, either one of uncongested neighboring nodes are randomly selected as next hop to distribute traffic load to multiple paths or Q-learning algorithm is applied to decide the next hop by modeling the state, Q-value, and reward function to set the desired path toward the destination. A new reward function that consists of a buffer occupancy, link reliability and hop count is considered. Moreover, look ahead algorithm is employed to update the Q-value with values within two hops simultaneously. This approach leads to a decision of the optimal next hop by taking congestion status in two hops into account, accordingly. Finally, the simulation results presented approximately 20% higher packet delivery ratio and 15% shorter end-to-end delay, compared to those with the existing scheme by avoiding congestion adaptively.     

基于迁移学习的多任务分配算法

对于多任务分配问题，传统的方法针对每一个任务独立地寻找一个最优分配方案，没有考虑任务间的关联以及历史经验对新任务分配的影响，因而复杂度较高。研究了多智能体系统中的多任务分配问题，通过迁移学习来加速任务分配以及子任务的完成。在分配目标任务时，通过计算当前任务和历史任务的相似度找到最适合的源任务，再将源任务的分配模式迁移到目标任务中，并在完成子任务的过程中使用迁移学习，从而提高效率，节约时间。最后，通过“格子世界”的实验证明了该算法在运行时间和平均带折扣回报方面都优于基于Q学习的任务分配算法。