MDP-Based Network Selection Scheme by Genetic Algorithm and Simulated Annealing for Vertical-Handover in Heterogeneous Wireless Networks

The hybrid algorithm for real-time vertical handover using different objective functions has been presented to find the optimal network to connect with a good quality of service in accordance with the user’s preferences. Markov processes are widely used in performance modelling of wireless and mobile communication systems. We address the problem of optimal wireless network selection during vertical handover, based on the received information, by embedding the decision problem in a Markov decision process (MDP) with genetic algorithm (GA), we use GA to find a set of optimal decisions that ensures the best trade-off between QoS based on their priority level. Then, we emerge improved genetic algorithm (IGA) with simulated annealing (SA) as leading methods for search and optimization problems in heterogeneous wireless networks. We formulate the vertical handoff decision problem as a MDP, with the objectives of maximizing the expected total reward and minimizing average number of handoffs. A reward function is constructed to assess the QoS during each connection, and the AHP method are applied in an iterative way, by which we can work out a stationary deterministic handoff decision policy. As it is, the characteristics of the current mobile devices recommend using fast and efficient algorithms to provide solutions near to real-time. These constraints have moved us to develop intelligent algorithm that avoid the slow and massive computations. This paper compares the formulation and results of five recent optimization algorithms: artificial bee colony, GA, differential evolution, particle swarm optimization and hybrid of (GA–SA). Simulation results indicated that choosing the SA rules would minimize the cost function, and also that, the IGA–SA algorithm could decrease the number of unnecessary handovers, and thereby prevent the ‘Ping-Pong’ effect.     

A genetic algorithm approach for multi-attribute vertical handover decision making in wireless networks

Mobile terminals can typically connect to multiple wireless networks which offer varying levels of suitability for different classes of service. Due to the changing dynamics of network attributes and mobile users’ traffic needs, vertical handovers across heterogeneous networks become highly desirable. Multiple attribute decision making (MADM) techniques offer an efficient approach for ranking competing networks and selecting the best one according to specific quality of service parameters. In this paper, a genetic algorithm (GA) is applied to optimize network attributes’ weighting by emphasizing ranking differences among candidate networks, thereby aiding correct decision making by reducing unnecessary handovers and ranking abnormalities. The performance of the proposed GA-based vertical handover is investigated with typical MADM techniques including Simple Additive Weighting (SAW) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). The results show that the proposed GA-based weight determination approach reduces the abnormality observed in the conventional SAW and TOPSIS techniques substantially. The results of this paper will help ensuring the application of MADM methods to more dynamic and challenging decision making problems encountered in wireless network.     

Dynamic service migration strategy based on MDP model with multiple parameter in vehicular edge network

To handle with the service interruption caused by vehicles’ mobility and limited service coverage of edge servers,a dynamic service migration algorithm based on multi-parameters Markov decision process (MDP) model was put forward for vehicular edge network,which was called as dynamic service migration algorithm based on multiple parameter (DSMMP).Combining delay,bandwidth,server capacity with vehicle motion information,DSMMP constructed a multi-parameters MDP revenue function to remedy the deficiency of distance-based schemes.By using vehicle motion and delay constraints,a candidate server set with several candidate servers was defined,and migration decision through long-term Bellman revenue values was made.In order to improve the dynamic adaptability of the proposed algorithm,the weight values were calculated and updated by leveraging historical information.Simulation results show that our strategy has a good performance in terms of delay,packet loss ratio and service migration times.     

基于区间标记判决的稳健垂直切换算法研究

在异构5G网络中,针对切换算法稳健性差引起的切换准确率低的问题,提出一种基于区间标记判决的稳健垂直切换算法.首先,引入中位值平均滤波法,获取更准确的网络参数.其次,基于参数波动特征分析,提出区间标记判决算法,保证候选网络筛选通过率和准确率的同时,提高区间标记判决算法的稳健性.再次,根据移动终端传输层需求,结合终端运动趋势,分别使用不同权值的效用函数获得最佳目标网络.最后,仿真结果表明,该算法能够有效提升切换触发和网络筛选的准确率,降低切换失败率和乒乓效应,提高系统吞吐量,并能够根据移动终端的需求选择最佳目标网络.     

MEC中基于改进遗传模拟退火算法的虚拟网络功能部署策略

为了有效改善多集群共存的移动边缘网络中业务流端到端服务时延,提出了一种基于改进遗传模拟退火算法的虚拟网络功能部署策略。通过开放Jackson排队网络对移动业务流的时延进行最优化建模,在证明其NP性的基础上提出了将遗传算法与模拟退火算法相结合的求解策略,该策略通过对服务节点的提前映射机制避免了可能带来的网络拥塞,并通过个体的约束性判断和纠正遗传的方法避免了局部最优的出现。在不同的服务请求量、服务节点规模、集群数量及虚拟网络功能之间的逻辑连接关系等参数下的对比实验表明,该策略能提供更低时延的端到端服务,使时延敏感类移动业务获得更好体验。     

基于在线拍卖的网络切片资源分配算法

为满足未来移动通信网络中多样化的业务需求,为用户提供定制化服务的同时提升网络经济效益,该文提出一种基于在线拍卖的网络切片资源分配算法。根据业务类型将用户的服务请求转化为相应投标信息,以最大化拍卖参与者的社会福利为目标,将切片资源分配问题建模为基于多业务的在线赢家确定问题。结合资源分配与价格更新策略,实现基于在线拍卖的资源优化配置。仿真结果表明,该算法能够在满足用户业务需求的同时,提升网络经济效益。     

用户偏好提取算法

将用户的需求抽象为可表示、可量化、可感知的特征是未来移动业务的重要特点,用户偏好提取算法是探索这一问题的关键。分析了用户偏好提取算法的数学结构、技术特点、算法类型及研究面临的挑战。针对异构网络环境下移动用户的业务需求特点,提出将传统用户偏好提取技术与马尔可夫决策过程建模方法相结合,创建用户偏好评估模型。解决动态判决环境下基于不完整信息的智能判决问题。对研究用户体验的评价问题和业务与业务环境的适配问题提供了新的思路。     

A personalized QoE-aware handover decision based on distributed reinforcement learning

Recent developments in heterogeneous mobile networks and growing demands for variety of real-time and multimedia applications have emphasized the necessity of more intelligent handover decisions. Addressing the context knowledge of mobile devices, users, applications, and networks is the subject of context-aware handoff decision as a recent effort to this aim. However, user perception has not been attended adequately in the area of context-aware handover decision making. Mobile users may have different judgments about the Quality of Service (QoS) depending on their environmental conditions, and personal and psychological characteristics. This reality has been exploited in this paper to introduce a personalized user-centric handoff decision method to decide about the time and target of handover based on User Perceived Quality (UPQ) feedbacks. The UPQ degradations are mainly for the sake of (1) exiting the coverage of the serving Point of Attachment (PoA) or (2) QoS degradation of serving access network. Using UPQ metric, the proposed method obviates the necessity of being aware about rapidly varying network QoS parameters and overcomes the complexity and overhead of gathering and managing some other context information. Moreover, considering the underlying network and geographical map, the proposed method is able to inherently exploit the trajectory information of mobile users for handover decision. UPQ degradation is not only due to the user behaviour, but also due to the behaviours of others users. As such, multi-agent reinforcement learning paradigm has been considered for target PoA selection. The employed decision algorithm is based on WoLF-PHC learning method where UPQ is used as a delayed reward for training. The proposed handoff decision has been implemented under IEEE 802.21 framework using NS2 network simulator. The results have shown better performance of the proposed method comparing to conventional methods assuming regular movement of mobile users.     

移动性感知下基于负载均衡的任务迁移方案

针对移动边缘计算中用户移动性导致服务器间负载分布不均,用户服务质量(Quality of Service, QoS)下降的问题,提出了一种移动性感知下的分布式任务迁移方案。首先,以优化网络中性能最差的用户QoS为目标,建立了一个长期极大极小化公平性问题(Max Min Fairness, MMF),利用李雅普诺夫(Lyapunov)优化将原问题转化解耦。然后,将其建模为去中心化部分可观测马尔可夫决策过程(Decentralized Partially Observable Markov Decision Process, Dec-POMDP),提出一种基于多智能体柔性演员-评论家(Soft Actor-Critic, SAC)的分布式任务迁移算法,将奖励函数解耦为节点奖励和用户个体奖励,分别基于节点负载均衡度和用户QoS施加奖励。仿真结果表明,相比于现有任务迁移方案,所提算法能够在保证用户QoS的前提下降低任务迁移率,保证系统负载均衡。     

Service migration in mobile edge computing: A deep reinforcement learning approach

In mobile edge computing, service migration can not only reduce the access latency but also reduce the network costs for users. However, due to bandwidth bottleneck, migration costs should also be considered during service migration. In this way, the trade-off between benefits of service migration and total service costs is very important for the cloud service providers. In this paper, we propose an efficient dynamic service migration algorithm named SMDQN, which is based on reinforcement learning. We consider each mobile application service can be hosted on one or more edge nodes and each edge node has limited resources. SMDQN takes total delay and migration costs into consideration. And to reduce the size of Markov decision process space, we devise the deep reinforcement learning algorithm to make a fast decision. We implement the algorithm and test the performance and stability of it. The simulation result shows that it can minimize the service costs and adapt well to different mobile access patterns.     

GRA Based Network Selection in Heterogeneous Wireless Networks

4G wireless networks will integrate heterogeneous technologies such as Wireless LAN and third generation (3G) cellular networks and have the capability to offer various services at any time as per user requirements, anywhere with seamless interoperability at affordable cost. One important challenge in such a heterogeneous wireless environment is to enable network selection mechanisms in order to keep the mobile users always best connected anywhere and at any time. In this paper, a multi-criteria access network selection algorithm is proposed in Worldwide Interoperability for Microwave Access–Wireless Fidelity environment, in order to facilitate the provision of high quality services and at the same time to satisfy different types of user service level agreements. Analytical hierarchy process (AHP) and grey relational analysis (GRA) methods are applying for optimal access network selection. The proposed methodology combines the AHP to decide the relative weights of criteria set according to network’s performance, as well as the GRA to rank the network alternatives. The advantages of the GRA method are that the results are based on the original data, the calculations are simple and straightforward, and finally it is one of the best methods to make decision under heterogeneous wireless network environment.     

Network selection based on multiple attribute decision making and group decision making for heterogeneous wireless networks

One of the key issues for radio resources management is network selection strategy in heterogeneous scenarios.In order to provide ubiquitous service,the paper puts forward a network selection algorithm based on multiple attribute decision making(MADM) and group decision making(GDM).Firstly,the proposed algorithm acquires attribute weights’ vectors of the subjective and objective decision makers based on MADM,and then the two attribute weights’ vectors are synthesized to be a new attribute weights’ vector by using GDM.Considering that the results of GDM should be reasonable and convincible,the criterion of consistency is adopted for judging the compatibility of group judgments.More specifically,the algorithm takes into account not only objective attributes of networks but also the preference of subscribers and traffic class.Hence it guarantees that the subscribers can not select the networks with poor performance depending on their preference.The simulation results show that the proposed algorithm can effectively reduce the handoff number and provide subscribers with satisfactory quality of service(QoS).     

Heterogeneous wireless network selection algorithm based on group decision

In the environment of heterogeneous wireless networks, it is vital to select a currently optimal network for applications and subscribers. The use of multiple attribute decision making （MADM） for heterogeneous network selection can provide subscribers with satisfactory service quality. Converting heterogeneous network selection into a MADM problem, the authors present an improved algorithm for MADM based on group decision theory. The algorithm combines weight vectors of multiple attribute decision making to obtain a combinational weight vector. Then the results＇ compatibility will be assessed. If they do not meet the requirements of compatibility, the judgment matrix will be modified until a comprehensive vector that satisfies compatibility requirements is produced. The vector is combined with simple weighting method （SAW） for network selection. Simulation shows that the algorithm can provide users with satisfactory quality of service （QoS）.     

An access cell selection algorithm for next generation mobile environments

Next generation of mobile communications will be based on a heterogeneous infrastructure comprising different wireless access systems in a complementary manner. This paper proposes a network selection algorithm based on user activity, user preferences, service requirements, and networks conditions which provides users a prospect of being always best connected within an environment of heterogeneous mobile networks. This is achieved by a learning process which allows user to select an access network based in previous connections and a cost function that helps the user to select the best network that adapts to the needs.     

基于动态复合优先级的海上节点网络选择算法

针对海上移动节点完成业务时出现高优先级业务完成率低、业务拥堵率大、网络接入算法对波动环境适应性不足的问题,提出一种基于动态复合优先级的网络接入算法.首先,移动节点收集周围机动站点的网络属性参数,计算产生业务的执行紧迫性和剩余价值,再将不同类型业务的参数权重加入VIKOR法,形成最终的动态复合优先级,针对环境网络参数提供...     

一种不稳定环境下的策略搜索及迁移方法

强化学习是一种Agent在与环境交互过程中,通过累计奖赏最大化来寻求最优策略的在线学习方法.由于在不稳定环境中,某一时刻的MDP模型在与Agent交互之后就发生了变化,导致基于稳定MDP模型传统的强化学习方法无法完成不稳定环境下的最优策略求解问题.针对不稳定环境下的策略求解问题,利用MDP分布对不稳定环境进行建模,提出一种基于公式集的策略搜索算法--FSPS.FSPS算法在学习过程中搜集所获得的历史样本信息,并对其进行特征信息的提取,利用这些特征信息来构造不同的用于动作选择的公式,采取策略搜索算法求解最优公式.在此基础之上,给出所求解策略的最优性边界,并从理论上证明了迁移到新MDP分布中策略的最优性主要依赖于MDP分布之间的距离以及所求解策略在原始MDP分布中的性能.最后,将FSPS算法用于经典的Markov Chain问题,实验结果表明,所求解的策略具有较好的性能.     

A Heterogeneous Network Access Selection Algorithm Based on Attribute Dependence

In order to solve the problem which fails to consider the degree of attribute dependence in current network access selection schemes, a novel heterogeneous network access selection scheme based on attribute dependence is proposed in this paper. The scheme translates the network access selection problem into the problem of multi-attribute decision making based on attribute dependence and solves it using the chaotic glowworm swarm based algorithm. First, the degree of attribute dependence is measured and the access selection model is established based on the degree of attribute dependence. Then, the chaotic glowworm swarm based algorithm is used to solve the optimal weight in the model. Finally, the user accesses the network with the best performance based on the access selection model. The simulation results demonstrate the improved performance of the proposed access selection scheme compared with other schemes. The proposed scheme can reduce blocking and handoff dropping rate, as well as the number of handoff. Moreover, the proposed scheme achieves the load balance of each network.     

多网络接入协同选择与聚合算法

针对异构无线网络场景,本文提出了一种基于协同学的多网络并行接入协同聚合算法,该算法基于吞吐量、可用信道数、功耗、费用及丢包率等多个参数构建了网络协同度评价体系,将属性要求作为协同子系统,属于同一属性的不同参数作为子系统的序参量,序参量之间相互协同和制约,以更加全面地衡量聚合网络的整体性能。多网络聚合过程分为两步:首先计算单个网络的协同度,以判断该网络是否为参与聚合的候选网络,多个候选网络的各种排列组合可以得到多种网络的候选方案;候选方案采用属性聚合形成聚合网络,然后计算聚合网络的协同度,选择协同度最大的多网络聚合方案。仿真结果证明,本文算法能够更加合理分配信道,降低用户接入阻塞率,增加用户的平均吞吐量和系统容量,同时降低单位吞吐量对应的功耗和费用。      

异构无线网络中新的成本感知网络切换方案

针对异构无线网络中用户希望通过较低的花费获取最满意的网络服务的需求,本文提出了一种以用户为中心的成本感知网络切换方案.首先,根据用户能够获取到的网络传输速率建立用户服务满足度模型,并结合归一化的网络费用,将网络切换问题转化为一个多目标优化问题.考虑到网络阻塞是由于大量用户同时选择同一个网络导致的,将这种背景下的用户决策行为转化为EI Farol酒吧问题,通过求解该问题纳什均衡状态下的用户获取网络服务的概率来得到用户服务满足度的期望值.最后,通过考虑用户服务满足度回报率将所构建的多目标优化问题转化为一个最大化问题.仿真结果表明本文算法能够提高网络总吞吐量,降低用户切换阻塞率,使网络负载更加均衡.     

基于5G无人机通信的多智能体异构网络选择方法

摘 要：5G无人机通信网络和各种不同无线接入技术的结合使无线异构网络呈现多样化的发展趋势。然而，用户繁多且不同的业务请求对网络要求也不同，造成网络接入选择问题。提出了一种基于5G无人机通信的多智能体异构网络选择方法，将用户分为多个智能体，从用户端和网络端两个方面出发，将用户侧的时延和传输速率需求与网络侧的负载均衡需求综合考虑作为即时回报的相关参数，通过基于Nash Q-Learning的算法进行学习，得到异构网络环境下的网络选择决策模型。仿真结果表明，所提异构网络选择方法针对不同业务类型用户的需求均能选择合适的网络，同时均衡网络的负载，充分利用异构无线网络的资源。