Deep reinforcement learning-based resource allocation for D2D communications in heterogeneous cellular networks

Device-to-Device (D2D) communication-enabled Heterogeneous Cellular Networks (HCNs) have been a promising technology for satisfying the growing demands of smart mobile devices in fifth-generation mobile networks. The introduction of Millimeter Wave (mm-wave) communications into D2D-enabled HCNs allows higher system capacity and user data rates to be achieved. However, interference among cellular and D2D links remains severe due to spectrum sharing. In this paper, to guarantee user Quality of Service (QoS) requirements and effectively manage the interference among users, we focus on investigating the joint optimization problem of mode selection and channel allocation in D2D-enabled HCNs with mm-wave and cellular bands. The optimization problem is formulated as the maximization of the system sum-rate under QoS constraints of both cellular and D2D users in HCNs. To solve it, a distributed multiagent deep Q-network algorithm is proposed, where the reward function is redefined according to the optimization objective. In addition, to reduce signaling overhead, a partial information sharing strategy that does not observe global information is proposed for D2D agents to select the optimal mode and channel through learning. Simulation results illustrate that the proposed joint optimization algorithm possesses good convergence and achieves better system performance compared with other existing schemes.     

Performance Tradeoff Between Energy Conservation and User Fairness for D2D Communication Underlaying Cellular Networks

Device-to-Device (D2D) communication is viewed as an emerging technology in the fifth generation systems to fully explore the proximate gain residing in lo-cal communicating pairs. But the interference imposed to the existing Cellular links (C-link) should be addressed in the optimization for underlaid D2D links (D-link). There is a performance tradeoffbetween Energy conservation (EC) and the number of links with satisfied Quality-of-service (QoS) requirement or System capacity (SC). We aim to weight the above tradeofffor D-links by specifying it into optimizing EC on the condition that SC is maximized and QoS requirements of C-links are fully guaranteed. A three-stage joint power control and channel assignment mecha-nism is proposed including feasibility check, SC maximiza-tion and greedy optimization for EC. Thanks to numeri-cal results, we illustrate the necessity to conduct our per-formance tradeoffand observe the gain when multiple re-source variables are in elaborate cooperation.     

蜂窝网络中全双工D2D通信功率控制

在蜂窝网络中,采用全双工传输的设备直通(D2D)通信可以共享蜂窝通信的信道资源,提升频谱利用率和系统吞吐量.针对单对全双工D2D用户复用单个蜂窝用户的上行信道资源时,用户之间会产生同频干扰的问题,提出了一种低复杂度的功率控制算法.该算法在保证全双工D2D用户和蜂窝用户(CU)的服务质量(QoS)的前提下,最大化全双工D2D链路的吞吐量.仿真结果表明,该算法能够提高全双工D2D链路的吞吐量;全双工D2D链路吞吐量取决于蜂窝用户的QoS要求、相对距离以及自干扰消除数量的限制.     

D2D通信中联合链路共享与功率分配算法研究

针对D2D （Device-to-Device,D2D）通信过程中的资源分配问题,提出一种联合链路共享和功率分配算法.在保证系统内蜂窝用户服务质量（Quality of Service,QoS）需求的前提下,利用系统的信道状态信息,为D2D用户生成一个由蜂窝用户组成的通信链路的候选集合;在通信链路候选集合内使用凸优化方法得到D2D用户最优功率分配策略;最后利用（Kuhn-Munkres,KM）算法求解最大加权二部图匹配（Maximum Weight Bipartite Matching,MWBM）问题,为D2D用户选择最优的蜂窝用户进行资源共享.仿真结果表明该算法能有效的提升通信网络的吞吐量,可以为D2D用户选择最优的资源分配策略.     

On Optimal Resource Allocation in Cellular Networks With Best-Effort Traffic

Efficient design of online power allocation policies relies strongly on convex-analytic and optimization-theoretic properties of the optimization problem on hand. In this context we study the optimization of power allocation in cellular networks with so-called best-effort traffic. Our results exhibit a specific role of link QoS parameters, for which the dependence on the corresponding link SINR is log-convex. In such case the region of achievable QoS vectors is shown to be convex, the considered problem is globally solvable and can be easily transformed into a favorable convex form.     

基于多智能体深度强化学习的D2D通信资源联合分配方法

设备对设备(D2D)通信作为一种短距离通信技术,能够极大地减轻蜂窝基站的负载压力和提高频谱利用率。然而将D2D直接部署在授权频段或者免授权频段必然导致与现有用户的严重干扰。当前联合部署在授权和免授权频段的D2D通信的资源分配通常被建模为混合整数非线性约束的组合优化问题,传统优化方法难以解决。针对这个挑战性问题,该文提出一种基于多智能体深度强化学习的D2D通信资源联合分配方法。在该算法中,将蜂窝网络中的每个D2D发射端作为智能体,智能体能够通过深度强化学习方法智能地选择接入免授权信道或者最优的授权信道并发射功率。通过选择使用免授权信道的D2D对(基于“先听后说”机制)向蜂窝基站的信息反馈,蜂窝基站能够在非协作的情况下获得WiFi网络吞吐量信息,使得算法能够在异构环境中执行并能够确保WiFi用户的QoS。与多智能体深度Q网络(MADQN)、多智能体Q学习(MAQL)和随机算法相比,所提算法在保证WiFi用户和蜂窝用户的QoS的情况下能够获得最大的吞吐量。     

异构云无线接入网场景下D2D通信资源共享与分配

针对异构云无线接入网中的前向链路受限问题,提出了一种基于干扰阈值的设备到设备(Device-to-Device,D2D)多用户分簇方案,并对系统信道和功率资源的分配进行优化。首先根据D2D用户之间的干扰级别,利用着色图理论对多用户进行分簇;然后,在满足D2D用户和蜂窝用户服务质量约束下,建立了基于D2D用户和速率最大化的资源分配模型,并进一步采用二分法对已分簇的D2D用户进行功率优化分配。仿真实验结果验证了所提方案相比传统方案,系统频谱利用率提高了55%以上。     

Power Allocation and Transmission Period Selection for Device-to-Device Communication as an Underlay to Cellular Networks

Efficient design of device-to-device (D2D) communication calls for D2D users to propose adaptive power allocation strategy and to establish reliable communication links while protecting the QoS of cellular communications. In this paper, we consider the D2D communication as an underlay to relay-assisted cellular networks. To maximize the ergodic capacity, we derive an optimal transmission power under an average power constraint. With the derived optimal transmission power, a transmission period selection strategy for D2D communication is firstly introduced to improve reliability. We derive the outage probability in closed forms and evaluate the ergodic capacity to show performances of the proposed system. Numerical results show that the D2D system can achieve high capacity gains by flexibly allocating transmission power based on channel state information and significantly enhance reliability by selecting a transmission period, while satisfying various QoS conditions for cellular communication.     

D2D/蜂窝通信模式切换与联合功率控制方案

设备直连（Device-to-Device, D2D）通信技术通过复用蜂窝系统的频谱资源提高频谱利用率,但D2D的引入会给蜂窝系统带来干扰。如何合理地选择D2D/蜂窝通信模式并进行功率优化控制,成为减小D2D和蜂窝系统间干扰、提升网络性能的关键。本文考虑D2D用户复用蜂窝上行链路场景,提出了一种基于距离和联合功率控制的通信模式选择方案。在该方案中,D2D用户和蜂窝用户与基站距离的比值决定了D2D用户是否采用Underlay模式进行通信,进而在约束蜂窝用户和D2D用户发射功率的条件下实现D2D链路和蜂窝链路的联合功率控制,最终推导出能够最大化系统总吞吐量的最优用户功率分配方案。根据仿真结果,本文提出的联合功率控制方案能够在降低系统间干扰的同时有效提高D2D和蜂窝系统的总吞吐量,进而提高了系统的性能。      

D2D通信中基于粒子群优化的能效最大化策略

5G中终端的能量消耗和频谱资源问题日益严重,在终端直通技术(D2D)中尤为突出.为了提高D2D对用户(DP)的能效和资源利用率,提出了一种基于粒子群算法的联合功率控制和资源分配策略.以最大化D2 D链路总能效为目标,将构造的资源分配矩阵和功率分配矩阵作为粒子的位置,依照蜂窝用户(CU)和DP服务质量的约束来修正粒子位置和速度,使之适合于原分式规划问题的求解,合理地提升了DP的总能效,实现了一个DP链路能复用多个CU资源.仿真结果表明,所提算法不仅使能效显著提升,而且使资源利用率提高了80%.     

Efficient target tracking in directional sensor networks with selective target area’s coverage

Device-to-Device (D2D) communication technology, under the standardization of third generation partnership project and a component of the evolving fifth generation architecture, is mainly aimed to increase system capacity and data rate via providing direct communications between end devices without the use of routing data through the network. Apart from the attracting features, due to the resource sharing between cellular user equipment (CUE) and D2D user equipment (DUE) in such communications, an efficient algorithm for resource and power allocation to DUE, especially for mobile users is necessary to maintain the performance. The current paper introduces a joint mode algorithm for mobile user to choose between cellular and D2D communications and thereby, analyzes resource allocation issues. We propose an efficient algorithm for mobility management of users based on their current connection modes. The locations of D2D pairs are estimated by Levenberg–Marquardt method based on the received signal strength (RSS) from different macrocells. Since the range of D2D communication is much shorter than that of cellular communication, in order to prevent ping-pong handoffs between cellular and D2D modes, we propose the estimation of the next RSS samples in cellular mode prior to switching to D2D mode. In both cellular and D2D modes, the allocated resource block (RB) is the one with the highest signal to interference plus noise ratio (SINR) in order to increase the throughput, under the condition of providing minimum SINR requirement of CUE. This is achieved via transmission power control of the D2D pair. For performance evaluation, we studied the effects of increasing velocity of D2D and cellular users, number of users, and SINR threshold. The results indicate that the proposed solution fairly manages the communication mode of mobile users and incurs improvement in system throughput.     

Resource allocation for NOMA based D2D underlaid cellular networks

This paper puts forward a user clustering and power allocation algorithm for non-orthogonal multiple access (NOMA) based device-to-device (D2D) cellular system. Firstly, an optimization problem aimed at maximizing the sum-rate of the system is constructed. Since the optimization problem is a mixed-integer non-convex optimization, it is decomposed into two subproblems, namely user clustering and power allocation subproblem. In the subproblem of user clustering, the clustering algorithms of cellular user and D2D pair are proposed respectively. In the power allocation subproblem, the gradient assisted binary search (GABS) algorithm and logarithmic approximation in successive convex approximation (SCA) are used to optimize the power of subchannel (SC) and D2D transmitted power respectively. Finally, an efficient joint iterative algorithm is proposed for the original mixed inter non-convex non-deterministic polynomial (NP)-hard problem. The simulation results show that the proposed algorithm can effectively improve the total system rate and the larger the ratio of cellular users (CUs) to total users, the larger the total system rate.     

Resource allocation and dynamic power control for D2D communication underlaying uplink multi-cell networks

Underlaying device-to-device (D2D) communication is suggested as a promising technology for the next generation cellular networks (5G), where users in close proximity can transmit directly to one another bypassing the base station. However, when D2D communications underlay cellular networks, the potential gain from resource sharing is highly determined by how the interference is managed. In order to mitigate the resource reuse interference between D2D user equipment and cellular user equipment in a multi-cell environment, we propose a resource allocation scheme and dynamic power control for D2D communication underlaying uplink cellular network. Specifically, by introducing the fractional frequency reuse (FFR) principle into the multi-cell architecture, we divide the cellular network into inner region and outer region. Combined with resource partition method, we then formulate the optimization problem so as to maximize the total throughput. However, due to the coupled relationship between resource allocation and power control scheme, the optimization problem is NP-hard and combinational. In order to minimize the interference caused by D2D spectrum reuse, we solve the overall throughput optimization problem by dividing the original problem into two sub-problems. We first propose a heuristic resource pairing algorithm based on overall interference minimization. Then with reference to uplink fractional power control (FPC), a dynamic power control method is proposed. By introducing the interference constraint, we use a lower bound of throughput as a cost function and solve the optimal power allocation problem based on dual Lagrangian decomposition method. Simulation results demonstrate that the proposed algorithm achieves efficient performance compared with existing methods.     

On selecting transmission mode for D2D transmitter in underlay cellular network with a multi-antenna base station

The Device-to-Device (D2D) communication underlaying cellular networks is considered in this study. The D2D transmitter in the D2D mode can directly transmit messages to a receiver, but it may interfere with the transmission of another cellular user who shares the same uplink channel. The transmitter can also operate in a cellular mode in which no interference to another cellular user occurs. We propose a mode selection scheme that aims to minimize the transmission power of the D2D transmitter subject to constraints on the minimum required data rate and maximum interference to other cellular users. The proposed scheme is based on bounds for transmission power and is less complex than the optimal scheme. Furthermore, it requires only a few statistics and does not need a fading channel distribution. The performance of the scheme is close to optimum when the number of Base Station (BS) antennas is large, and the mean absolute deviation of the fading terms is small. We verify this with numerical results of the Rician and Rayleigh fading channels by assuming that the BS antennas are independent. The simulation results for the two correlated BS antennas are presented herein.     

Network coded multicast for multi-hop D2D communication systems

ABSTRACT

In this paper, we consider the wireless communication systems where multi-hop Device-to-Device (D2D) networks can coexist with the conventional cellular networks by sharing the downlink resource of cellular users (CUs). A multicast data flow is distributed over the multi-hop D2D networks where network coding (NC) can be employed at the intermediate nodes. To maximize the utility of the multicast flow, we formulate a joint optimization problem for the systems while guaranteeing the quality-of-service (QoS) for regular CUs. We propose a subgradient algorithm to solve the optimization problem by decomposing it into three sub-problems: multicast rate control, NC subgraph selection, and downlink resource reusing. In particular, we develop a greedy algorithm to deal with the downlink resource reusing sub-problem for it is NP hard. Numerical and simulation results prove the superior performance of the proposed techniques compared with the conventional routing scheme.     

D2D网络中信道选择与功率控制策略研究

针对D2D通信的资源分配问题,该文研究了D2D信道选择与功率控制策略。在保证蜂窝用户服务质量(QoS)的前提下,提出一种基于启发式的D2D信道选择算法,为系统内的D2D用户找到合适的信道复用资源。同时,利用拉格朗日对偶方法求解得到D2D用户最优传输功率。仿真结果表明当蜂窝用户与多对D2D用户共享信道资源时能够大幅度提升系统平均吞吐量。在相同条件下,该算法的性能要明显优于现有算法。     

基于合作中继的OFDM蜂窝网络的QoS感知资源调度算法

针对基于中继的OFDM蜂窝网络,该文考虑具有不同QoS要求的混合业务场景,引入合作传输机制,提出了一种基于合作中继的QoS感知资源调度算法,解决了合作中继节点选取,子载波分配以及功率控制等问题。以最大化系统效用为目标,在考虑QoS业务的速率要求与基站功率约束的同时,针对中继结构引入了中继节点的功率约束。为降低计算复杂度,将原非线性组合优化问题分解为子载波分配与功率控制两个子问题。仿真结果表明,该文所提算法在能量节约、系统效用,吞吐量等性能方面都有显著优势。     

Optimal transmission mode proportion for heterogeneous networks of cellular and device-to-device communications

This paper discusses about the optimal mode allocation for the heterogeneous networks, in which the network can schedule users working in the device-to-device （D2D） mode or cellular mode. The D2D user is allowed to reuse the uplink resource of cellular system and the problem is formed as a sum-capacity optimization issue with outage constraints for both cellular and D2D links. The method for the optimal user proration is proved to be divided into three cases according to the total user density： when the total user density is small, the optimal proration trends to all users utilizing one mode; when the total user density is large, the optimal proration is all of users choosing D2D mode; and when the total user density situates in the between, there is a unique optimal transmission mode proportion for the hybrid networks to maximize its sumcapacity. The simulation results demonstrate the validity of the conclusions in the analysis part.     

Energy-efficient and QoS-supported deployment strategy of dense Macro-Femto cellular network

With the exponentially increasing of users’ demand for mobile data traffic,massive small cells have been deployed to satisfy the users’ quality of service (QoS) by operators.However,a significant energy would be consumed caused by dense deployment.To this end,a dense heterogeneous cellular network deployment strategy was proposed with QoS guarantee to decline system energy consumption.Firstly,a dense Macro-Femto biased cellular network was used to build the network model,the two QoS indicators of SINR coverage and user average rate was analyzed by stochastic geometry theory.Then,under the condition of QoS constraint,average spatial power consumption (ASPC) was taken as the optimization goal,optimal deploy density and transmission power of Femto were achieved by using optimization theory.Finally,the experimental results show that,comparing with the traditional strategy which only considering base station density,the proposed strategy has advantages in terms of energy efficiency while QoS guarantee.     

Joint power allocation and channel assignment for device-to-device communication using the Hungarian model and enhanced hybrid Red Fox-Harris Hawks Optimization

Multihop device-to-device (D2D) communication is a promising advancement since it allows wireless users in close proximity to communicate directly with one another without using base stations (BSs). With two nearby users, this technology intends to keep up with the accelerating development of mobile devices and the rising demands of local traffic loads. High-data-rate short-range transmission is made possible by this paradigm, which also lowers network backhaul expenses while improving end-user experience, spectral efficiency (SE) and network coverage. In order to maximize the cell's overall sum rates, this article examines the problem of selecting the optimal M-D2D linkages and cellular users (CUs) to form spectrum-sharing partners while taking quality of service (QoS) and energy efficiency (EE) needs into consideration. This study examines the use of a channel assignment system and a power allocation technique to manage interference in D2D communication scenarios involving many hops (more than one hop). The suggested channel assignment approach is based on the Hungarian method, but the power allocation system is based on the time-efficient enhanced Harris hawks optimization (HHO) and red fox (RF) algorithms. A genetic algorithm (GA)-based methodology and various baseline approaches are used to compare the effectiveness of the suggested system. Because it combines exploitation and exploration using a memory-based local search methodology and a perturbation mechanism, the suggested approach outperforms the GA in simulations. The results clearly demonstrate that the suggested method increases EE by up to 13% by producing the appropriate channel and power assignment for CUs and M-D2D users.