Implementation of call admission control scheme in next generation mobile communication networks using particle swarm optimization and fuzzy logic systems

In the present and next generation wireless networks, cellular system remains the major method of telecommunication infrastructure. Since the characteristic of the resource constraint, call admission control is required to address the limited resource problem in wireless network. The call dropping probability and call blocking probability are the major performance metrics for quality of service (QoS) in wireless network. Many call admission control mechanisms have been proposed in the literature to decrease connection dropping probability for handoffs and new call blocking probability in cellular communications. In this paper, we proposed an adaptive call admission control and bandwidth reservation scheme using fuzzy logic control concept to reduce the forced termination probability of multimedia handoffs. Meanwhile, we adopt particle swarm optimization (PSO) technique to adjust the parameters of the membership functions in the proposed fuzzy logic systems. The simulation results show that the proposed scheme can achieve satisfactory performance when performance metrics are measured in terms of the forced termination probability for the handoffs, the call blocking probability for the new connections and bandwidth utilization.     

Integration of pricing with call admission control to meet QoS requirements in cellular networks

Call admission control (CAC) plays a significant role in providing the desired quality of service (QoS) in cellular networks. We investigate the role of pricing as an additional dimension of the call admission control process in order to efficiently and effectively control the use of wireless network resources. First, we prove that, for a given wireless network, there exists a new call arrival rate which can maximize the total utility of users while maintaining the required QoS. Based on this result and observation, we propose an integrated pricing and call admission control scheme where the price is adjusted dynamically based on the current network conditions in order to alleviate the problem of congestion. Our proposed integrated approach implicitly implements a distributed user-based prioritization mechanism by providing negative incentives according to the current network conditions and therefore shaping the aggregate traffic in the network. We compare the performance of our approach in terms of congestion prevention, achievable total user utility, and obtained revenue, with the corresponding results of conventional systems where pricing is not taken into consideration in the call admission control process. These performance results verify the considerable improvement that can be achieved by the integration of pricing in the call admission control process in cellular networks.     

基于学习的鲁棒自适应评判控制研究进展

在作为人工智能核心技术的机器学习领域,强化学习是一类强调机器在与环境的交互过程中进行学习的方法,其重要分支之一的自适应评判技术与动态规划及最优化设计密切相关.为了有效地求解复杂动态系统的优化控制问题,结合自适应评判,动态规划和人工神经网络产生的自适应动态规划方法已经得到广泛关注,特别在考虑不确定因素和外部扰动时的鲁棒自适应评判控制方面取得了很大进展,并被认为是构建智能学习系统和实现真正类脑智能的必要途径.本文对基于智能学习的鲁棒自适应评判控制理论与主要方法进行梳理,包括自学习鲁棒镇定,自适应轨迹跟踪,事件驱动鲁棒控制,以及自适应H_∞控制设计等,并涵盖关于自适应评判系统稳定性、收敛性、最优性以及鲁棒性的分析.同时,结合人工智能、大数据、深度学习和知识自动化等新技术,也对鲁棒自适应评判控制的发展前景进行探讨.     

Online actor-critic algorithm to solve the continuous-time infinite horizon optimal control problem

In this paper we discuss an online algorithm based on policy iteration for learning the continuous-time (CT) optimal control solution with infinite horizon cost for nonlinear systems with known dynamics. That is, the algorithm learns online in real-time the solution to the optimal control design HJ equation. This method finds in real-time suitable approximations of both the optimal cost and the optimal control policy, while also guaranteeing closed-loop stability. We present an online adaptive algorithm implemented as an actor/critic structure which involves simultaneous continuous-time adaptation of both actor and critic neural networks. We call this ‘synchronous’ policy iteration. A persistence of excitation condition is shown to guarantee convergence of the critic to the actual optimal value function. Novel tuning algorithms are given for both critic and actor networks, with extra nonstandard terms in the actor tuning law being required to guarantee closed-loop dynamical stability. The convergence to the optimal controller is proven, and the stability of the system is also guaranteed. Simulation examples show the effectiveness of the new algorithm.     

An adaptive call admission algorithm for cellular networks

In this paper, we first propose a new continuous action-set learning automaton and theoretically study its convergence properties and show that it converges to the optimal action. Then we give an adaptive and autonomous call admission algorithm for cellular mobile networks, which uses the proposed learning automaton to minimize the blocking probability of the new calls subject to the constraint on the dropping probability of the handoff calls. The simulation results show that the performance of the proposed algorithm is close to the performance of the limited fractional guard channel algorithm for which we need to know all the traffic parameters in advance.     

ATM communications network control by neural networks

A learning method that uses neural networks for service quality control in the asynchronous transfer mode (ATM) communications network is described. Because the precise characteristics of the source traffic are not known and the service quality requirements change over time, building an efficient network controller which can control the network traffic is a difficult task. The proposed ATM network controller uses backpropagation neural networks for learning the relations between the offered traffic and service quality. The neural network is adaptive and easy to implement. A training data selection method called the leaky pattern table method is proposed to learn precise relations. The performance of the proposed controller is evaluated by simulation of basic call admission models.     

Robust adaptive critic control of nonlinear systems using fuzzy basis function networks: An LMI approach

This paper proposes an adaptive critic tracking control design for a class of nonlinear systems using fuzzy basis function networks (FBFNs). The key component of the adaptive critic controller is the FBFN, which implements an associative learning network (ALN) to approximate unknown nonlinear system functions, and an adaptive critic network (ACN) to generate the internal reinforcement learning signal to tune the ALN. Another important component, the reinforcement learning signal generator, requires the solution of a linear matrix inequality (LMI), which should also be satisfied to ensure stability. Furthermore, the robust control technique can easily reject the effects of the approximation errors of the FBFN and external disturbances. Unlike traditional adaptive critic controllers that learn from trial-and-error interactions, the proposed on-line tuning algorithm for ALN and ACN is derived from Lyapunov theory, thereby significantly shortening the learning time. Simulation results of a cart-pole system demonstrate the effectiveness of the proposed FBFN-based adaptive critic controller.     

Reinforcement learning for an ART-based fuzzy adaptive learningcontrol network

This paper proposes a reinforcement fuzzy adaptive learning control network (RFALCON), constructed by integrating two fuzzy adaptive learning control networks (FALCON), each of which has a feedforward multilayer network and is developed for the realization of a fuzzy controller. One FALCON performs as a critic network (fuzzy predictor), the other as an action network (fuzzy controller). Using temporal difference prediction, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network performs a stochastic exploratory algorithm to adapt itself according to the internal reinforcement signal. An ART-based reinforcement structure/parameter-learning algorithm is developed for constructing the RFALCON dynamically. During the learning process, structure and parameter learning are performed simultaneously. RFALCON can construct a fuzzy control system through a reward/penalty signal. It has two important features; it reduces the combinatorial demands of system adaptive linearization, and it is highly autonomous.     

Reinforcement learning for resource allocation in LEO satellite networks.

In this paper, we develop and assess online decision-making algorithms for call admission and routing for low Earth orbit (LEO) satellite networks. It has been shown in a recent paper that, in a LEO satellite system, a semi-Markov decision process formulation of the call admission and routing problem can achieve better performance in terms of an average revenue function than existing routing methods. However, the conventional dynamic programming (DP) numerical solution becomes prohibited as the problem size increases. In this paper, two solution methods based on reinforcement learning (RL) are proposed in order to circumvent the computational burden of DP. The first method is based on an actor-critic method with temporal-difference (TD) learning. The second method is based on a critic-only method, called optimistic TD learning. The algorithms enhance performance in terms of requirements in storage, computational complexity and computational time, and in terms of an overall long-term average revenue function that penalizes blocked calls. Numerical studies are carried out, and the results obtained show that the RL framework can achieve up to 56% higher average revenue over existing routing methods used in LEO satellite networks with reasonable storage and computational requirements.     

Adaptive critic motion control design of autonomous wheeled mobile robot by dual heuristic programming

Autonomous wheeled mobile robot (WMR) needs implementing velocity and path tracking control subject to complex dynamical constraints. Conventionally, this control design is obtained by analysis and synthesis or by domain expert to build control rules. This paper presents an adaptive critic motion control design, which enables WMR to autonomously generate the control ability by learning through trials. The design consists of an adaptive critic velocity control loop and a self-learning posture control loop. The neural networks in the velocity neuro-controller (VNC) are corrected with the dual heuristic programming (DHP) adaptive critic method. Designer simply expresses the control objective by specifying the primary utility function then VNC will attempt to fulfill it through incremental optimization. The posture neuro-controller (PNC) learns by approximating the specialized inverse velocity model of WMR so as to map planned positions to suitable velocity commands. Supervised drive supplies variant velocity commands for PNC and VNC to set up their neural weights. During autonomous drive, while PNC halts learning VNC keeps on correcting its neural weights to optimize the control performance. The proposed design is evaluated on an experimental WMR. The results show that the DHP adaptive critic design is a useful base of autonomous control.