GPSPA: a new adaptive algorithm for maintaining shortest path routing trees in stochastic networks

This paper presents a new efficient solution to the Dynamic Shortest Path Routing Problem, using the principles of Generalized Pursuit Learning. It proposes an efficient algorithm for maintaining shortest path routing trees in networks that undergo stochastic updates in their structure. It involves finding the shortest path in a stochastic network, where there are continuous probabilistically based updates in link‐costs. In vast, rapidly changing telecommunications (wired or wireless) networks, where links go up and down continuously and rapidly, and where there are simultaneous random updates in link costs, the existing algorithms are inefficient. In such cases, shortest paths need to be computed within a very short time (often in the order of microseconds) by scanning and processing the minimal number of nodes and links. The proposed algorithm, referred to as the Generalized Pursuit Shortest Path Algorithm (GPSPA), will be very useful in this regard, because after convergence, it seems to be the best algorithm to‐date for this purpose. Indeed, it has the advantage that it can be used to find the shortest path within the ‘statistical’ average network, which converges irrespective of whether there are new changes in link‐costs or not. Existing algorithms are not characterized by such a behaviour inasmuch as they would recalculate the affected shortest paths after each link‐cost update. The algorithm has been rigorously evaluated experimentally, and it has been found to be a few orders of magnitude superior to the algorithms available in the literature. Copyright © 2004 John Wiley & Sons, Ltd.     

Virtual network construction with K‐shortest path algorithm and optimization problems for robust physical networks

In network virtualization, it has been considered that virtual networks are constructed over a physical network where conventional data transmission services have been utilized. Here, virtual networks have to be constructed while keeping qualities of the conventional services. In this paper, we propose a new virtual network construction in order to construct many virtual networks while keeping the robustness of a physical network by using network resources effectively. The proposed method consists of three processes: K ‐shortest path algorithm and Prim's minimum spanning tree algorithm, path splitting, and path migration. In the proposed method, at first, multiple topologies are designed by using the K ‐shortest path algorithm and the Prim's MST algorithm according to the user's request. After the topology design is completed, an admission control with network robustness of the physical network is performed. Then, if one of the designed topologies can satisfy the construction conditions, a virtual network is constructed and provided with the user. Otherwise, the path splitting and path migration are performed. Here, the path splitting is utilized to design another topology of a virtual network and path migration is used to change the topologies of the virtual networks that have already been constructed. These processes are formulated as optimization problems and those are processed by solving the optimization problems. In numerical examples, we show that our proposed method can construct a higher number of virtual networks while keeping the robustness of a physical network by comparing with the conventional method where only the Kou–Markowsky–Berman algorithm is used. Copyright © 2015 John Wiley & Sons, Ltd.     

Adaptive quality of service‐based routing approaches: development of neuro‐dynamic state‐dependent reinforcement learning algorithms

In this paper, we propose two adaptive routing algorithms based on reinforcement learning. In the first algorithm, we have used a neural network to approximate the reinforcement signal, allowing the learner to take into account various parameters such as local queue size, for distance estimation. Moreover, each router uses an online learning module to optimize the path in terms of average packet delivery time, by taking into account the waiting queue states of neighbouring routers. In the second algorithm, the exploration of paths is limited to N‐best non‐loop paths in terms of hops number (number of routers in a path), leading to a substantial reduction of convergence time. The performances of the proposed algorithms are evaluated experimentally with OPNET simulator for different levels of traffic's load and compared with standard shortest‐path and Q‐routing algorithms. Our approach proves superior to classical algorithms and is able to route efficiently even when the network load varies in an irregular manner. We also tested our approach on a large network topology to proof its scalability and adaptability. Copyright © 2006 John Wiley & Sons, Ltd.     

SEAL2: An SDN‐enabled all‐Layer2 packet forwarding network architecture for multitenant datacenter networks

This paper presents the design and development of a new network virtualization scheme to support multitenant datacenter networking (MT‐DCN) based on software‐defined networking (SDN) technologies. Effective multitenancy supports are essential and challenging for datacenter networking designs. In this study, we propose a new network virtualization architecture framework for efficient packet forwarding in MT‐DCN. Traditionally, an internet host uses IP addresses for both host identification and location information, which causes mobile IP problems whenever the host is moved from one IP subnet to another. Unfortunately, virtual machine (VM) mobility is inevitable for cloud computing in datacenters for reasons such as server consolidation and network traffic flow optimization. To solve the problems, we decouple VM identification and location information with two independent values neither by IP addresses. We redefine the semantics of Ethernet MAC address to embed tenant ID information to the MAC address field without violating its original functionality. We also replace traditional Layer2/Layer3 two‐stage routing schemes (MAC/IP) with an all‐Layer2 packet forwarding mechanism that combines MAC addresses (for VM identification and forwarding in local server groups under an edge switch gateway) and multiprotocol label switching (MPLS) labels (for packet transportation between edge switch gateways across the core label switching network connecting all the edge gateways). To accommodate conventional IP packet architecture in a multitenant environment, SDN (OpenFlow) technology is used to handle all this complex network traffics. We verified the design concepts by a simple system prototype in which all the major system components were implemented. Based on the prototype system, we evaluated packet forwarding efficiency under the proposed network architecture and compared it with conventional IP subnet routing approaches. We also evaluated the incurred packet processing overhead caused by each of the packet routing components.     

An efficient actor‐critic reinforcement learning for device‐to‐device communication underlaying sectored cellular network

In this paper, a novel reinforcement learning (RL) approach with cell sectoring is proposed to solve the channel and power allocation issue for a device‐to‐device (D2D)‐enabled cellular network when the prior traffic information is not known to the base station (BS). Further, this paper explores an optimal policy for resource and power allocation between users intending to maximize the sum‐rate of the overall system. Since the behavior of wireless channel and traffic request of users in the system is stochastic in nature, the dynamic property of the environment allows us to employ an actor‐critic RL technique to learn the best policy through continuous interaction with the surrounding. The proposed work comprises of four phases: cell splitting, clustering, queuing model, and channel allocation and power allocation simultaneously using an actor‐critic RL. The implementation of cell splitting with novel clustering technique increases the network coverage, reduces co‐channel cell interference, and minimizes the transmission power of nodes, whereas the queuing model solves the issue of waiting time for users in a priority‐based data transmission. With the help of continuous state‐action space, the actor‐critic RL algorithm based on policy gradient improves the overall system sum‐rate as well as the D2D throughput. The actor adopts a parameter‐based stochastic policy for giving continuous action while the critic estimates the policy and criticizes the actor for the action. This reduces the high variance of the policy gradient. Through numerical simulations, the benefit of our resource sharing scheme over other existing traditional scheme is verified.     

A distributed adaptive landmark clustering algorithm based on mOverlay and learning automata for topology mismatch problem in unstructured peer‐to‐peer networks

Peer‐to‐peer networks are overlay networks that are built on top of communication networks that are called underlay networks. In these networks, peers are unaware of the underlying networks, so the peers choose their neighbors without considering the underlay positions, and therefore, the resultant overlay network may have mismatches with its underlying network, causing redundant end‐to‐end delay. Landmark clustering algorithms, such as mOverlay , are used to solve topology mismatch problem. In the mOverlay algorithm, the overlay network is formed by clusters in which each cluster has a landmark peer. One of the drawbacks of mOverlay is that the selected landmark peer for each cluster is fixed during the operation of the network. Because of the dynamic nature of peer‐to‐peer networks, using a non‐adaptive landmark selection algorithm may not be appropriate. In this paper, an adaptive landmark clustering algorithm obtained from the combination of mOverlay and learning automata is proposed. Learning automata are used to adaptively select appropriate landmark peers for the clusters in such a way that the total communication delay will be minimized. Simulation results have shown that the proposed algorithm outperforms the existing algorithms with respect to communication delay and average round‐trip time between peers within clusters. Copyright © 2015 John Wiley & Sons, Ltd.     

An optimized trusted‐cluster–based routing in disruption‐tolerant network using experiential learning model

Disruption‐tolerant network (DTN) implementation is subject to many routing constraints like limited knowledge of the network and intermittent connections with no end‐to‐end path existence. In this paper, the researchers propose trusted‐cluster–based routing protocol (TCR) for routing in DTN. TCR uses the experiential learning model that integrates neural network‐based bipolar sigmoid activation function to form trusted‐cluster DTN. TCR works in two phases: firstly to form a trusted‐cluster and secondly to identify cluster heads to direct network traffic through them. After the formation of the trusted‐cluster, a cluster head is chosen for a set period, thus instigating stability in the network. These trust values are attached to the node's route cache to make competitive routing decisions by relaying a message to the other trusted intermediate nodes only. With negative trust value, any node is deprived of participation in the network. This way, TCR eliminates malicious or selfish nodes to participate in the DTN network and minimizes the number of messages forwarded in a densely populated DTN. Also, this implementation conserves sufficient buffer memory to reach the destined node. The TCR's performance with other DTN routing schemes, namely, epidemic and trust‐based routing, is compared using multiple simulations runs. The proposed work is verified using mobility traces from Community Resource for Archiving Wireless Data At Dartmouth, and the experimental result shows the elimination of selfish nodes participating in the DTN. The simulation result shows an increase of 19% in message delivery by forwarding only to a trusted intermediate node possible.     

Estimator learning automata for feature subset selection in high‐dimensional spaces,case study: Email spam detection

One of the difficult challenges facing data miners is that algorithm performance degrades if the feature space contains redundant or irrelevant features. Therefore, as a critical preprocess task, dimension reduction is used to build a smaller space containing valuable features. There are 2 different approaches for dimension reduction: feature extraction and feature selection, which itself is divided into wrapper and filter approaches. In high‐dimensional spaces, feature extraction and wrapper approaches are not applicable due to the time complexity. On the other hand, the filter approach suffers from inaccuracy. One main reason for this inaccuracy is that the subset's size is not determined considering specifications of the problem. In this paper, we propose ESS (estimator learning automaton‐based subset selection) as a new method for feature selection in high‐dimensional spaces. The innovation of ESS is that it combines wrapper and filter ideas and uses estimator learning automata to efficiently determine a feature subset that leads to a desirable tradeoff between the accuracy and efficiency of the learning algorithm. To find a qualified subset for a special processing algorithm that functions on an arbitrary dataset, ESS uses an automaton to score each candidate subset upon the scale of the subset and accuracy of the learning algorithm using it. In the end, the subset with the highest score is returned. We have used ESS for feature selection in the framework of spam detection, a text classification task for email as a pervasive communication medium. The results show achievement in reaching the goal stated above.     

E‐TDGO: An encrypted trust‐based dolphin glowworm optimization for secure routing in mobile ad hoc network

Mobile ad hoc networks (MANETs) are independent networks, where mobile nodes communicate with other nodes through wireless links by multihop transmission. Security is still an issue to be fixed in MANETs. Hence, a routing protocol named encrypted trust‐based dolphin glowworm optimization (DGO) (E‐TDGO) is designed using Advanced Encryption Standard‐128 (AES‐128) and trust‐based optimization model for secure routing in MANET. The proposed E‐TDGO protocol includes three phases, namely, k‐path discovery, optimal path selection, and communication. At first, k paths are discovered based on the distance and the trust level of the nodes. From the k paths discovered, the optimal path is selected using a novel algorithm, DGO, which is developed by combining glowworm swarm optimization (GSO) algorithm and dolphin echolocation algorithm (DEA). Once the optimal path is selected, communication begins in the network such that E‐TDGO protocol ensures security. The routing messages are encrypted using AES‐128 with shared code and key to offer security. The experimental results show that the proposed E‐TDGO could attain throughput of 0.11, delay of 0.01 second, packet drop of 0.44, and detection rate of 0.99, at the maximum number of rounds considered in the network of 75 nodes with attack consideration.