Modelling and analysis of pipelined circuit switching in interconnection networks with bursty traffic and hot-spot destinations

Pipelined circuit switching (PCS) that combines the advantages of both circuit switching and wormhole switching is an efficient method for passing messages in interconnection networks. Analytical modelling is a cost-effective tool and plays an important role in achieving a clear understanding of the network performance. However, most of the existing models for PCS are unable to capture the realistic nature of message behaviours generated by real-world applications, which have a significant impact on the design and performance of communication networks. This paper presents a new analytical model for PCS in interconnection networks in the presence of bursty and correlated message arrivals coupled with hot-spot destinations, which can capture the bursty message arrival process and non-uniform distribution of message destinations. Such a traffic pattern has been found in many practical communication environments. The accuracy of the proposed analytical model is validated through extensive simulation experiments. The model is then applied to investigate the effects of the bursty message arrivals and hot-spot destinations on the performance of interconnection networks with PCS.     

Communication Delay in Wormhole-Switched Tori Networks under Bursty Workloads

Workloads generated by the real-world parallel applications that are executed on a multicomputer have a strong effect on the performance of its interconnection network—the hardware fabric supporting communication among individual processors. Existing multicomputer networks have been primarily designed and analysed under the assumption that the workload follows the non-bursty Poisson arrival process. As a step towards obtaining a clear understanding of network performance under various workloads, this paper presents a new analytical model for computing message latency in wormhole switched torus networks in the presence of bursty traffic, based on the well-known Markov-Modulated Poisson Process (MMPP). In order to derive the model, the approach for accurately capturing the properties of the composite MMPPs is applied to characterize traffic on network channels. Moreover, a general method has been proposed for calculating the probability of virtual channel occupancy when the traffic on network channels follows a multi-state MMPP process. Simulation experiments reveal that the model exhibits a good degree of accuracy.     

Analytical modelling of networks in multicomputer systems under bursty and batch arrival traffic

The hypercube and torus are two important message-passing network architectures of high-performance multicomputers. Analytical models of multicomputer networks under the non-bursty Poisson traffic have been widely reported. Motivated by the convincing evidence of bursty and batch arrival nature of traffic generated by many real-world parallel applications in high-performance computing environments, we develop a new and concise analytical model in this paper for hypercube and torus networks in the presence of batch message arrivals modelled by the compound Poisson process with geometrically distributed batch sizes. The average degree of virtual channel multiplexing is derived by employing a Markov chain which can capture the batch arrival nature. An attractive advantage of the model is its constant computation complexity independent of the network size. The accuracy of the analytical performance results is validated against those obtained from simulation experiments of an actual system.     

Adaptive routing in wormhole-switched necklace-cubes: Analytical modelling and performance comparison

The necklace hypercube has recently been introduced as an attractive alternative to the well-known hypercube. Previous research on this network topology has mainly focused on topological properties, VLSI and algorithmic aspects of this network. Several analytical models have been proposed in the literature for different interconnection networks, as the most cost-effective tools to evaluate the performance merits of such systems. This paper proposes an analytical performance model to predict message latency in wormhole-switched necklace hypercube interconnection networks with fully adaptive routing. The analysis focuses on a fully adaptive routing algorithm which has been shown to be the most effective for necklace hypercube networks. The results obtained from simulation experiments confirm that the proposed model exhibits a good accuracy under different operating conditions.     

An accurate mathematical performance model of adaptive routing in the star graph

Analytical modelling is indeed the most cost-effective method to evaluate the performance of a system. Several analytical models have been proposed in the literature for different interconnection network systems. This paper proposes an accurate analytical model to predict message latency in wormhole-switched star graphs with fully adaptive routing. Although the focus of this research is on the star graph but the approach used for modelling can be, however, used for modelling some other regular and irregular interconnection networks. The results obtained from simulation experiments confirm that the proposed model exhibits a good accuracy for various network sizes and under different operating conditions.     

Message latency in hypercubic computer networks with the bursty traffic pattern

An interconnection network is a crucial component of parallel computers because the overall system performance is very sensitive to the latency of messages delivered by the network to communicate among collaborating processors. This paper presents an analytical performance model to calculate message latency in circuit-switched hypercubic networks in the presence of bursty traffic pattern, which is a typical scenario for multimedia applications. A message in circuit switching may need a number of connection attempts before successfully setting up a path from source to destination. The proposed model uses the approach of superposing infinite bursty traffic streams to capture the effective traffic entering the network from a source node, which includes the traffic generated by the source and those due to many connection attempts. Results obtained from simulation experiments confirm that the proposed model exhibits a good degree of accuracy for various network sizes and under different operating conditions.     

Stochastic Analysis of Deterministic Routing Algorithms in the Presence of Self-Similar Traffic

Many performance models for deterministic routing in multicomputer interconnection networks have been derived and analyzed under the assumption of the traditional Poisson stochastic arrival process, which is inherently unable to capture traffic self-similarity revealed by many real-world parallel applications. In an effort towards understanding the network performance under various traffic loads and different design alternatives, this paper presents an analytical model for dimension-ordered routing in k-ary n-cubes when subjected to self-similar traffic. As the service time, blocking probability and waiting time experienced by a message vary from a dimension to another, the design of such a model for dimension-ordered routing poses greater challenges. The developed analytical model is then used to investigate the efficiency of two different ways to organize virtual channels for deterministic routing and to evaluate the impact of self-similar traffic with various Hurst parameters on network performance.     

Prediction of communication delay in torus networks under multiple time-scale correlated traffic

The efficiency of a large-scale multicomputer is critically dependent on the performance of its interconnection network. Current multicomputers have widely employed the torus as their underlying network topology for efficient interprocessor communication. In order to ensure a successful exploitation of the computational power offered by multicomputers it is essential to obtain a clear understanding of the performance capabilities of their interconnection networks under various system configurations. Analytical modelling plays an important role in achieving this goal. This study proposes a concise performance model for computing communication delay in the torus network with circuit switching in the presence of multiple time-scale correlated traffic which is found in many real-world parallel computation environments and has strong impact on network performance. The tractability and reasonable accuracy of the analytical model demonstrated by extensive simulation experiments make it a practical and cost-effective evaluation tool to investigate network performance with various alternative design solutions and under different operating conditions.     

基于InfiniBand的多链路mesh/torus大规模并行系统互连网络

在大规模并行系统中,系统级互连网络的设计至关重要.InfiniBand作为一种高性能交换式网络被广泛应用于大规模并行处理系统中.mesh/torus拓扑结构相较于目前普遍应用于InfiniBand网络的胖树拓扑结构拥有更好的性能与可扩展性.尽管如此,研究发现,用传统的mesh/torus拓扑结构构建InfiniBand互连网络存在诸多问题.分析了传统网络拓扑结构的缺陷,并提出了一种基于InfiniBand的多链路mesh/torus互连网络.这种改进型的拓扑结构通过充分利用交换机间的多链路可以获得比传统mesh/torus网络更高的带宽.另外,同时给出了与该网络拓扑结构相配套的高效路由算法.最后,通过网络仿真技术对提出的算法进行了评估,实验结果显示提出的路由算法相较于其他路由算法拥有更好的性能与可扩展性.     

Modeling the effects of hot-spot traffic load on the performance of wormhole-switched hypermeshes

Hypermesh is a promising network topology and is suitable for a range of network-based computing systems. Although there are few models reported for hypermeshes with uniform traffic pattern, no analytical model has been reported to deal with hot-spot traffic. Since many parallel applications exhibit non-uniform traffic patterns such as hot-spots, uniform traffic assumption is not always justifiable in practice. In this study, we propose a new analytical model to predict the mean message latency in wormhole-switched hypermeshes in the presence of hot-spot traffic. The proposed model can also calculate the mean latency under uniform traffic load when the hot-spot ratio is set to zero. Simulation results show the validity and the reasonable accuracy of the model even under heavy traffic loads and near saturation region.     

Analytical modeling of interconnection networks in heterogeneous multi-cluster systems

The study of interconnection networks is important because the overall performance of a distributed system is often critically hinged on the effectiveness of its interconnection network. This paper addresses the problem of interconnection networks performance modeling of large-scale distributed systems with emphases on heterogeneous multi-cluster computing systems. We present an analytical model to predict message latency in multi-cluster systems in the presence of node, network and system organization heterogeneity. The model is validated through comprehensive simulation, which demonstrates that the proposed model exhibits a good degree of accuracy for various system organizations and under different working conditions.

片上光网络:一种新型片上互连网络

随着单个芯片上集成的处理器的个数越来越多,传统的电互连网络已经无法满足对互连网络性能的需求,需要一种新的互连方式,因此光互连网络技术应运而生.目前,电互连的片上网络在功耗、性能、带宽、延迟等方面遇到了瓶颈,而光互连作为一种新的互连方式引用到片上网络具有低损耗、高吞吐率、低延迟等无可比拟的优势.本文主要探讨了片上光网络的...     

The impact of virtual channel allocation on the performance of deterministic wormhole-routed k-ary n-cubes

Virtual channels yield significant improvement in the performance of wormhole-routed networks as they can greatly reduce message blocking over network resources. K-ary n-cubes with deterministic routing have been widely analysed using analytical modelling tools. Most existing models, however, have either entirely ignored the effects of virtual channel multiplexing or have not considered the impact of virtual channels allocation on message latency. This paper discusses two different organisations of virtual channels in k-ary n-cubes, resulting in two deterministic routing algorithms. It then proposes an analytical model to compute message latency for the two routing algorithms. The proposed model is used in a case study to demonstrate the sensitivity of network latency to the way virtual channels are allocated to messages.     

Analyzing and measuring the latency of the Totem multicast protocols

This article presents, by analysis and measurement, the probability density functions (pdfs) for the latency from origination to message delivery for the Totem multicast protocols, in the presence of message loss and token loss. The Totem protocols provide reliable totally ordered delivery of messages across single and multiple local-area networks (LANs), using a logical token-passing ring on each LAN with gateways that forward messages selectively between LANs. The analysis of the pdfs for the latency involves decomposing the latency into independent components and convolving the pdfs of those components. A comparison of the performance of single-ring, two-ring and four-ring networks shows that, with message filtering in the gateways, multiple-ring networks achieve lower mean latency, less variability, and shorter tails of the latency distribution than an equivalent single ring. The experimental measurements of the pdfs for the latency are similar to those obtained from the analytical formulas, particularly for values of the latency with high probabilities. Limitations of the analytical model, and insight into communication and scheduling of the operating system, are discussed.     

新型互连网络NIN研究

以通信延迟和网络吞吐率这两个重要参数为出发点,以大规模并行处理系统的新型互连网络（ＮＩＮ）进行了理论分析和模拟测试,并与二维网格、环形网以及反图拓扑互连网络进行了比较,结果表明：与其它三种网络相比,ＮＩＮ不仅保持了反图拓扑互连网络可以连接更多处理机的优点,同时还具有较短的通信延迟和较高的网络吞吐率。     

Communication delay analysis of fault-tolerant pipelined circuit switching in torus

Large-scale parallel systems, Multiprocessors System-on-Chip (MP-SoCs), multicomputers, and cluster computers are often composed of hundreds or thousands of components (such as routers, channels and connectors) that collectively possess failure rates higher than what arise in the ordinary systems. One of the most important issues in the design of such systems is the development of the efficient fault-tolerant mechanisms that provide high throughput and low latency in communications to ensure that these systems will keep running in a degraded mode until the faulty components are repaired. Pipelined Circuit Switching (PCS) has been suggested as an efficient switching method for supporting inter-processor communications in networks due to its ability to preserve both communication performance and fault-tolerant demands in such systems. This paper presents a new mathematical model to investigate the effects of failures and capture the mean message latency in torus using PCS in the presence of faulty components. Simulation experiments confirm that the analytical model exhibits a good degree of accuracy under different working conditions.     

Performance analysis of a QoS capable cluster interconnect

The growing use of clusters in diverse applications, many of which have real-time constraints, requires quality-of-service (QoS) support from the underlying cluster interconnect. All prior studies on QoS-aware cluster routers/networks have used simulation for performance evaluation. In this paper, we present an analytical model for a wormhole-switched router with QoS provisioning. In particular, the model captures message blocking due to wormhole switching in a pipelined router, and bandwidth sharing due to a rate-based scheduling mechanism, called VirtualClock. Then we extend the model to a hypercube-style cluster network. Average message latency for different traffic classes and deadline missing probability for real-time applications are computed using the model.

We evaluate a 16-port router and hypercubes of different dimensions with a mixed workload of real-time and best-effort (BE) traffic. Comparison with the simulation results shows that the single router and the network models are quite accurate in providing the performance estimates, and thus can be used as efficient design tools.     

多机互连网络与并行算法结构关系的分析及其在Transputer网上的仿真研究

本文通过对多机互连网络建模,着重分析了并行算法结构对各种拓扑互连结构性能的影响,以及多机系统中用于结点间交换信息的通信开销,并对该模型在Transputer网上用TRANSIM进行了仿真研究。     

There is no optimal routing policy for the torus

A routing policy is the method used to select a specific output channel for a message from among a number of acceptable output channels. An optimal routing policy is a policy that maximizes the probability of a message reaching its destination without delays. Optimal routing policies have been proposed for several regular networks, including the mesh and the hypercube. An open problem in interconnection network research has been the identification of an optimal routing policy for the torus. In this paper, we show that there is no optimal routing policy for the torus. Our result is demonstrated by presenting a detailed example in which the best choice of output channel is dependent on the probability of each channel being available. This result settles, in the negative, a conjecture by Wu concerning an optimal routing policy for the torus.     

An analytical performance model for the Spidergon NoC with virtual channels

The Spidergon Network-on-Chip (NoC) was proposed to address the demand for a fixed and optimized communication infrastructure for cost-effective multi-processor Systems-on-Chip (MPSoC) development. To deal with the increasing diversity in quality of service requirements of SoC applications, the performance of this architecture needs to be improved. Virtual channels have traditionally been employed to enhance the performance of the interconnect networks. In this paper, we present analytical models to evaluate the message latency and network throughput in the Spidergon NoC and investigate the effect of employing virtual channels. Results obtained through simulation experiments show that the model exhibits a good degree of accuracy in predicting average message latency under various working conditions. Moreover an FPGA implementation of the Spidergon has been developed to provide an accurate analysis of the cost of employing virtual channels in this architecture.