Evaluation of Two Programming Paradigms for Heterogeneous Computing

In this paper, we evaluate two different programming paradigms for heterogeneous computing, Cluster-M and Heterogeneous Associative Computing (HAsC). These paradigms can efficiently support heterogeneous networks by preserving a level of abstraction without containing any architectural details. The paradigms are architecturally independent and scalable for various network and problem sizes. Cluster-M can be applied to both coarse-grained and fine-grained networks. Cluster-M provides an environment for porting heterogeneous tasks onto the machines in a heterogeneous suite such that resource utilization is maximized and the overall execution time is minimized. HAsC models a heterogeneous network as a coarse-grained associative computer. It is designed to optimize the execution of problems where the program size is small compared with the amount of data processed. Unlike other existing heterogeneous orchestration tools which are MIMD based, HAsC is for data-parallel SIMD associative computing. Ease of programming and execution speed are the primary goals of HAsC. We evaluate how these two paradigms can be used together to provide an efficient scheme for heterogeneous programming. Finally, their scalability issues are discussed.     

Distributed computing methodology for training neural networks in an image-guided diagnostic application

Distributed computing is a process through which a set of computers connected by a network is used collectively to solve a single problem. In this paper, we propose a distributed computing methodology for training neural networks for the detection of lesions in colonoscopy. Our approach is based on partitioning the training set across multiple processors using a parallel virtual machine. In this way, interconnected computers of varied architectures can be used for the distributed evaluation of the error function and gradient values, and, thus, training neural networks utilizing various learning methods. The proposed methodology has large granularity and low synchronization, and has been implemented and tested. Our results indicate that the parallel virtual machine implementation of the training algorithms developed leads to considerable speedup, especially when large network architectures and training sets are used.     

Experiences with networked parallel computing

The performance and proliferation of workstations continues to increase at a rapid rate. However, the practical utilization of workstation networks for parallel computing is still in its infancy. This is due to the relative immaturity of programming tools, low bandwidth networks such as Ethernet, and high message latencies. However, programming tools are becoming more mature and network bandwidths are increasing rapidly. Hence, networks of commodity workstations may prove to be practical for certain classes of parallel applications. This paper describes our experiences with two applications parallelized on a network of Sun workstations. The first application is from Shell's petroleum engineering department. This program quantitatively derives rock and porefill composition from well-log data, using a compute-intensive iterative optimization procedure. The second application is time filtering, which is a fundamental operation performed on seismic traces. Through our experiments we identify the limits of networked parallel computing based on the current state of network technology. We also provide a discussion on the possible impact of future high speed networks on networked parallel computing.     

Fusing functional connectivity with network nodal information for sparse network pattern learning of functional brain networks

Sparse learning methods have been powerful tools for learning compact representations of functional brain networks consisting of a set of brain network nodes and a connectivity matrix measuring functional coherence between the nodes. However, these tools typically focus on the functional connectivity measures alone, ignoring the brain network nodal information that is complementary to the functional connectivity measures for comprehensively characterizing the functional brain networks. In order to provide a comprehensive delineation of the functional brain networks, we develop a new data fusion method for heterogeneous data, aiming at learning sparse network patterns to characterize both the functional connectivity measures and their complementary network nodal information within a unified framework. Experimental results have demonstrated that our method outperforms the best alternative method under comparison in terms of accuracy on simulated data as well as both reproducibility and prediction performance of brain age on real resting state functional magnetic resonance imaging data.     

Startup comparison for message passing libraries with DTM on linux clusters

In clusters containing heterogeneous systems, message passing libraries (distributed computing tools) are employed for harnessing the computing and other resources. Task is submitted to a tool and the actual execution is carried out on aggregated network resources. Tools take care of scheduling, distributing subtasks and gathering results along with synchronization and message exchange requirements. They need initialization and synchronization routines for the submitted task. These tools also provide many other features like transparency, fault tolerance and load balancing. Some times all these features or initialization may not be required. The aim of tool designers should be to provide quality performance with add-on request initialization and feature provision. Initialization routines and special features provision take their own time over core distributed computing, affecting overall computational cost. In this paper a two purpose tool (Distributed Task Measure: DTM) is implemented. DTM is primarily used for placing other distributed computing tools on a performance index, judging their startup and performance. DTM may also serve to achieve macro level parallelization where requirements are such.     

Heterogeneous parallel and distributed computing

Heterogeneous network-based distributed and parallel computing is gaining increasing acceptance as an alternative or complementary paradigm to multiprocessor-based parallel processing as well as to conventional supercomputing. While algorithmic and programming aspects of heterogeneous concurrent computing are similar to their parallel processing counterparts, system issues, partitioning and scheduling, and performance aspects are significantly different. In this paper, we discuss the evolution of heterogeneous concurrent computing, in the context of the parallel virtual machine (PVM) system, a widely adopted software system for network computing. In particular, we highlight the system level infrastructures that are required, aspects of parallel algorithm development that most affect performance, system capabilities and limitations, and tools and methodologies for effective computing in heterogeneous networked environments. We also present recent developments and experiences in the PVM project, and comment on ongoing and future work.     

Performance bottlenecks and potentials of parallel computing on networks of workstations

The network of workstations (NOW) we consider for parallel computing is heterogeneous and nondedicated (time-sharing), where computing power varies among the workstations, and multiple jobs may interact with each other in execution. We address three performance issues in this paper. First, we examine the effects of heterogeneity on co-scheduling and local scheduling policies for parallel computing. Through experimentation and quantitative comparisons, we discuss features and requirements of scheduling policies on heterogeneous NOW. Second, the heterogeneity and non-dedication of NOW introduce new performance factors into parallel computing, which make traditional performance metrics for parallel computing under homogeneous platforms not suitable. We conducted a collection of experimental measurements to show the performance impact to parallel computing. Finally, using network latencies we experimentally evaluate the parallel computing scalability on NOW. Our objective of this study is to provide insights into unique performance bottlenecks and potentials of networks of workstations.     

HINMINE: heterogeneous information network mining with information retrieval heuristics

The paper presents an approach to mining heterogeneous information networks by decomposing them into homogeneous networks. The proposed HINMINE methodology is based on previous work that classifies nodes in a heterogeneous network in two steps. In the first step the heterogeneous network is decomposed into one or more homogeneous networks using different connecting nodes. We improve this step by using new methods inspired by weighting of bag-of-words vectors mostly used in information retrieval. The methods assign larger weights to nodes which are more informative and characteristic for a specific class of nodes. In the second step, the resulting homogeneous networks are used to classify data either by network propositionalization or label propagation. We propose an adaptation of the label propagation algorithm to handle imbalanced data and test several classification algorithms in propositionalization. The new methodology is tested on three data sets with different properties. For each data set, we perform a series of experiments and compare different heuristics used in the first step of the methodology. We also use different classifiers which can be used in the second step of the methodology when performing network propositionalization. Our results show that HINMINE, using different network decomposition methods, can significantly improve the performance of the resulting classifiers, and also that using a modified label propagation algorithm is beneficial when the data set is imbalanced.     

Engineering PROFIBUS networks with heterogeneous transmission media

A significant number of process control and factory automation systems use PROFIBUS as the underlying fieldbus communication network. The process of properly setting up a PROFIBUS network is not a straightforward task. In fact, a number of network parameters must be set for guaranteeing the required levels of timeliness and dependability. Engineering PROFIBUS networks is even more subtle when the network includes various physical segments exhibiting heterogeneous specifications, such as bus speed or frame formats, just to mention a few. In this paper we provide underlying theory and a methodology to guarantee the proper operation of such type of heterogeneous PROFIBUS networks. We additionally show how the methodology can be applied to the practical case of PROFIBUS networks containing simultaneously DP (Decentralised Periphery) and PA (Process Automation) segments, two of the most used commercial-off-the-shelf (COTS) PROFIBUS solutions. The importance of the findings is however not limited to this case. The proposed methodology can be generalised to cover other heterogeneous infrastructures. Hybrid wired/wireless solutions are just an example for which an enormous eagerness exists.     

A machine learning based intrusion detection scheme for data fusion in mobile clouds involving heterogeneous client networks

The combination of traditional cloud computing and mobile computing leads to the novel paradigm of mobile cloud computing. Due to the mobility of network nodes in mobile cloud computing, security has been a challenging problem of paramount importance. When a mobile cloud involves heterogeneous client networks, such as Wireless Sensor Networks and Vehicular Networks, the security problem becomes more challenging because the client networks often have different security requirements in terms of computational complexity, power consumption, and security levels. To securely collect and fuse the data from heterogeneous client networks in complex systems of this kind, novel security schemes need to be devised. Intrusion detection is one of the key security functions in mobile clouds involving heterogeneous client networks. A variety of different rule-based intrusion detection methods could be employed in this type of systems. However, the existing intrusion detection schemes lead to high computation complexity or require frequent rule updates, which seriously harms their effectiveness. In this paper, we propose a machine learning based intrusion detection scheme for mobile clouds involving heterogeneous client networks. The proposed scheme does not require rule updates and its complexity can be customized to suit the requirements of the client networks. Technically, the proposed scheme includes two steps: multi-layer traffic screening and decision-based Virtual Machine (VM) selection. Our experimental results indicate that the proposed scheme is highly effective in terms of intrusion detection.     

Prediction of the impact of network switch utilization on application performance via active measurement

Although one of the key characteristics of High Performance Computing (HPC) infrastructures are their fast interconnecting networks, the increasingly large computational capacity of HPC nodes and the subsequent growth of data exchanges between them constitute a potential performance bottleneck. To achieve high performance in parallel executions despite network limitations, application developers require tools to measure their codes’ network utilization and to correlate the network’s communication capacity with the performance of their applications.This paper presents a new methodology to measure and understand network behavior. The approach is based in two different techniques that inject extra network communication. The first technique aims to measure the fraction of the network that is utilized by a software component (an application or an individual task) to determine the existence and severity of network contention. The second injects large amounts of network traffic to study how applications behave on less capable or fully utilized networks. The measurements obtained by these techniques are combined to predict the performance slowdown suffered by a particular software component when it shares the network with others. Predictions are obtained by considering several training sets that use raw data from the two measurement techniques. The sensitivity of the training set size is evaluated by considering 12 different scenarios. Our results find the optimum training set size to be around 200 training points. When optimal data sets are used, the proposed methodology provides predictions with an average error of 9.6% considering 36 scenarios.     

A Grid resource brokering strategy based on resource and network performance in Grid

To achieve high performance distributed data access and computing in Grid environment, monitoring of resource and network performance is vital. Our proposed Grid network monitoring architecture is modeled by the Grid scheduler. The proposed Grid network monitoring retrieves network metrics using sensors as network monitoring tools. The mobile agents are migrated to start the sensors to measure the network metrics in all Grid Resources from the Resource Broker. The raw data provided by the monitoring tools is used to produce a high level view of the Grid through the set of internal cost functions. The network cost function is formed by combining various network metrics such as bandwidth, Round Trip Time, jitter and packet loss to measure the network performance. This paper presents the Grid Resource Brokering strategy which analyzes the network metrics along with the resource metrics for the selection of the Grid resource to submit the job and the proposed approach is integrated with CARE Resource Broker (CRB) for job submission. The experimental results are evident for the minimization of job completion time for the submitted job. The simulation results also prove that the more number of jobs are completed with the proposed strategy which influences the better utilization of the Grid resources.     

An Effective and Practical Performance Prediction Model for Parallel Computing on Nondedicated Heterogeneous NOW

Networks of workstations (NOW) are receiving increased attention as a viable platform for high performance parallel computations. Heterogeneity and time-sharing are two characteristics that distinguish the NOW systems from conventional multiprocessor/multicomputer systems which are homogeneous and dedicated. It is important to have a practical model for users to predict the execution times of large-scale parallel applications on nondedicated heterogeneous NOW. Another objective of this study is to provide insight into the dynamic performance of parallel computing and into the effects of program structures and system factors on such a platform. In this paper, we study performance predictions for parallel computing on nondedicated heterogeneous networks of workstations. Our approach is based on a two-level model. On the top level, a semideterministic task graph is used to capture the parallel execution behavior including the variances of communication and synchronization. On the bottom level, a discrete time model is used to quantify effects from NOW systems. An iterative process is used to determine the interactive effects between network contention and task execution. We validate the prediction model using experiments on a nondedicated heterogeneous NOW. The maximum differences between predicted results and measured results were less than 10% in most cases and 15% in the worst cases.     

异构混合云服务下的多任务算力度量方法

异构网络中混合云服务下的任务数据量复杂程度较高、数据维度高,影响多任务算力度量的效率及完成率,提出了异构混合云服务下的多任务算力度量方法。分析异构混合云服务系统,获取多任务算力数据,依据获取结果从中提取多任务算力特征集。利用构建的度量模型对多任务算力特征进行标识及量化,通过对模型及特征的实例相似性计算,建立多任务算力度量矩阵,根据该矩阵实现多任务算力度量。实验结果表明：通过对该方法实施度量性能及度量效果对比测试,验证了该方法对任务算力度量的耗时短,多任务算力度量完成率较高。     

Enterprise Network Traffic Monitoring, Analysis, and Reporting Using Web Technology

Today's enterprise networks are composed of multiple types of interconnected networks. Furthermore, organizations use a variety of systems and applications on these networks. Operations and management staff must provide an efficient, reliable and secure operating environment to support an organization's daily activities. Enterprise networks must be monitored for performance, configuration, security, accounting and fault management. Current management practices typically involve the use of complex, hard-to-learn and hard-to-use tools. What is needed desperately is a set of simple, uniform, ubiquitous tools for managing networks. Web-based management promises to provide such solutions. This paper focuses on the use of Web technology and the Multi-Router Traffic Grapher (MRTG) for the purposes of enterprise network traffic monitoring and reporting. In this paper, we first examine the requirements for enterprise network traffic monitoring, analysis and reporting, and then present the design and implementation of a Web-based network traffic monitoring and reporting system that satisfies those requirements. We also present guidelines we have formulated and used for analyzing enterprise network traffic. We then discuss our experiences in using such a system for traffic monitoring on two large enterprise networks.     

Leveraging deployment models on low-resource devices for cloud services in community networks

Community networks are crowd-sourced IP networks that evolved into regional-scale computing platforms. This has led to adapting the cloud computing model for services that can operate and use computing resources inside a community network. The network and computing infrastructure is contributed by individuals, companies, organizations and maintained by its members. Community cloud devices are often low-capacity computing devices, such as home gateways or cabinet servers, with limited capabilities. These devices are used to install and operate specific personal or community services, but can be turned into multi-purpose execution environments applying machine or operating system (container) virtualization. However that requires addressing the problems of resource sharing in low-capacity devices, related to predictable performance and isolation. Our comparative analysis with the current infrastructure in community networks gives evidence about how devices can concurrently run multiple services, the trade offs between the number and resource requirements of services and the degradation of quality that services may suffer.     

Intelligent social network analysis using granular computing

An introduction to some basic ideas of graph (relational network) theory is first provided. We then discuss some concepts from granular computing in particular the fuzzy set paradigm of computing with words. The natural connection between graph theory and granular computing, particularly fuzzy set theory, is pointed out. This connection is grounded in the fact that these are both set‐based technologies. Our objective here is to take a step toward the development of intelligent social network analysis using granular computing. In particular one can start by expressing in a human‐focused manner concepts associated with social networks then formalize these concepts using fuzzy sets and then evaluate these concepts with respect to social networks that have been represented using set‐based relational network theory. We capture this approach in what we call the paradigm for intelligent social network analysis, PISNA. Using this paradigm, we provide definitions of a number of concepts related to social networks. © 2008 Wiley Periodicals, Inc.     

A network multi-processor for experiments in parallelism

Although both shared memory and loosely coupled parallel computing systems are now common, many still do not offer an easy way to design, implement, and test parallel algorithms. Our system provides software tools that make possible a variety of connection structures between processes. These structures are said to form a 'Network Multi-Processor', which is implemented on a local area network of heterogeneous UNIX-based timesharing computers, plus a set of processor boards dedicated to an application so that accurate timing measurements can be made. We explain how these tools have been used both to aid parallel algorithm development and to explore the properties of different computer interconnection methods.     

Communication Benchmarking and Performance Modelling of MPI Programs on Cluster Computers

This paper gives an overview of two related tools that we have developed to provide more accurate measurement and modelling of the performance of message-passing communication and application programs on distributed memory parallel computers. MPIBench uses a very precise, globally synchronised clock to measure the performance of MPI communication routines. It can generate probability distributions of communication times, not just the average values produced by other MPI benchmarks. This allows useful insights to be made into the MPI communication performance of parallel computers, and in particular how performance is affected by network contention. The Performance Evaluating Virtual Parallel Machine (PEVPM) provides a simple, fast and accurate technique for modelling and predicting the performance of message-passing parallel programs. It uses a virtual parallel machine to simulate the execution of the parallel program. The effects of network contention can be accurately modelled by sampling from the probability distributions generated by MPIBench. These tools are particularly useful on clusters with commodity Ethernet networks, where relatively high latencies, network congestion and TCP problems can significantly affect communication performance, which is difficult to model accurately using other tools. Experiments with example parallel programs demonstrate that PEVPM gives accurate performance predictions on commodity clusters. We also show that modelling communication performance using average times rather than sampling from probability distributions can give misleading results, particularly for programs running on a large number of processors.     

异构网络环境下最大化谱间距的拓扑设计

分布式平均共识和去中心化机器学习是具有广泛应用的去中心化计算方法.两种方法的收敛率主要由拓扑的谱间距所决定.节点网络环境的异构性包括节点带宽和节点间连接可用性的不同.异构网络环境对去中心化计算的效率提出了挑战.本文研究异构网络环境下最大化谱间距的拓扑设计问题,推导了谱间距针对拓扑任一条边的梯度,并设计了基于该梯度的增删边算法来构建目标拓扑.构建的拓扑具有更大谱间距,且各节点的数据通信时间相近.拓扑构建算法的性能在不同程度的异构网络环境下能够保持稳定,且生成的拓扑在分布式共识中以更快的收敛率和更短的时间达到收敛.基于该算法,本文进一步验证了最新发现的谱间距与去中心化机器学习收敛率的弱相关性.