Distributed computing as a virtual supercomputer: Tools to run and manage large-scale BOINC simulations

Distributed computing (DC) projects tackle large computational problems by exploiting the donated processing power of thousands of volunteered computers, connected through the Internet. To efficiently employ the computational resources of one of world's largest DC efforts, GPUGRID, the project scientists require tools that handle hundreds of thousands of tasks which run asynchronously and generate gigabytes of data every day. We describe RBoinc, an interface that allows computational scientists to embed the DC methodology into the daily work-flow of high-throughput experiments. By extending the Berkeley Open Infrastructure for Network Computing (BOINC), the leading open-source middleware for current DC projects, with mechanisms to submit and manage large-scale distributed computations from individual workstations, RBoinc turns distributed grids into cost-effective virtual resources that can be employed by researchers in work-flows similar to conventional supercomputers. The GPUGRID project is currently using RBoinc for all of its in silico experiments based on molecular dynamics methods, including the determination of binding free energies and free energy profiles in all-atom models of biomolecules.     

Metrics of Efficiency and Performance when Using Evolutionary Algorithm on Desktop Grids

Programming and Computer Software - The scope of volunteer computing systems is permanently expanding. The Berkeley Open Infrastructure for Network Computing (BOINC) is currently the most popular...     

ACI试点系统中资源管理模块的设计与实现

先进计算基础设施（ACI）是把地理上分布的高性能计算系统和其他资源用高速网络连接起来的一种基础设施。它是促进高性能计算的应用，推动跨地区，跨学科的科研合作的重要基础设施。为了在ACI试点系统的管理系统ACI－Manager中实现有效的资源共享和管理，本文设计并实现了ACI试点系统中的资源管理模块，在该模块中，资源被分为计算资源和信息资源两大类，并分别面向任务调度和用户操作实现了对两类资源不同方式的管理，实际应用明，这种设计与实现方式能够很好地解决资源的共享和管理问题。     

On the powerful use of simulations in the Quake-Catcher Network to efficiently position low-cost earthquake sensors

The Quake-Catcher Network (QCN) represents a paradigm shift in seismic networks by involving the general public in the collection, detection, and recognition of seismic events. The QCN uses low-cost sensors connected to volunteer computers across the world to monitor seismic events. The location and density of these sensors can impact the accuracy of event detection. Testing different arrangements of new sensors could disrupt the currently active project; thus such an experiment is best accomplished in a simulated environment.This paper presents an accurate and efficient framework for simulating low-cost QCN sensors and identifying their most effective locations and densities. To build the framework, we extend EmBOINC, an emulator of Berkeley Open Infrastructure for Network Computing (BOINC) projects, to handle the trickle messages generated by sensors connected to volunteer hosts and sent to the QCN server when strong ground motion is detected. EmBOINC allows us to rigorously study QCN simulations at 100,000 or even 1,000,000 sensors, highlight strengths and weaknesses of different sensor density and placement, and test the network with various parameters, conditions, and earthquake scenarios. Results obtained with EmBOINC and presented in this paper show how our simulations can reliably study diverse sensor densities and seismic scenarios under different geographical and infrastructural constraints.     

The Virtual Laboratory: a toolset to enable distributed molecular modelling for drug design on the World‐Wide Grid

Computational Grids are emerging as a new paradigm for sharing and aggregation of geographically distributed resources for solving large‐scale compute and data intensive problems in science, engineering and commerce. However, application development, resource management and scheduling in these environments is a complex undertaking. In this paper, we illustrate the development of a Virtual Laboratory environment by leveraging existing Grid technologies to enable molecular modelling for drug design on geographically distributed resources. It involves screening millions of compounds in the chemical database (CDB) against a protein target to identify those with potential use for drug design. We have used the Nimrod‐G parameter specification language to transform the existing molecular docking application into a parameter sweep application for executing on distributed systems. We have developed new tools for enabling access to ligand records/molecules in the CDB from remote resources. The Nimrod‐G resource broker along with molecule CDB data broker is used for scheduling and on‐demand processing of docking jobs on the World‐Wide Grid (WWG) resources. The results demonstrate the ease of use and power of the Nimrod‐G and virtual laboratory tools for grid computing. Copyright © 2003 John Wiley & Sons, Ltd.     

A multipath resource updating approach for distributed controllers in software-defined network软件定义网络中一种分布式控制器间的多路径资源更新方法

Finding effective ways to collect the usage of network resources in all kinds of applications to ensure a distributed control plane has become a key requirement to improve the controller’s decision making performance. This paper explores an efficient way in combining dynamic NetView sharing of distributed controllers with the behavior of intra-service resource announcements and processing requirements that occur in distributed controllers, and proposes a rapid multipathing distribution mechanism. Firstly, we establish a resource collecting model and prove that the prisoner’s dilemma problem exists in the distributed resource collecting process in the Software-defined Network (SDN). Secondly, we present a bypass path selection algorithm and a diffluence algorithm based on Q-learning to settle the above dilemma. At last, simulation results are given to prove that the proposed approach is competent to improve the resource collecting efficiency by the mechanism of self-adaptive path transmission ratio of our approach, which can ensure high utilization of the total network we set up.     

先进计算基础设施(ACI)试验平台软件系统的设计和实现

本文阐述先进计算基础设施（ACI）的重要性和发展情况，ACI是把地理分布位置不同的高性能计算机，用高速网络联接起来，并用中间件使它对用户来说，可看成是统一映象的虚拟系统，本文说明了ACI实验平台的建立及其结构，描述了开发的中间件软件，包括图形用户界面，用户管理，任务管理，资源管理，安全管理和计费管理，最后，本文给出了ACI的计算资源库和ACI的一些典型应用。     

昊宇网络计算平台体系结构研究

利用连接到Internet上的大量计算机的空闲资源 ,网络计算平台能够以较低的代价 ,获得极大的计算能力 提出了昊宇网络计算平台的设计 ,该平台并不针对于某个特定的应用程序设计 ,任何大计算量的、可以分解成独立计算子任务的应用都可以方便地利用该平台运算 ;该平台具有可伸缩性、高可用性、容错性、有效调度以及强大的计算能力 还在昊宇网络计算平台上运行了一个典型的分布式应用程序     

A taxonomy and survey of grid resource management systems for distributed computing

The resource management system is the central component of distributed network computing systems. There have been many projects focused on network computing that have designed and implemented resource management systems with a variety of architectures and services. In this paper, an abstract model and a comprehensive taxonomy for describing resource management architectures is developed. The taxonomy is used to identify approaches followed in the implementation of existing resource management systems for very large‐scale network computing systems known as Grids. The taxonomy and the survey results are used to identify architectural approaches and issues that have not been fully explored in the research. Copyright © 2001 John Wiley & Sons, Ltd.     

MidHDC: Advanced topics on middleware services for heterogeneous distributed computing. Part 2

Currently distributes systems support different computing paradigms like Cluster Computing, Grid Computing, Peer-to-Peer Computing, and Cloud Computing all involving elements of heterogeneity. These computing distributed systems are often characterized by a variety of resources that may or may not be coupled with specific platforms or environments. All these topics challenge today researchers, due to the strong dynamic behavior of the user communities and of resource collections they use.The second part of this special issue presents advances in allocation algorithms, service selection, VM consolidation and mobility policies, scheduling multiple virtual environments and scientific workflows, optimization in scheduling process, energy-aware scheduling models, failure Recovery in shared Big Data processing systems, distributed transaction processing middleware, data storage, trust evaluation, information diffusion, mobile systems, integration of robots in Cloud systems.     

一种基于聚类的虚拟计算环境资源聚合方法

作为面向互联网资源共享的虚拟计算环境的实例,iVCE(Internet based virtual computing environment) for Memory致力于解决广域分布的内存资源的共享与综合利用问题.由于内存资源的特殊性,传统的资源管理方法很难适用.以iVCE for Memory作为背景,提出一种基于聚类的虚拟计算环境资源聚合方法,有效降低了资源聚合的问题规模;借鉴物理学中的力场和势能理论,建立了实现资源聚合的基本模型和力场-势能模型以及相应的分布式算法;通过基于真实网络拓扑的模拟,对两种模型和算法分别进行了评估和验证.     

P2P-Grid结构模型研究与设计

网格和P2P网络技术在当今分布式计算研究中有十分重要的意义和地位,这两种技术都是通过汇聚互联网中来自不同机构的数量庞大的资源,来满足用户之间的资源共享需求。但是它们在具体实现与应用中遇到一定困难。文中在分析了其各自的优点和不足的同时,比较了它们的相似性,提出了一种结合二者技术优势的新的分布式模型———P2P-Grid模型。它是由一些被称之为Super-Peer的小型的网格系统通过P2P技术互连构成。在充分比较该模型与银行运作模式的基础上,设计了整体结构模型,包括系统结构模型、功能模型和任务调度模型。     

ARMCO: Advanced topics in resource management for ubiquitous cloud computing: An adaptive approach

Cloud Computing can be seen as one of the latest major evolution in computing offering unlimited possibility to use ICT in various domains: business, smart cities, medicine, environmental computing, mobile systems, design and implementation of cyber-infrastructures. The recent expansion of Cloud Systems has led to adapting resource management solutions for large number of wide distributed and heterogeneous datacenters. The adaptive methods used in this context are oriented on: self-stabilizing, self-organizing and autonomic systems; dynamic, adaptive and machine learning based distributed algorithms; fault tolerance, reliability, availability of distributed systems. The pay-per-use economic model of Cloud Computing comes with a new challenge: maximizing the profit for service providers, minimizing the total cost for customers and being friendly with the environment.This special issue presents advances in virtual machine assignment and placement, multi-objective and multi-constraints job scheduling, resource management in federated Clouds and in heterogeneous environments, dynamic topology for data distribution, workflow performance improvement, energy efficiency techniques and assurance of Service Level Agreements.     

Sharc: managing CPU and network bandwidth in shared clusters

We argue the need for effective resource management mechanisms for sharing resources in commodity clusters. To address this issue, we present the design of Sharc-a system that enables resource sharing among applications in such clusters. Sharc depends on single node resource management mechanisms such as reservations or shares, and extends the benefits of such mechanisms to clustered environments. We present techniques for managing two important resources-CPU and network interface bandwidth-on a cluster-wide basis. Our techniques allow Sharc to 1) support reservation of CPU and network interface bandwidth for distributed applications, 2) dynamically allocate resources based on past usage, and 3) provide performance isolation to applications. Our experimental evaluation has shown that Sharc can scale to 256 node clusters running 100,000 applications. These results demonstrate that Sharc can be an effective approach for sharing resources among competing applications in moderate size clusters.     

基于网格计算的校园数字化网络建设

网格计算（Grid Computing）,伴随着互联网技术迅速发展起来。它将地理上分布的计算资源充分整合,以协同解决大规模的复杂问题。本文分析了校园网的局限性,将网格计算技术引入到校园网中,以期解决校园网内资源共享问题。     

Time-Series Forecast Modeling on High-Bandwidth Network Measurements

With the increasing number of geographically distributed scientific collaborations and the growing sizes of scientific data, it has become challenging for users to achieve the best possible network performance on a shared network. We have developed a model to forecast expected bandwidth utilization on high-bandwidth wide area networks. The forecast model can improve the efficiency of the resource utilization and scheduling of data movements on high-bandwidth networks to accommodate ever increasing data volume for large-scale scientific data applications. A univariate time-series forecast model is developed with the Seasonal decomposition of Time series by Loess (STL) and the AutoRegressive Integrated Moving Average (ARIMA) on Simple Network Management Protocol (SNMP) path utilization measurement data. Compared with the traditional approach such as Box-Jenkins methodology to train the ARIMA model, our forecast model reduces computation time up to 92.6 %. It also shows resilience against abrupt network usage changes. Our forecast model conducts the large number of multi-step forecast, and the forecast errors are within the mean absolute deviation (MAD) of the monitored measurements.     

Collective agility,paradox and organizational improvisation: the development of a particle physics grid

This paper examines systems development in a global collaborative community of high‐energy physics and offers insights and implications for agile systems development in other large scale and distributed settings. The paper studies the ongoing construction of the UK's computing grid for particle physics (GridPP), a grid that is itself part of the world's largest grid, the Large Hadron Collider Computing Grid. We observe in this project a collective, agile and distributed performance through which the Grid is constructed. We express this through the concept of ‘collective agility’ which captures a large distributed performance rather than the more conventional sense of agility as small‐group and deliberate systems development practices. The collective agility of GridPP is analysed as a process of ‘enacted emergence’ expressed through the dynamics of six improvisation paradoxes.     

基于RDF多代理网络管理模型的设计与实现

网络管理是典型的需要数据共享和互操作的分布式环境。利用强大的语义支撑工具资源描述框架RDF及其扩展,提出了一个新型的多代理网络管理知识模型。该知识模型能有效地进行数据共享和互操作,其中采用了本体方法论。     

基于模态逻辑的网络测量策略模型

当前大规模网络测量系统中不维护状态,因此也很难支持可靠的协作测量.针对这一问题提出了一种基于模态逻辑的网络测量策略模型(ML-NMPM),定义了基于模态逻辑的策略语言和执行模型. ML-NMPM将大规模分布式系统分成自治管理的对等监测域,通过策略代理维护资源状态信息形成全局状态视图,从而支持基于系统状态的协作测量.提高了测量系统的灵活性、可靠性和协作能力.最后分析了ML-NMPM的性能并给出原型系统.