Workflow-Based Data Parallel Applications on the EGEE Production Grid Infrastructure

Setting up and deploying complex applications on a Grid infrastructure is still challenging and the programming models are rapidly evolving. Efficiently exploiting Grid parallelism is often not straight forward. In this paper, we report on the techniques used for deploying applications on the EGEE production Grid through four experiments coming from completely different scientific areas: nuclear fusion, astrophysics and medical imaging. These applications have in common the need for manipulating huge amounts of data and all are computationally intensive. All the cases studied show that the deployment of data intensive applications require the development of more or less elaborated application-level workload management systems on top of the gLite middleware to efficiently exploit the EGEE Grid resources. In particular, the adoption of high level workflow management systems eases the integration of large scale applications while exploiting Grid parallelism transparently. Different approaches for scientific workflow management are discussed. The MOTEUR workflow manager strategy to efficiently deal with complex data flows is more particularly detailed. Without requiring specific application development, it leads to very significant speed-ups.     

Practical Experience from Porting and Executing the Wien2k Application on the EGEE Production Grid Infrastructure

While the Grid promises to deliver a large number of computation nodes to a user, this computation power is not usable without the proper adaption of the application for the Grid. In this paper, we describe the methods used to port and execute a particular application, Wien2k, on the EGEE production Grid. First, the process of porting the application is described. Then, we investigate the measures necessary to execute the application in this production Grid environment efficiently. Although the focus is on this special application, we describe generic methods which can be applied to all applications. We specifically address: Creating a workflow from an application and mapping this workflow to a Grid workflow using the activity attraction pattern. We discuss workflow engines which support cycles in their application workflow. We investigate naïve and worker scheduling techniques. A short introduction into licensing on the Grid is given. Optimisation techniques such as deployment re-use are discussed. Different data transfer mechanisms, centralised data transfer, data re-use, storage element data transfer, and peer-to-peer data transfer are compared. The paper is concluded with suggestions for further workflow porting.     

Statistical Analysis and Modeling of Jobs in a Grid Environment

The existence of good probabilistic models for the job arrival process and the delay components introduced at different stages of job processing in a Grid environment is important for the improved understanding of the Grid computing concept. In this study, we present a thorough analysis of the job arrival process in the EGEE infrastructure and of the time durations a job spends at different states in the EGEE environment. We define four delay components of the total job delay and model each component separately. We observe that the job inter-arrival times at the Grid level can be adequately modelled by a rounded exponential distribution, while the total job delay (from the time it is generated until the time it completes execution) is dominated by the computing element’s register and queuing times and the worker node’s execution times. Further, we evaluate the efficiency of the EGEE environment by comparing the job total delay performance with that of a hypothetical ideal super-cluster and conclude that we would obtain similar performance if we submitted the same workload to a super-cluster of size equal to 34% of the total average number of CPUs participating in the EGEE infrastructure. We also analyze the job inter-arrival times, the CE’s queuing times, the WN’s execution times, and the data sizes exchanged at the kallisto.hellasgrid.gr cluster, which is node in the EGEE infrastructure. In contrast to the Grid level, we find that at the cluster level the job arrival process exhibits self-similarity/long-range dependence. Finally, we propose simple and intuitive models for the job arrival process and the execution times at the cluster level.     

Resources and Services of the EGEE Production Infrastructure

This paper presents the pan-European EGEE Grid focusing on aspects such as production infrastructure, the management tools and the operational services offered. Usage statistics and the provided Quality of Service are analysed to assess the maturity level, the current penetration of Grid technologies in Europe and the current expansion trends. Being EGEE a large distributed infrastructure, operations are a joint effort of different regional centres with central coordination. EGEE operations rely on a common and agreed set of procedures, policies and interfaces, which are the foundation of operational services such as middleware deployment, Grid oversight, accounting, operational security management and support. A transition is in place to lead EGEE to a more sustainable approach based on a set of integrated National Grid Initiatives. With the support of the EGI-InSPIRE project the EGEE e-infrastructure and its services will migrate into a new governance model for the future sustainability of Grids in Europe.     

Availability Modeling and Cost Optimization for the Grid Resource Management System

Grid computing is a recently developed technique for complex systems with large-scale resource sharing, wide-area communication, and multi-institutional collaboration. Although the development tools and techniques for the grid have been extensively investigated, the availability of the grid resource management system (RMS) has not been comprehensively studied. In order to contribute to this lacking but important field, this paper first models the grid RMS availability by considering both the failures of resource management (RM) servers and the length limitation of request queues. A hierarchical Markov reward model is implemented to evaluate the grid RMS availability. Based on the availability model, an optimization problem for designing the grid RMS is studied in order to minimize the cost by determining the best number of RM servers. Then, the sensitivity analysis is conducted, and a dynamic switching scheduling method is further presented based on the sensitivity analysis.     

利用ASP.NET的Web数据网格控件实现远程批量提交的方法

该文以班级德育量化考评数据批量提交为实例,重点阐述了利用Web数据网格控件GridView实现基于网页的批量更新提交的方法。     

基于Web的网格作业提交系统的研究与实现

作业管理是用户使用网格服务和网格资源的接口.本文提出了通用的作业管理系统框图和基于Web的作业提交模型,并详细分析了作业提交的过程,时用户管理、提交服务和任务处理等关键问题的实现做了阐述.     

断层地质体的三维建模技术研究

对复杂地质体中的断层三维建模技术进行研究与讨论。分析断层三维数据来源给出数据空间模型;系统地对‘已有的断层三维建模的技术进行分析比较,在此基础上提出一种使用TIN和三棱柱混合数据模型的断层地质体建模技术,从而有效地解决了断层三维可视化问题。     

整理车间多Agent生产管理系统的建模与过程优化

借助多Agent技术,提出了一种基于多Agent的生产管理结构模型,对用户实际需求和结构模型的合理性进行了分析.在此基础上,开发了一个柔性、动态、高效的生产管理系统,对车间目前的生产管理模式进行了优化.同时,对多个体Agent的主要功能、工作原理和系统建模过程进行了介绍.实际应用表明,多Agent的生产管理系统,解决了...     

基于人工鱼群算法的高级综合生产计划优化研究

充分考虑了多产品多阶段多分销商企业生产过程各种影响因素及其相互关系后,构造了它的计划优化模型.在此基础上,应用人工鱼群算法,对该模型进行研究,并在计算机上得以仿真实现.仿真结果表明,该算法不受问题规模的限制,对于解决大规模复杂的问题更显示出其优越性,为企业生产计划优化提供了有利工具.     

基于人工鱼群算法的高级综合生产计划优化研究

充分考虑了多产品多阶段多分销商企业生产过程各种影响因素及其相互关系后，构造了它的计划优化模型。在此基础上，应用人工鱼群算法，对该模型进行研究，并在计算机上得以仿真实现。仿真结果表明，该算法不受问题规模的限制，对于解决大规模复杂的问题更显示出其优越性，为企业生产计划优化提供了有利工具。     

一类含同工件流水线调度问题的优化研究

流水线调度问题是具有很强工程背景的典型NP完全问题,当其含有同工件时,批量和排序的相关性使得问题的求解更为复杂。文章有机结合模拟退火的概率突跳性和遗传算法的并行搜索结构,提出了处理一类含同工件流水线调度问题的混合优化策略。算法不仅能够动态缩小搜索空间以提高搜索效率,而且在保优策略的基础上利用重升温技术来增强克服陷入局部极小的能力,其有效性和快速性通过仿真得到了验证。     

基于信誉度的网格资源质量优化

网格由分布在地理上的各种各样的计算机资源组成,用户在调用资源时,只会根据资源提供商所制定的条件来选择适合自己的资源,但是资源质量的优劣只有在至少用完一次后才可以得出结论.现今许多基于经济模型的资源调度策略只是在提高资源提供商的效益及资源调用时的负载平衡方面加以研究,而网格资源的质量并无过多深入的分析.文中提出的基于信誉度的网格资源调度算法,在用户与资源之间插入一个信誉度计算器,用户每调用一次便会对此资源进行一次信誉度评估.实验证明此算法不仅提高了资源提供商获得的效益,并且使得网格资源的质量得到进一步的优化.     

Towards Efficient Execution of MPI Applications on the Grid: Porting and Optimization Issues

The message passing interface (MPI) is a standard used by many parallel scientific applications. It offers the advantage of a smoother migration path for porting applications from high performance computing systems to the Grid. In this paper Grid-enabled tools and libraries for developing MPI applications are presented. The first is MARMOT, a tool that checks the adherence of an application to the MPI standard. The second is PACX-MPI, an implementation of the MPI standard optimized for Grid environments. Besides the efficient development of the program, an optimal execution is of paramount importance for most scientific applications. We therefore discuss not only performance on the level of the MPI library, but also several application specific optimizations, e.g., for a sparse, parallel equation solver and an RNA folding code, like latency hiding, prefetching, caching and topology-aware algorithms.     

MDA:模式驱动体系的软件建模

模型驱动体系(Mode Driver Architecture,MDA)是一个新的软件开发框架.本文研究MDA的建模方法及模型在整个软件开发周期中起的核心作用,通过模型之间的变换实现的从PIM到J2EE平台上的PSM自动转换.     

数据库网格:基于网格的多数据库系统

近年来,数据网格技术的快速发展为广域分布环境下的海量数据共享提供了强有力的支持。数据库网格是指主要由各种数据库管理系统构成的数据网格环境,其技术难点是在网格环境下研究解决分布式查询、事务、多数据源视图等传统的数据库领域的问题。该文通过讨论数据网格、数据库网格和多数据库的概念和特点,并比较上述问题在多数据库和数据库网格环境下的异同,对数据库网格的特点进行了总结,对数据库网格技术的研究和发展方向进行了探讨和展望。     

基于数据仓库的维度建模优化

维度建模是个概念模型,星型架构可视为其逻辑模型,至于其物理实现的数据库可以是关系数据库、多维数据库或其他数据库.由于多维数据库发展的不成熟,目前,一般利用关系数据库使用星型模型来实现维度建模.     

仿真网格中服务的最优化调度机制研究

在网格中,如何为任务提供最优化资源服务是一个十分复杂的问题.本文以仿真网格为基础,建立一种服务的最优化调度的理论模型.根据这种理论模型,提出一种新的遗传模拟退火算法.从而形成一种仿真网格环境下服务的最优化调度机制.为了验证这种机制的可行性和有效性,开发一个基于此服务调度机制的仿真网格运行管理系统.通过这个系统的仿真实验表明:这种服务的最优化调度机制具有良好的可行性和有效性.     

基于神威·太湖之光的非结构网格众核优化技术

为解决高性能计算中的非结构网格离散访存问题,以神威·太湖之光国产超级计算机为平台,根据异构众核处理器SW26010的体系结构特点,提出一种基于排序思想的通用众核优化算法,以减少非结构网格计算中的随机访存。基于网格划分原理,在O(n)时间内对生成的稀疏矩阵非零元素进行并行重排序。采用一种内部映射方式对计算向量实现扩展或变换,将细粒度访存转化为无写冲突的粗粒度访存。对多个实际应用算例的通量计算进行众核优化,结果表明,相比主核上的串行算法,该算法能够获得平均10倍以上的加速效果。     

基于DoDAF的网格环境下虚拟组织认证系统体系结构建模

建模技术是体系结构研究、系统分析与设计所采用的最重要的方法之一.探讨DoDAF体系结构建模方法的特点和步骤,并以网格环境下虚拟组织认证系统为例展示其建模过程.所建模型说明,该方法能够满足系统准确建模的要求,可对系统研制过程的优化设计、变更影响分析和风险控制等方面提供技术支撑.