首页 | 本学科首页   官方微博 | 高级检索  
相似文献
 共查询到10条相似文献,搜索用时 133 毫秒
1.
随着NOW在科学研究中白益广泛的应用,如何为NOW上的科学计算提供高性能的输入输出成为我们面临的一个新课题。作者根据NOW的特点,设计并实现了一个具有NOW特色的基于CollectiveI/Q的并行I/O系统,吸取了DDIO与two-phaseI/O的优点,从而有效地解决了高带宽和低延迟问题。初步的系统吞吐量测试显示了良好的性能。  相似文献   

2.
Scientific applications often need to write out large arrays and associated metadata periodically for visualization or restart purposes. In this paper, we present active buffering, a high-level transparent buffering scheme for collective I/O, in which processors actively organize their idle memory into a hierarchy of buffers for periodic output data. It utilizes idle memory on the processors, yet makes no assumption regarding runtime memory availability. Active buffering can perform background I/O while the computation is going on, is extensible to remote I/O for more efficient data migration, and can be implemented in a portable style in today's parallel I/O libraries. It can also mask performance problems of scientific data formats used by many scientists. Performance experiments with both synthetic benchmarks and real simulation codes on multiple platforms show that active buffering can greatly reduce the visible I/O cost from the application's point of view.  相似文献   

3.
一个基于NOW的并行I/O系统   总被引:1,自引:0,他引:1  
李冀  陈晓林  陆桑璐  陈贵海  谢立 《软件学报》2001,12(11):1654-1659
随着NOW(networksofworkstations)在科学研究中的应用日益广泛,如何为NOW上的科学计算提供高性能的输入、输出成为人们所面临的一个新课题.根据NOW的特点,设计并实现了一个具有NOW特色的采用CollectiveI/O技术的并行I/O系统CION(collectiveI/Oonnowsystem).CION吸取了DDIO(disk-directedI/O)与two-phaseI/O的优点,同时采用了数据筛选等一系列优化技术.初步的测试已经显示了良好的系统性能.  相似文献   

4.
曾碧卿  陈志刚 《计算机工程》2006,32(19):38-39,4
在大型多维数据集合处理中,对多维数据集合的拆分及其在磁盘上的存储分配是重要的研究课题。由于磁盘的机械运动已形成了数据I/O时的速度瓶颈,因此通过采用并行I/O技术,将多维数据进行有效的拆分,并在多个磁盘间进行分布存储是克服瓶颈的有效办法。基于此,论文中提出了一种多维数据的循环拆分方法,它是对二维数据集合循环拆分分配方法的扩展,性能比较与分析表明了新算法的有 效性。  相似文献   

5.
A critical performance issue for a number of scientific and engineering applications is the efficient transfer of data to secondary storage. Languages such as High Performance Fortran (HPF) have been introduced to allow programming distributed-memory systems at a relatively high level of abstraction. However, the present version of HPF does not provide appropriate constructs for controlling the parallel I/O capabilities of these systems. In this paper, constructs to specify parallel I/O operations on multidimensional arrays in the context of HPF are proposed. The paper also presents implementation concepts that are based on the HPF compiler VFC and the parallel I/O run-time system Panda. Experimental performance results are discussed in the context of financial management and traffic simulation applications.  相似文献   

6.
为了缓解I/O瓶颈问题,可以从应用程序、可扩展算法、编译器和语言、运行时库、操作系统和体系结构六方面展开研究。其中,I/O体系结构是所有技术途径的关键支撑。当前并行I/O性能分析缺乏科学的理论模型为I/O体系结构设计提供理论依据。本文针对并行计算机系统的可扩展性问题,研究了I/O负载对并行计算机系统可扩展性的影响,建立了I/O受限的并行加速比性能模型,对目前大规模并行计算机系统中三种常用I/O体系结构的可扩展性进行了分析;以此为理论依据,提出了一种面向高性能计算的可扩展并行I/O系统结构。同时,还提出了几种有效降低I/O操作服务时间的策略,从而达到增强系统可扩展性的目的,为后续研究奠定了基础。  相似文献   

7.
采样数据的并行I/O制约一些并行应用的运行效率。设计、实现了采样数据的聚集并行I/O方法。该方法在客户端部署采样数据缓存,然后合并数据到输出进程,再存储到文件。为了保障并行程序长时间运行过程中采样数据的存储一致性,该方法在JASMIN框架中监测应用程序的运行状态,当并行程序发生负载平衡或者重启动时刷新或者恢复数据。I/O过程中,进一步使用HDF5的分块I/O提高列存储数据的读写效率。测试表明,新方法不仅具有较好的可扩展性,还能在具有负载平衡与重启动等复杂功能的并行应用中提高采样数据的并行 I/O 效率7.5倍以上。  相似文献   

8.
杨启亮  邢建春  王平 《计算机工程》2005,31(14):231-232,F003
给出了基于DCOM的DLLI/O驱动的封装方案,使客户程序具备了跨进程及计算机边界从而远程访问I/O设备的能力。文章在讨论封装方案的基础上,重点研究了客户程序与I/O组件服务器间的指针、数组、字符串及字符串数组等重要数据类型的数据传递技术。  相似文献   

9.
10.
《Parallel Computing》2014,40(10):697-709
In order to run tasks in a parallel and load-balanced fashion, existing scientific parallel applications such as mpiBLAST introduce a data-initializing stage to move database fragments from shared storage to local cluster nodes. Unfortunately, with the exponentially increasing size of sequence databases in today’s big data era, such an approach is inefficient.In this paper, we develop a scalable data access framework to solve the data movement problem for scientific applications that are dominated by “read” operation for data analysis. SDAFT employs a distributed file system (DFS) to provide scalable data access for parallel sequence searches. SDAFT consists of two interlocked components: (1) a data centric load-balanced scheduler (DC-scheduler) to enforce data-process locality and (2) a translation layer to translate conventional parallel I/O operations into HDFS I/O. By experimenting our SDAFT prototype system with real-world database and queries at a wide variety of computing platforms, we found that SDAFT can reduce I/O cost by a factor of 4–10 and double the overall execution performance as compared with existing schemes.  相似文献   

设为首页 | 免责声明 | 关于勤云 | 加入收藏

Copyright©北京勤云科技发展有限公司  京ICP备09084417号