一种海量数据流应用并行优化模型

计算进入了多核时代,处理器的发展不再由更快的主频带动,而是依靠增加片上的多个核心.但是,对于高性能应用来说,多核平台的并行处理由于缺少适合的并行程序开发工具还处于初始阶段,对应用的优化需要对底层线程结构的深入了解和正确使用.从海量数据流应用的特点出发,提出了三级流水多线程模型,它的线程同步机制没有竞争,并且实现了不同特征数据流的差别服务.然后,在遥感图像处理和骨干网网络入侵检测系设计中,应用了海量数据流应用模型,并在多个多核平台下对骨干网网络入侵检测系统进行了性能评价.对SPARC T1平台上的线程映射方法进行研究,测试了不同映射方法的性能,并归纳出应用在体系结构方面的特征;采用Sun SPARC T1架构8核32线程服务器和曙光x86架构8处理器16核服务器对系统吞吐率进行了测试,实验结果都表现了良好的可扩展性;使用真实骨干网络流量记录文件回放产生的模拟流量,对比测试了模型应用前后数据流的服务时间,改进系统的响应时间获得了显著的提高;针对系统连接数大、负载重和处理多样性的特点,采用基于探针流的采样算法准确测试了在精确预测IP网段策略下系统的服务质量,同时也测试了增加服务质量优化后系统的延迟开销,实验结果表明,系统在引入较少延迟下提高了数据流的服务质量.

A Parallel Optimization Model for Massive Data Stream Application

While computing is entering a new phase in which CPU improvements are driven by the addition of multiple cores on a single chip, rather than higher frequencies. Parallel processing on these systems is in a primitive stage, and requires the explicit use and knowledge of underlying thread architecture. Based on the features of massive data stream application, this paper proposes three-level pipelining programming model of multithreading system, which realizes the new synchronization mechanism with no contention of shared structures and is capable to provide differential service for data streams. Then the paper applies the new model to remote sensing information processing system and backbone network intrusion detection system, and evaluates the improved system on several multicore platforms. In performance analysis, the optimized effects of backbone network intrusion detection system are evaluated in several aspects of throughput scalability on both SPARC T1 and x86 platforms, the impacts of different multithreading mapping methods on throughput, and the comparison of response time and service quality before and after optimization. The experimental results show that the system throughput has good scalability on both platforms, the values of response time are greatly improved and the prioritized streams achieve better response time with the differential service mechanism.