通用处理器的高带宽访存流水线研究

存储器访问速度的发展远远跟不上处理器运算速度的发展,日益严峻的访存速度问题严重制约了处理器速度的进一步发展.降低load-to-use延迟是提高处理器访存性能的关键,在其他条件确定的情况下,增加访存通路的带宽是降低load-to-use延迟的最有效途径,但增加带宽意味着增加访存通路的硬件逻辑复杂度,势必会增加访存通路的功耗.文中的工作立足于分析程序固有的访存特性,探索高带宽访存流水线的设计和优化空间,分析程序访存行为的规律性,并根据这些规律性给出高带宽访存流水线的低复杂度、低延迟、低功耗解决方案.文中的工作大大简化了高带宽访存流水线的设计,降低了关键路径的时延和功耗,被用于指导Godsonx处理器的访存设计.在处理器整体面积增加1.7%的情况下,将访存流水线的带宽提高了一倍,处理器的整体件能平均提高了8.6%.     

结合访存失效队列状态的预取策略

随着存储系统的访问速度与处理器的运算速度的差距越来越显著,访存性能已成为提高计算机系统性能的瓶颈.通过对指令Cache和数据Cache失效行为的分析,提出一种预取策略--结合访存失效队列状态的预取策略.该预取策略保持了指令和数据访问的次序,有利于预取流的提取.并将指令流和数据流的预取相分离,避免相互替换.在预取发起时机的选择上,不但考虑当前总线是否空闲,而且结合访存失效队列的状态,减小对处理器正常访存请求的影响.通过流过滤机制提高预取准确性,降低预取对访存带宽的需求.结果表明,采用结合访存失效队列状态的预取策略,处理器的平均访存延时减少30%,SPEC CPU2000程序的IPC值平均提高8.3%.     

面向实时流处理的多核多线程处理器访存队列

针对多核多线程处理器中乱序访存影响计算实时性的问题,在对典型访存队列进行研究的基础上提出了一种新的访存队列构建模型及其硬件结构.该模型采用窗口优化算法控制最差情况下的访存延迟,保证访存的实时性,同时又利用优化的乱序调度策略减少访存延迟.实验证明,该访存队列可控制最大访存延迟,与顺序访存相比,存储器具备更高的带宽,与传统的乱序访存相比较,可以充分满足计算的实时性需求,而存储器有效带宽基本不受影响,解决了多核多线程处理器承担实时流计算的基础难题.     

基于机器学习的软件模块访存压力优化仿真

采用当前方法对软件模块访存压力进行优化时,优化后的软件模块带宽较高、数据传输延时高,存在有效性差的问题。将机器学习应用在软件模块的访存压力优化过程中,提出基于机器学习的软件模块访存压力优化方法。计算链路的使用率,并将计算结果传送到每条流对应的发送端中,发送端根据接收到的信息对发送速率进行调整,实现拥塞控制。采用多目标规划方法,根据预算值和实际值之间存在的偏差,构建软件模块访存压力优化模型,通过二进制粒子群算法对软件模块访存压力优化模型进行求解,实现软件模块访存压力的优化。仿真结果表明,所提方法的带宽高、数据传输延时小,验证了基于机器学习的软件模块访存压力优化方法的有效性。     

MACT:高通量众核处理器离散访存请求批量处理机制

网络服务等新型高通量应用的迅速兴起给传统处理器设计带来了巨大的挑战.高通量众核处理器作为面向此类应用的新型处理器结构成为研究热点.然而,随着片上处理核数量的剧增,加之高通量应用的数据密集型特点,“存储墙”问题进一步加剧.通过分析高通量应用访存行为,发现此类应用存在着大量的细粒度访存,降低了访存带宽的有效利用率.基于此分析,在高通量处理器设计中通过添加访存请求收集表(memory access collection table,MACT)硬件机制,结合消息式内存机制,用于收集离散的访存请求并进行批量处理.MACT硬件机制的实现,提高了访存带宽的有效利用率,同时也提高了执行效率;并通过时间窗口机制,确保访存请求在最晚期限之前发送出去,保证任务的实时性.实验以典型高通量应用WordCount,TeraSort,Search为基准测试程序.添加MACT硬件机制后,访存数量减少约49％,访存带宽提高约24％,平均执行速度提高约89％.     

一种用于通用处理器结构优化的矩阵乘法性能模型

矩阵乘法作为高性能计算中的关键组成部分,是一种具有计算和访存密集特点的典型应用,因此优化矩阵乘法的性能对通用处理器是非常重要的.为了提高矩阵乘法的性能,本文提出了一种性能模型,用于预测通用处理器上矩阵乘法的执行时间.该模型反映了矩阵乘法执行时间与通用处理器的运算部件、访存带宽、寄存器个数等结构参数之间的关系,可以指导处理器结构的优化来平衡计算和访存能力、提高执行速度.基于该模型本文给出了在一个优化的通用处理器结构中,寄存器个数和访存带宽应满足的理论下界.本文在Godson-3B处理器平台上对该性能模型进行了验证,实验结果表明矩阵乘法执行时间的预测精确度达到95％以上.基于该模型,本文还提出了一种对Godson-3B结构进行优化的方法,使矩阵乘法的执行时间减少了50％左右.     

基于PCIe的多路传输系统的DMA控制器设计

为了避免PCIe传输过程中PIO写延时、主机与嵌入式处理系统交互次数过多等问题对于传输带宽的影响,设计了一种基于命令缓冲机制的直接存储访问（DMA）控制器以提高传输带宽利用率。采用FPGA端内部设置命令缓冲区的方式,使得DMA控制器可以缓存PC端的数据传输请求,FPGA根据自身需求动态地访问PC端存储空间,增强了传输灵活性;同时,提出一种动态拼接的DMA调度方法,通过合并相邻存储区访问请求的方式,进一步减少主机与硬件的交互次数和中断产生次数。系统传输速率测试实验中,DMA写最高速率可达1631 MB/s,DMA读最高速率可达1582 MB/s,带宽最大值可达PCIe总线理论带宽值的85.4%;与传统PIO方式的DMA传输方法相比,DMA读带宽提升58%,DMA写带宽提升36%。实验结果表明,本设计能够有效提升DMA传输效率,明显优于PIO方式。     

多线程向量处理器中向量数据存储结构的设计与实现

多线程和向量技术相结合是当前微处理器设计的一个重要趋势.提出一种多线程向量处理器中向量数据存储结构,利用多线程切换来隐藏访存延迟,并让向量数据直接访问二级cache来提高带宽.模拟实验表明在所提出的存储结构下,访存带宽随线程数线性增长,向量数据访问带宽明显高于标量数据访问带宽.     

存算解耦合的粗粒度可重构阵列访存结构设计

粗粒度可重构阵列架构兼具灵活性和高效性,但高计算吞吐量的特性也会给访存带来压力.在片下动态存储器带宽相对固定的情况下,设计一种存算解耦合的访存结构.将控制逻辑集成在轻量级的存储空间中,通过可配置的存储空间隔离访存和计算的循环迭代,从而掩盖内存延时,同时利用该结构进行串联和对齐操作,以适配不同的计算访存频率比并优化间接访...     

基于Cache锁和直接缓存访问的网络处理优化方法

通过分析计算机系统网络数据处理相关程序的访存行为、局部性特点和系统交互等问题,指出在高速网络环境下传统处理器网络子系统设计存在很大缺陷,并进一步提出一种基于软硬件协同设计的优化方案.该方案具体包括改进的直接缓存访问技术、关键程序的cache锁策略和相应系统互连结构及一致性协议等.实验表明,与传统方案相比,基于该方案的网络TCP传输带宽提高约48%,极限情况下UDP丢包率下降40%,传输延时降低超过10%.网络测试程序在与SPEC2000测试程序并发执行情况下,网络数据带宽提高约44%.此外还讨论了该优化方案与其他网络优化技术共同使用的基本原则和相应策略.     

Microwave tomography for breast cancer detection on Cell broadband engine processors

Microwave tomography (MT) is a safe screening modality that can be used for breast cancer detection. The technique uses the dielectric property contrasts between different breast tissues at microwave frequencies to determine the existence of abnormalities. Our proposed MT approach is an iterative process that involves two algorithms: Finite-Difference Time-Domain (FDTD) and Genetic Algorithm (GA). It is a compute intensive problem: (i) the number of iterations can be quite large to detect small tumors; (ii) many fine-grained computations and discretizations of the object under screening are required for accuracy. In our earlier work, we developed a parallel algorithm for microwave tomography on CPU-based homogeneous, multi-core, distributed memory machines. The performance improvement was limited due to communication and synchronization latencies inherent in the algorithm. In this paper, we exploit the parallelism of microwave tomography on the Cell BE processor. Since FDTD is a numerical technique with regular memory accesses, intensive floating point operations and SIMD type operations, the algorithm can be efficiently mapped on the Cell processor achieving significant performance. The initial implementation of FDTD on Cell BE with 8 SPEs is 2.9 times faster than an eight node shared memory machine and 1.45 times faster than an eight node distributed memory machine. In this work, we modify the FDTD algorithm by overlapping computations with communications during asynchronous DMA transfers. The modified algorithm also orchestrates the computations to fully use data between DMA transfers to increase the computation-to-communication ratio. We see 54% improvement on 8 SPEs (27.9% on 1 SPE) for the modified FDTD in comparison to our original FDTD algorithm on Cell BE. We further reduce the synchronization latency between GA and FDTD by using mechanisms such as double buffering. We also propose a performance prediction model based on DMA transfers, number of instructions and operations, the processor frequency and DMA bandwidth. We show that the execution time from our prediction model is comparable (within 0.5 s difference) with the execution time of the experimental results on one SPE.     

The Bottom-Up Implementation of One MILC Lattice QCD Application on the Cell Blade

We report the results of the bottom-up implementation of one MILC lattice quantum chromodynamics (QCD) application on the Cell Broadband Engine™ processor. In our implementation, we preserve MILC’s framework for scaling the application to run on a large number of compute nodes and accelerate computationally intensive kernels on the Cell’s synergistic processor elements. Speedups of 3.4 × for the 8 × 8 × 16 × 16 lattice and 5.7 × for the 16 × 16 × 16 × 16 lattice are obtained when comparing our implementation of the MILC application executed on a 3.2 GHz Cell processor to the standard MILC code executed on a quad-core 2.33 GHz Intel Xeon processor. We provide an empirical model to predict application performance for a given lattice size. We also show that performance of the compute-intensive part of the application on the Cell processor is limited by the bandwidth between main memory and the Cell’s synergistic processor elements, whereas performance of the application’s parallel execution framework is limited by the bandwidth between main memory and the Cell’s power processor element.     

Region-based parallelization of irregular reductions on explicitly managed memory hierarchies

Multicore architectures are evolving with the promise of extreme performance for the classes of applications that require high performance and large bandwidth of memory. Irregular reduction is one of important computation patterns for many complex scientific applications, and it typically requires high performance and large bandwidth of memory. In this article, we propose region-based parallelization techniques for irregular reductions on multicore architectures with explicitly managed memory hierarchies. Managing memory hierarchy in software requires a lot of programming efforts and tends to be error-prone. The difficulties are even worse for applications with irregular data access patterns. To relieve the burden of memory management from programmers, we develop abstractions, particularly targeted to irregular reduction, for structuring parallel tasks, mapping the parallel tasks to processing units and scheduling data transfers between the memory hierarchies. Our framework employs iteration reordering based on regions of data along with dynamic scheduling of parallel tasks. We experimentally evaluate the effectiveness of our techniques for irregular reduction kernels on the Cell processor embedded in a Sony PlayStation3. Experimental results show the speedups of 8 to 14 on the six available SPEs.     

基于虚通道的SDRAM访存调度器研究

随着半导体工艺水平的进步,CPU与存储器的速度差距越来越大,存储器带宽已成为计算机系统的关键资源。根据目前广泛使用的SDRAM存储器多体并行存储的结构特点,提出了一种基于虚通道的访存调度器和最小等待时间-读请求优先调度策略,避免了访存请求之间的数据相关性,加快了访存请求的调度,提高了存储器带宽的利用率。     

Limited bandwidth to affect processor design

This paper quantifies and compares the performance impacts of memory latencies and finite bandwidth. We show that the implementation of aggressive latency tolerance techniques aggravates stalls due to finite memory bandwidth, which actually become more significant than stalls resulting from uncongested memory latency alone. We expect that memory bandwidth limitations across the processor pins will drive significant architectural change. An execution-driven simulation measures the time that several SPEC95 benchmarks spend stalled for memory latency, limited-memory bandwidth and computing     

Bridging the processor-memory performance gap with 3D IC technology

Microprocessor performance has been improving at roughly 60% per year. Memory access times, however, have improved by less than 10% per year. The resulting gap between logic and memory performance has forced microprocessor designs toward complex and power-hungry architectures that support out-of-order and speculative execution. Moreover, processors have been designed with increasingly large cache hierarchies to hide main memory latency. This article examines how 3D IC technology can improve interactions between the processor and memory. Our work examines the performance of a single-core, single-threaded processor under representative work loads. We have shown that reducing memory latency by bringing main memory on chip gives us near-perfect performance. Three-dimensional IC technology can provide the much needed bandwidth without the cost, design complexity, and power issues associated with a large number of off-chip pins. The principal challenge remains the demonstration of a highly manufacturable 3D IC technology with high yield and low cost.     

Parallel exact inference on the Cell Broadband Engine processor

We present the design and implementation of a parallel exact inference algorithm on the Cell Broadband Engine (Cell BE) processor, a heterogeneous multicore architecture. Exact inference is a key problem in exploring probabilistic graphical models, where the computation complexity increases dramatically with the network structure and clique size. In this paper, we exploit parallelism in exact inference at multiple levels. We propose a rerooting method to minimize the critical path for exact inference, and an efficient scheduler to dynamically allocate SPEs. In addition, we explore potential table representation and layout to optimize DMA transfer between local store and main memory. We implemented the proposed method and conducted experiments on the Cell BE processor in the IBM QS20 Blade. We achieved speedup up to 10 × on the Cell, compared to state-of-the-art processors. The methodology proposed in this paper can be used for online scheduling of directed acyclic graph (DAG) structured computations.     

系统域网络适配器设计

作为系统域网络接入设备,适配器的功能和性能对整个机群系统的性能有着至关重要的影响.鉴于嵌入式技术的发展,提出了基于Intel IOP310 I/O处理器的曙光4000A超级计算机DCNet系统域网络适配器设计.适配器在原嵌入式系统基础上将本地内存总线扩展为用于网络互连的局部总线,并基于该总线设计实现了网络接口部件. DCNet适配器不但实现了与Myrinet,SCI和QsNet适配器相近的性能,而且证明了基于嵌入式系统和内存总线扩展网络接口方法实现高性能适配器是有效可行的.     

Towards optimized tensor code generation for deep learning on sunway many-core processor

The flourish of deep learning frameworks and hardware platforms has been demanding an efficient compiler that can shield the diversity in both software and hardware in order to provide application portability. Among the existing deep learning compilers, TVM is well known for its efficiency in code generation and optimization across diverse hardware devices. In the meanwhile, the Sunway many-core processor renders itself as a competitive candidate for its attractive computational power in both scientific computing and deep learning workloads. This paper combines the trends in these two directions. Specifically, we propose swTVM that extends the original TVM to support ahead-of-time compilation for architecture requiring cross-compilation such as Sunway. In addition, we leverage the architecture features during the compilation such as core group for massive parallelism, DMA for high bandwidth memory transfer and local device memory for data locality, in order to generate efficient codes for deep learning workloads on Sunway. The experiment results show that the codes generated by swTVM achieve 1.79\times improvement of inference latency on average compared to the state-of-the-art deep learning framework on Sunway, across eight representative benchmarks. This work is the first attempt from the compiler perspective to bridge the gap of deep learning and Sunway processor particularly with productivity and efficiency in mind. We believe this work will encourage more people to embrace the power of deep learning and Sunway many-core processor.     

High-speed data paths in host-based routers

In networking today, host workstations are increasingly being used as routers. Host based routers offer a number of advantages, but they suffer from inefficient support for high bandwidth interfaces. The authors' work has focused on the technology's major drawback its inefficiency in supporting high bandwidth interfaces. Their approach is to optimize packet processing by applying techniques that transfer packets directly among host interfaces, thus removing an extra data copy. This technique increases data throughput by 45 percent while reducing the host's CPU load. They found that peer DMA forwarding can increase host based router throughput by up to 45 percent, supporting bandwidths of 480 Mbps. Peer DMA host based forwarding requires network interface cards with substantial shared memory resources, because packet queues are stored on the interfaces themselves, rather than in host RAM. The queuing algorithm remains in the host CPU, supporting advanced queue management. Current systems have limited packet processing. A combination of streamlined forwarding algorithms and aggregate interrupt processing should further increase host based capability. Moving some of the IP processing out to the NIC coprocessor may enable this. It is also apparent that as processor speeds increase, the advantages of peer DMA will aid throughput for small packet sizes