一种基于Inter-warp异构性的缓存管理与内存调度机制

在GPU中，一个warp内的所有线程在锁步中执行相同的指令。某些线程的内存请求可以得到快速处理，而其余请求会经历较长时间。在最慢的请求完成之前，warp不能执行下一条指令，导致内存发散。对GPU中warp间的异构性进行了研究，实现并优化了一种基于inter warp异构性的缓存管理机制和内存调度策略，以减少内存发散和缓存排队延迟的负面影响。根据缓存命中率将warp分类，以驱动后面的3个组件：（1）基于warp类型的缓存旁路技术组件，使低缓存利用率的warp进入旁路，不访问L2缓存；（2）基于warp类型的缓存插入/提升策略组件，防止来自高缓存利用率warp的数据被过早清除；（3）基于warp类型的内存控制器组件，优先处理从高缓存利用率的warp接收到的请求，并优先处理来自相同warp的请求。基于warp间异构性的缓存管理和内存调度机制在8种不同的GPGPU应用中，与基准GPU相比，平均加速18.0％。

Cache management and memory scheduling based on inter-warp heterogeneity

All threads within a warp execute the same instruction in the lockstep in a GPU. Memory requests from some threads are served early while requests from some other threads have to experience long time latency. Warp cannot execute the next instruction before the last request is served, which can cause memory divergence. We study the inter-warp heterogeneity in GPU, implement and optimize a cache management mechanism and a memory scheduling policy based on inter warp heterogeneity, which can reduce the negative impact of memory divergence and cache queuing latency. Warps are classified according to the hit rate of L2 cache to drive the following three components: (1) A warp-type based cache bypassing mechanism to bypass the L2 cache for warps with low cache utilization; (2) A warp type based cache insert/improvement policy to prevent the data from warps with high cache utilization being cleared prematurely; and (3) A warp-type based memory scheduler to prioritize requests received from warps with high cache utilization and the requests from the same warp. Compared with the baseline GPU, the cache management mechanism and the memory scheduling policy based on inter-warp heterogeneity can speed up 8 different GPGPU applications by 18.0% on average.