Advanced hashing schemes for packet forwarding using set associative memory architectures

Building a high performance IP packet forwarding (PF) engine remains a challenge due to increasingly stringent throughput requirements and the growing size of IP forwarding tables. The router has to match the incoming packet’s IP address against all entries in the forwarding table. The matching process has to be done at increasingly higher wire speed; hence, scalability and low power consumption are critical for PF engines.Various hash table based schemes have been considered for use in PF engines. Set associative memory can be used for hardware implementations of hash tables with the property that each bucket of a hash table can be searched in a single memory cycle. However, the classic hashing downsides, such as collisions and worst case memory access time have to be dealt with. While open addressing hash tables, in general, provide good average case search performance, their memory utilization and worst case performance can degrade quickly due to collisions (that lead to bucket overflows).The two standard solutions to the overflow problem are either to use predefined probing (e.g., linear or quadratic probing) or to use multiple hash functions. This work presents two new simple hash schemes that extend both aforementioned solutions to tackle the overflow problem efficiently. The first scheme is a hash probing scheme that is called Content-based HAsh Probing (CHAP). As the name suggests, CHAP, based on the content of the hash table, avoids the classical side effects of predefined hash probing methods (i.e., primary and secondary clustering phenomena) and at the same time reduces the overflow. The second scheme, called Progressive Hashing (PH), is a general multiple hash scheme that reduces the overflow as well. The basic idea of PH is to split the prefixes into groups where each group is assigned one hash function, then reuse some hash functions in a progressive fashion to reduce the overflow. Both schemes are amenable to high-performance hardware implementations with low overflow and constant worst-case memory access time. We show by experimenting with real IP lookup tables and synthetic traces that both schemes outperform other existing hashing schemes.     

一种适应GPU的混合访问缓存索引框架

散列表(hashtable)作为一类根据关键码值(keyvalue)提供高效数据访问的数据索引结构,其广泛应用于各类计算机应用中,尤其是在对性能要求极高的系统软件、数据库以及高性能计算领域.在网络、云计算和物联网服务方面,以散列表为核心结构已经成为缓存系统的重要系统组件.然而,随着大规模数据量的大幅度增加,以多核CPU为核心设计散列表结构的系统已经逐渐出现性能瓶颈,亟需进一步改进散列表的高性能和可扩展性.随着通用图形处理器(graphic processing unit,简称GPU)的日益普及以及硬件计算能力和并发性能的大幅度提升,各类以并行计算为核心的系统软件任务在GPU上进行了优化设计并得到可观的性能提升.由于存在稀疏性和随机性,采用现有散列表的并行结构直接在GPU上应用势必会带来高频次的内存访问和频繁的总线数据传输,影响了散列表在GPU上的性能发挥.重点分析了缓存系统中散列表索引的内存访问、命中率与索引开销,提出并设计了一种适应GPU的混合访问缓存索引框架CCHT(cache cuckoo hash table),提供了两种适应不同命中率和索引开销要求的缓存策略,允许写入与查询操...     

Hybrid Access Cache Indexing Framework Adapted to GPU

Hash tables, as a type of data indexing structure that provides efficient data access based on key values, are widely used in various computer applications, especially in system software, databases, and high-performance computing field that requires extremely high performance. In network, cloud computing and IoT services, hash tables have become the core system components of cache systems. However, with the large-scale increase in the amount of large-scale data, performance bottlenecks have gradually emerged in systems designed with a multi-core CPU as the core of the hash table structure. There is an urgent need to further improve the high performance and scalability of the hash tables. With the increasing popularity of general-purpose Graphic Processing Units (GPUs) and the substantial improvement of hardware computing capabilities and concurrency performance, various types of system software tasks with parallel computing as the core have been optimized on the GPU and have achieved considerable performance promotion. Due to the sparseness and randomness, using the existing parallel structure of the hash tables directly on the GPUs will inevitably bring high-frequency memory access and frequent bus data transmission, which affects the performance of the hash tables on the GPUs. This study focuses on the analysis of memory access, hit ratio, and index overhead of hash table indexes in the cache system. A hybrid access cache indexing framework CCHT (Cache Cuckoo Hash Table) adapted to GPU is proposed and provided. The cache strategy suitable to different requirements of hit ratios and index overheads allows concurrent execution of write and query operations, maximizing the use of the computing performance and concurrency characteristics of GPU hardware, reducing memory access and bus transferring overhead. Through GPU hardware implementation and experimental verification, CCHT has better performance than other cache indexing hash tables while ensuring cache hit ratios.     

Efficient combination of index tables and hashing

It is shown that for data stored on direct access devices access time can be reduced without increasing storage demand, through a single level index table which itself is accessible by hashing. If the complete index table can be stored in main memory this method is always superior to direct hashing and to sequentially organized index tables. If the index table is stored on disk it always yields smaller access time than multi-level index tables and—depending on the size of the index table and on the number of records per track—it is comparable or better than hashing the data directly. Expressions are given to determine in this case which method is more efficient.     

Fast and deterministic hash table lookup using discriminative bloom filters

Hash tables are widely used in network applications, as they can achieve O(1) query, insert, and delete operations at moderate loads. However, at high loads, collisions are prevalent in the table, which increases the access time and induces non-deterministic performance. Slow rates and non-determinism can considerably hurt the performance and scalability of hash tables in the multi-threaded parallel systems such as ASIC/FPGA and multi-core. So it is critical to keep the hash operations faster and more deterministic.This paper presents a novel fast collision-free hashing scheme using Discriminative Bloom Filters (DBFs) to achieve fast and deterministic hash table lookup. DBF is a compact summary stored in on-chip memory. It is composed of an array of parallel Bloom filters organized by the discriminator. Each element lookup performs parallel membership checks on the on-chip DBF to produce a possible discriminator value. Then, the element plus the discriminator value is hashed to a possible bucket in an off-chip hash table for validating the match. This DBF-based scheme requires one off-chip memory access per lookup as well as less off-chip memory usage. Experiments show that our scheme achieves up to 8.5-fold reduction in the number of off-chip memory accesses per lookup than previous schemes.     

移动计算环境中的自适应混合广播

提出一种自适应混合广播算法。在周期广播数据分配时采用多信道非均匀分配,使高访问率的数据获得高广播频率。在确定周期广播数据和联机请求数据个数时考虑联机请求信道响应时间和访问率之间的关系,从而在少量比较次数后获得数据最佳分割点。实验结果表明,该算法可以根据系统负载和用户访问模式的变化动态调节信道和数据的分配,性能优于纯广播和纯基于请求的广播,访问时间少于现有的混合数据广播方式。     

面向MIC协处理器的OLAP外键连接算法

众核架构协处理器Xeon Phi成为新兴的主流高性能计算平台.对于数据库应用而言,内存分析处理是一种计算密集型负载,其主要的性能取决于大事实表与维表之间的内存外键连接性能.本文关注于一种相对于缓存相关的分区哈希连接算法和缓存不相关的无分区哈希连接算法的缓存友好型外键连接算法,以适应Xeon Phi协处理器较小的LLC和高并发线程的特点.通过挖掘OLAP模式中的代理键特征,基于键值匹配的哈希探测操作可以进一步简化为事实表与维表之间基于主-外键参照完整性约束的代理键参照访问,因此复杂的哈希表和CPU代价较高的哈希探测操作可以简化为通过映射外键值为代理键向量内存偏移地址的方法对代理向量直接访问.基于代理向量参照访问的外键连接算法能够简单并高效地应用于Xeon Phi协处理器平台,通过更多的核心和高并发线程来掩盖内存访问延迟.实验中对传统的哈希连接算法（无分区哈希连接算法和基数分区哈希连接算法）和基于代理向量参照技术的外键连接算法在Xeon E5-2650 v3 10核处理器平台和Xeon Phi 5110P 60核协处理器平台进行性能测试和比较,实验结果给出了主流的内存外键连接算法在不同数据集和不同平台上全面的性能特征.     

Implementation and modeling of two-phase locking concurrency control—a performance study

Two-phase locking (2PL) is the concurrency control mechanism that is used in most commercial database systems. In 2PL, for a transaction to access a data item, it has to hold the appropriate lock (read or write) on the data item by issuing a lock request. While the way transactions set their lock requests and the way the requests are granted would certainly affect a system's performance, such aspects have not received much attention in the literature. In this paper, a general transaction-processing model is proposed. In this model, a transaction is comprised of a number of stages, and in each stage the transaction can request to lock one or more data items. Methods for granting transaction requests and scheduling policies for granting blocked transactions are also proposed. A comprehensive simulation model is developed from which the performance of 2PL with our proposals is evaluated. Results indicate that performance models in which transactions request locks on an item-by-item basis and use first-come-first-served (FCFS) scheduling in granting blocked transactions underestimate the performance of 2PL. The performance of 2PL can be greatly improved if locks are requested in stages as dictated by the application. A scheduling policy that uses global information and/or schedules blocked transactions dynamically shows a better performance than the default FCFS.     

基于完美彩虹表的检查点算法改进研究

本文对完美彩虹表下的检查点算法进行了研究和改进.时间存储折中攻击是由Hellman于1980年提出的一种适用于分组密码和哈希函数的算法.该算法具有可以用空间复杂度来换取时间复杂度的特点,然而由于链之间的碰撞,算法具有较高的误报率.其一个变种,Oechslin于2003年提出的彩虹表算法可以大幅减少碰撞的数量,从而提升效率.2005年,Avoine等人提出了另一种名为"检查点"的改进,该算法从另一个角度,即降低误报的影响来提升效率.然而,检查点的设置问题(数量和位置)仍未得到完全的解答.在本文中,我们对检查点算法在基于完美彩虹表的条件下进行研究,对检查点的设置进行理论分析,推导出最佳位置的计算式,并构造实验来检验最优选择的结果.在空间复杂度相当的条件下,相较于没有设置检查点的彩虹表,攻击时间可以减少10%到30%.     

用双链哈希表提高基于网络处理器的防火墙性能

基于网络处理器的防火墙中大量的内存访问会影响对高速网络流的处理速度。哈希表是防火墙中重要的数据结构,用拉链法解决冲突时一次查表的平均内存访问次数与相应拉链的长度成正比。把一条拉链划分成两条可以缩短链的长度,减少总的内存访问次数,从而提高系统性能。介绍了用两条链处理哈希表冲突问题的方法,分析了它对性能的影响,并以网络处理器IXP2400为例给出了具体设计和实现。     

一种基于硬件的大规模哈希流表设计与实现

基于流的报文处理是防火墙、入侵检测等网络安全应用的重要组成功能,其中流表是流处理技术的关键数据结构,流表的规模及访问性能直接影响到流处理的能力和速度。着眼于高速网络下大规模流表的硬件实现,设计了一种基于硬件的千万级哈希流表查找架构,并在FPGA平台上进行了实现和测试。该方案在保证访存效率的同时很好地解决了冲突的难题,利用有限的存储资源,满足了高达4 900万项的流表查找需求,测试能够实现92Mdesc/s的表查找速度,支持约220Gbps高速以太网的处理能力。     

基于期限的多请求数据检索问题研究

给定一个包含多条信道的集合以及一个包含多个请求的集合,其中每一个请求包含多个请求数据项并且希望在一定期限内下载到,基于期限的多请求数据检索问题指当客户配有多条天线时寻找一个在期限内下载多个请求的数据检索序列,使得所有天线的最大访问延迟最小化。大多数现有数据检索方法关注于单个请求或者单条天线,很少研究当客户配有多条天线时多请求的数据检索问题,尤其是每一个请求的检索有时间约束。基于此,本文提出一种多请求的数据检索算法,以调度合适地天线检索这些请求并找到关于这些请求的检索序列,从而平衡在各天线上的访问延迟。针对单请求的数据检索,本文采用最大团思想寻找下载该请求中所有请求数据项的访问模式,使得检索该请求的访问延迟以及期限丢失率最小化。     

A periodic tabular policy for scheduling of a single stage production-inventory system

In this paper, we consider scheduling of a multi-item single stage production-inventory system in the presence of uncertainty regarding demand patterns, production times and switchover times. For a given specification of base-stock levels of individual items and under (S − 1, S) requests for replenishment policy, a mathematical program to minimize long-run average system wide costs is formulated. We derive approximations for the first two moments of demand over lead time using residual service analysis of vacation queue models. Subsequently, we develop an approximate convex program for the original cost model and determine optimal production frequencies for individual types. Based on these relative frequencies, we determine a table size and devise an efficient heuristic to construct a tabular sequence in which individual items appear according to their respective absolute frequencies and items are positioned such that variance of their inter-visit times is minimized. A numerical study that demonstrates effectiveness of the proposed policy against cyclic policies is given.     

一种动态哈希树网络流量跟踪算法

本文提出一种基于动态哈希树的流量跟踪算法DHT（Dynamic Hash Tree）。该算法利用网络会话的长时稳定性,动态搭建一个由多哈希表组成的树,以提高实际网络环境中会话识别和流量跟踪的速度。试验结果表明该算法的效率明显优于目前流行的哈希链表算法,能够满足骨干网络的实时监测要求。     

Using Parallel Bloom Filters for Multiattribute Representation on Network Services

One widely used mechanism for representing membership of a set of items is the simple space-efficient randomized data structure known as Bloom filters. Yet, Bloom filters are not entirely suitable for many new network applications that support network services like the representation and querying of items that have multiple attributes as opposed to a single attribute. In this paper, we present an approach to the accurate and efficient representation and querying of multiattribute items using Bloom filters. The approach proposes three variant structures of Bloom filters: Parallel Bloom Filter (referred as PBF) structure, PBF with a hash table (PBF-HT), and PBF with a Bloom filter (PBF-BF). PBF stores multiple attributes of an item in parallel Bloom filters. The auxiliary HT and BF provide functions to capture the inherent dependency of all attributes of an item. Compared to standard Bloom filters to represent items with multiple attributes, the proposed PBF facilitates much faster query service and both PBF-HT and PBF-BF structures achieve much lower false positive probability with a result to save storage space. Simulation and experimental results demonstrate that the new space-efficient Bloom filter structures can efficiently and accurately represent multiattribute items and quickly respond queries at the cost of a relatively small false positive probability.     

面向视觉搜索的空间局部敏感哈希方法

目的 视觉检索需要准确、高效地从大型图像或者视频数据集中检索出最相关的视觉内容,但是由于数据集中图像数据量大、特征维度高的特点,现有方法很难同时保证快速的检索速度和较好的检索效果。方法 对于面向图像视频数据的高维数据视觉检索任务,提出加权语义局部敏感哈希算法（weighted semantic locality-sensitive hashing, WSLSH）。该算法利用两层视觉词典对参考特征空间进行二次空间划分,在每个子空间里使用加权语义局部敏感哈希对特征进行精确索引。其次,设计动态变长哈希码,在保证检索性能的基础上减少哈希表数量。此外,针对局部敏感哈希（locality sensitive hashing, LSH）的随机不稳定性,在LSH函数中加入反映参考特征空间语义的统计性数据,设计了一个简单投影语义哈希函数以确保算法检索性能的稳定性。结果 在Holidays、Oxford5k和DataSetB数据集上的实验表明,WSLSH在DataSetB上取得最短平均检索时间0.034 25 s;在编码长度为64位的情况下,WSLSH算法在3个数据集上的平均精确度均值（mean average precision,mAP）分别提高了1.2%32.6%、1.7%19.1%和2.6%28.6%,与几种较新的无监督哈希方法相比有一定的优势。结论 通过进行二次空间划分、对参考特征的哈希索引次数进行加权、动态使用变长哈希码以及提出简单投影语义哈希函数来对LSH算法进行改进。由此提出的加权语义局部敏感哈希（WSLSH）算法相比现有工作有更快的检索速度,同时,在长编码的情况下,取得了更为优异的性能。     

Using large tables on small display devices

The next evolutionary step in wireless Internet information management is to provide support for tasks, which may be collaborative and may include multiple target devices, from desktop to handheld. This means that the information architecture supports the processes of the task, recognizes group interaction, and lets users migrate seamlessly among internet-compatible devices without losing the thread of the session. If users are free to migrate amongst devices during the course of a session then intelligent transformation of data is required to exploit the screen size and input characteristics of the target appliance with minimal loss of task effectiveness.In this paper we first review general characteristics related to the performance of users on small screens and then examine the navigation of full tables on small screens for users in multi-device scenarios. We examine the methodologies available for access to full tables in environments where the full table cannot be viewed in its entirety. In particular, we examine the situation where users are collaborating across platform and referring to the same table of data. We ask three basic questions: Does screen size affect the performance of table lookup tasks? Does a search function improve performance of table lookup based tasks on reduced screen sizes? Does including context information improve the performance of table lookup based tasks on reduced screen sizes? The answers to these questions are important as individual and intuitive responses are used by the designers of small screen interfaces for use with large tables of data. We report on the results of a user study that examines factors that may affect the use of large tables on small display devices. The use of large tables on small devices in their native state becomes important in at least two circumstances. First, when collaboration involves two or more users sharing a view of data when the individual screen sizes are different. Second, when the exact table structure replication may be critical as a user moves quickly from a larger to a smaller screen or back again mid-task. Performance is measured by both effectiveness, correctness of result, and efficiency, effort to reach a result.     

MS SQL Server数据库锁技术研究

数据库涣是保证网络数据库中的数据安全的重要手段。文章通过对Microsoft SQL Server的锁管理机制的分析,讨论了锁的多粒度性和镇的多模式性,以及由此引出的涣升级和锁的兼容性,还有减少死锁可能性的几种方法,之后给出了手工加锁的方法,以保证应用程序的正确运行及保持数据的一致性和正确性。     

基于HEVC屏幕图像编码的哈希表的优化算法

仿2维匹配算法对屏幕图像中的非连续色调区域有很好的压缩性能,但该算法中哈希表的空间开销较大,不利于硬件实现。为了减小哈希表的空间,通过对原算法优化提出了一种3字节计算哈希值方法,将源数据看作是一个由以YUV三元组为元素组成的数据集合,然后以YUV三元组为单位计算哈希值,这样不但减少了哈希值的计算量,而且使哈希表的存储空间得到很大的节省。实验结果表明,3字节计算哈希值方法使哈希表的存储空间减少为原算法的1/3,所测试屏幕图像的BD-rate性能也有所提高。     

面向持久化键值数据库的自适应热点感知哈希索引

针对现有键值数据库存储系统缺乏热点意识,导致系统在高度倾斜的工作负载下性能较差且不可靠,提出了一种自适应热点感知哈希索引模型,该模型基于key值摘要信息实现了一个高性能哈希表。首先,利用key的摘要信息代替key值,压缩key的存储空间,优化哈希表中桶的数据结构;其次,利用CPU的数据级并行技术以及CPU cache line,对哈希表的探查操作进行优化;最后,为解决摘要信息导致key值无法精准比较,需要额外磁盘I/O的问题,设计了一种自适应key值调度算法,该算法根据当前可用内存大小、哈希索引负载以及访问热点情况动态地调整key值的存储位置。在YCSB仿真数据集上进行了实验,实验表明,相较于最先进的哈希表,自适应热点感知哈希索引在相同内存使用率的情况下,将速度提升至1.2倍。