适用于高速检索的完美Hash函数

软件实现的Hash函数在当前检索领域应用非常广泛,但是由于处理速度不高,很难满足骨干网以及服务器海量数据的高速实时查找要求.硬件Hash函数处理速度快,但普遍存在设计电路复杂、存储空间利用率不高以及无法支持数据集动态更新等问题.基于位提取(Bit-extraction)算法,利用位选择(Bit-Selection)操作与位逻辑运算在FPGA上仿真实现一种Hash函数,可生成负载因子(Load factor)接近于1的近似最小完美Hash表.仿真结果表明,该Hash函数中每个24 bits长度Key的存储空间只要2.8-5.6 bits,系统时钟频率可以达到300MHz左右(吞吐率超过14Gbps).可以应用于IP地址查找、数据包分类、字符串匹配以及入侵检测等需要实时高速表查找的场景.     

Large-scale SRAM-based IP lookup architectures using compact trie search structures

SRAM-based pipelined architectures for high-speed IP lookup using Field Programmable Gate Arrays (FPGAs) has recently attracted a great deal of attention from researchers. Due to the limited amount of on-chip memory and the number of I/O pins of FPGAs, compact data structures providing high memory efficiency are in great demand.     

一种基于硬件的大规模哈希流表设计与实现

基于流的报文处理是防火墙、入侵检测等网络安全应用的重要组成功能,其中流表是流处理技术的关键数据结构,流表的规模及访问性能直接影响到流处理的能力和速度。着眼于高速网络下大规模流表的硬件实现,设计了一种基于硬件的千万级哈希流表查找架构,并在FPGA平台上进行了实现和测试。该方案在保证访存效率的同时很好地解决了冲突的难题,利用有限的存储资源,满足了高达4 900万项的流表查找需求,测试能够实现92Mdesc/s的表查找速度,支持约220Gbps高速以太网的处理能力。     

Advanced hashing schemes for packet forwarding using set associative memory architectures

Building a high performance IP packet forwarding (PF) engine remains a challenge due to increasingly stringent throughput requirements and the growing size of IP forwarding tables. The router has to match the incoming packet’s IP address against all entries in the forwarding table. The matching process has to be done at increasingly higher wire speed; hence, scalability and low power consumption are critical for PF engines.Various hash table based schemes have been considered for use in PF engines. Set associative memory can be used for hardware implementations of hash tables with the property that each bucket of a hash table can be searched in a single memory cycle. However, the classic hashing downsides, such as collisions and worst case memory access time have to be dealt with. While open addressing hash tables, in general, provide good average case search performance, their memory utilization and worst case performance can degrade quickly due to collisions (that lead to bucket overflows).The two standard solutions to the overflow problem are either to use predefined probing (e.g., linear or quadratic probing) or to use multiple hash functions. This work presents two new simple hash schemes that extend both aforementioned solutions to tackle the overflow problem efficiently. The first scheme is a hash probing scheme that is called Content-based HAsh Probing (CHAP). As the name suggests, CHAP, based on the content of the hash table, avoids the classical side effects of predefined hash probing methods (i.e., primary and secondary clustering phenomena) and at the same time reduces the overflow. The second scheme, called Progressive Hashing (PH), is a general multiple hash scheme that reduces the overflow as well. The basic idea of PH is to split the prefixes into groups where each group is assigned one hash function, then reuse some hash functions in a progressive fashion to reduce the overflow. Both schemes are amenable to high-performance hardware implementations with low overflow and constant worst-case memory access time. We show by experimenting with real IP lookup tables and synthetic traces that both schemes outperform other existing hashing schemes.     

散列索引多分支Trie树快速路由查找算法

路由器的主要任务是转发IP分组,实现高速分组转发的关键是快速的路由查找算法。我们针对IPv4地址,首先建立前缀长度为8、16和24的3张hash表,在此基础上,再分别针对不同长度的前缀建立最多只涉及其余8比特的多分支Trie树。在这种结构中进行IP路由查找,其存储器访问次数最多为7次,而且还具有易于更新、易于扩展等特点。     

基于快速搜索树的路由查表算法*

根据路由表中前缀的分布特点,将路由集合分割成几个子集,然后分别针对每个子集建立搜索树来实现路由查表。借助哈希压缩索引表使搜索树的深度降低到3,加快了搜索树的查找速度。而Bloom Filters的应用,使几乎平均一次搜索树的查找就可以完成一次路由查表。该算法可以满足OC768链路的处理速度要求,支持达106数量级的路由表项,适于硬件流水线方式实现,具有很高的实用价值。这种方法用到IPv6同样可以收到很好的效果。     

Reconfigurable memory architecture for scalable IP forwarding engines

In this paper, a reconfigurable memory architecture and lookup technique for IP packet forwarding engines is presented. This is achieved by partitioning the forwarding table into smaller partial lookup tables for each output port and allowing a forwarding engine to process them in parallel. This effectively reduces the complexity of finding the ‘longest prefix match’ problem to the ‘first prefix match’ problem. Our method is a flexible technique that significantly elevates the scalability of the next generation network processors and other packet processing devices. Such scalability facilitates migration to IPv6 and benefits network equipments especially in terms of growing routing table size, traffic, frequency of route updates and bandwidth requirement.     

Hardware implementation analysis of SHA-256 and SHA-512 algorithms on FPGAs

Hash functions are common and important cryptographic primitives, which are very critical for data integrity assurance and data origin authentication security services. Field programmable gate arrays (FPGAs) being reconfigurable, flexible and physically secure are a natural choice for implementation of hash functions in a broad range of applications with different area-performance requirements. In this paper, we explore alternative architectures for the implementation of hash algorithms of the secure hash standards SHA-256 and SHA-512 on FPGAs and study their area-performance trade-offs. As several 64-bit adders are needed in SHA-512 hash value computation, new architectures proposed in this paper implement modulo-64 addition as modulo-32, modulo-16 and modulo-8 additions with a view to reduce the chip area. Hash function SHA-512 is implemented in different FPGA families of ALTERA to compare their performance metrics such as area, memory, latency, clocking frequency and throughput to guide a designer to select the most suitable FPGA for an application. In addition, a common architecture is designed for implementing SHA-256 and SHA-512 algorithms.     

Fast and deterministic hash table lookup using discriminative bloom filters

Hash tables are widely used in network applications, as they can achieve O(1) query, insert, and delete operations at moderate loads. However, at high loads, collisions are prevalent in the table, which increases the access time and induces non-deterministic performance. Slow rates and non-determinism can considerably hurt the performance and scalability of hash tables in the multi-threaded parallel systems such as ASIC/FPGA and multi-core. So it is critical to keep the hash operations faster and more deterministic.This paper presents a novel fast collision-free hashing scheme using Discriminative Bloom Filters (DBFs) to achieve fast and deterministic hash table lookup. DBF is a compact summary stored in on-chip memory. It is composed of an array of parallel Bloom filters organized by the discriminator. Each element lookup performs parallel membership checks on the on-chip DBF to produce a possible discriminator value. Then, the element plus the discriminator value is hashed to a possible bucket in an off-chip hash table for validating the match. This DBF-based scheme requires one off-chip memory access per lookup as well as less off-chip memory usage. Experiments show that our scheme achieves up to 8.5-fold reduction in the number of off-chip memory accesses per lookup than previous schemes.     

面向IP流测量的哈希算法研究

为了解决计算资源和高速网络流量之间的矛盾,需要对IP流进行抽样或负载均衡等处理,而哈希算法是资源代价的核心.首先提出评价哈希算法性能的随机测度;其次从理论上证明比特之间异或运算和位移运算能够提高哈希值的随机特性,提出比特流之间哈希算法的原则;然后分析IP报文的4个字段:源IP、宿IP、源端口和宿端口的特性,由此提出相关的哈希算法;最后使用CERNET主干流量和PMA的数据验证算法的性能,并与IPSX和CRC32算法进行比较.研究表明,基于异或、位移原则的比特流哈希算法的执行效率和哈希值的均匀性两方面具有较好的性质,能够满足高速网络流量测量需求.     

基于XOR Hash的快速IP数据包分类算法研究

文章在哈希算法的基础上,提出了一种基于异或哈希的IP分类算法,该算法的核心有三点:一是将目的/源IP、目的/源端口和协议五域连成比特串,然后分为五块后进行异或,获得分类关键值;二是为了降低冲突率,将异或后的关键值再与一个随机数进行异或,获得最终分类索引值;三是为了保证查找到的规则的正确性,对每一个索引值的源/目的IP地址均匹配一次。通过以上三点改进一般会降低算法的时间复杂度和空间复杂度,通过仿真,当对1万条分类规则进行包分类时,该算法的包分类速度可以达到2Mpps,所消耗的最大内存为6MB。     

How much logic should go in an FPGA logic block

The logic blocks of most FPGAs contain clusters of lookup tables and flip-flops yet little is known about good choices for key parameters. How many lookup tables should a cluster contain, how should FPGA routing flexibility change as cluster size changes, and how many inputs should programmable routing provide each cluster?     

基于软件的网络处理器的路由高速缓存算法研究

路由器需要以较低的代价灵活、高速地实现路由查找这一基本功能.为网络处理器设计了一种基于软件的路由查找高速缓存算法.网络处理器片上高速存储器中的一部分空间被划分出来,由指令代码来维护一个路由查找结果缓存表.通过选择合适的哈希函数,平衡表项之间的冲突并刷新复杂度,该算法可以缩短路由查找的延迟,减少多处理单元对存储器总线的竞争,为其他网络应用提供更多的处理时间.基于真实网络流量的实验表明,即便每个处理单元中仅有少量表项,网络处理器的吞吐量仍然可以得到有效的提升.     

建立高效的网络安全设备IP数据包转发机制

因为网络安全设备对数据包转发的特殊需求,需要比一般设备更高效稳定的数据包转发机制。通过对IP数据包转发机制研究,针对特殊需求,提出了一种基于（IFPLUT＋TCAM）的IP数据包转发机制。该机制将查找表根据输出端口分割为若干个小查找表,并允许查找引擎对每个小查找表进行并行处理,有效地将寻找“最长前缀匹配”的复杂问题简化为“第1前缀匹配”问题。     

SAVI DHCPv6数据报文源地址验证方法研究

由于现今的网络缺乏源地址验证机制,导致多种依靠IP欺骗的恶意攻击时有发生。在DHCPv6场景中防止IP欺骗的源地址验证改进（SAVI）工作,目前正由互联网工程任务组（IETF）驱动,但尚未给出切确地源地址验证方法。为此,本文首先提出两个验证方法：改进的多比特Trie树算法和改进的哈希查找算法,然后实现了SAVI DHCPv6的仿真系统,并使用该系统进行不同验证方法的对比实验。结果表明,本文提出的两种改进方法,比顺序查找方法具有更优的时间性能。     

防火墙寄存器技术的FPGA低功耗布线算法研究*

针对减少毛刺能够有效地降低电路功耗,提出了一种基于防火墙寄存器技术的FPGA低功耗布线算法。在布线过程中,一方面运用算法增加防火墙寄存器滤掉毛刺;另一方面通过修改代价函数,动态地调节输入信号的路径,使信号到达查找表输入端的时间基本趋于一致,从而有效地减少毛刺,降低电路的动态功耗。实验结果表明,在运算时间相同的情况下,与其他算法相比,该算法平均能消除约72%～81%的毛刺,降低约4%～8%的功耗,减少约23%～26%的关键路径延时,而只增加4%的触发器。     

基于P2P的简单查找算法研究

随着P2P系统的发展,它在Internet上的应用越来越广泛,尤其是对分布式数据对象的查找。为此,提出二种基于对等网络系统的简单查找方法。利用一致哈希函数把给定的关键字映射到一个虚拟标识环上的节点,在标识环中查找指定关键字的映射。这种方法能够正确高效地完成数据对象的查找。     

一种基于传输触发体系结构的可重构Hash函数处理器：TTAH

Hash函数是密码学中保证数据完整性的有效手段,性能需求使得某些应用必须采用硬件实现。本文通过分析常用Hash函数在算法上的相似性设计出了专用可重构单元,并将这些可重构单元耦合到传输触发体系结构中,得到一种可重构Hash函数处理器TTAH。常用Hash算法在TTAH上的映射结果表明：与细粒度可重构结构相比,其速度快,资源利用率高;与ASIC相比,可以在额外开销增加较小的前提下有效地支持多种常用Hash函数。     

基于异构网络处理平台的可配置并行路由查表算法研究

基于通用多核的网络转发性能难以满足高速网络流量线速处理的需求.软硬件结合的异构网络处理平台以其较高的性能和灵活性在网络处理领域得到广泛应用,但是如何基于异构平台实现高效的路由查表算法仍需进行深入研究,多核资源利用率低、共享冲突严重和访存次数多的问题是制约传统路由查表算法在异构网络处理平台实现性能提升的主要问题.为此,基于异构网络处理平台（network processing platform,简称NPP)提出一种可配置并行路由查表机制（configurable parallel lookup,简称CPL).CPL中的多线程并行查找和路由表的多副本存储技术在提高多核资源利用率的同时,实现了零冲突访问路由表项.此外,考虑到不同场景下路由前缀分布的差异,CPL支持通过配置对多级路由表的组织结构进行调整,从而有效地减少了路由表访问次数.最后在NPP上,对CPL和传统的查表算法进行性能测试和对比,验证了CPL的可用性和高效性.     

基于改进哈希编码的路由查询匹配算法

路由查找算法是网络路由器关键技术之一,为了提高数据查询性能,提出一种基于改进哈希编码的路由查询匹配算法。利用哈希函数压缩数据名字,采用状态转换阵列实现名称最长前缀的快速匹配,将路由节点缓存因素引入路由决策,采用仿真对比实验对算法的性能进行测试。结果表明,与其它路由查询匹配算法相比,本文算法不仅降低了数据内存开销,大幅度减少平均查询时间,而且提高了数据路由查询的效率。