On the design of multiple key hashing files for concurrent orthogonal range retrieval between two disks

This paper concerns the following problem: given a set of multi-attribute records, a fixed number of buckets and a two-disk system, arrange the records into the buckets and then store the buckets between the disks in such a way that, over all possible orthogonal range queries (ORQs), the disk access concurrency is maximized. We shall adopt the multiple key hashing (MKH) method for arranging records into buckets and use the disk modulo (DM) allocation method for storing buckets onto disks. Since the DM allocation method has been shown to be superior to any other allocation methods for allocating an MKH file onto a two-disk system for answering ORQs, the real issue is knowing how to determine an optimal way for organizing the records into buckets based upon the MKH concept.

A performance formula that can be used to evaluate the average response time, over all possible ORQs, of an MKH file in a two-disk system using the DM allocation method is first presented. Based upon this formula, it is shown that our design problem is related to a notoriously difficult problem, namely the Prime Number Problem. Then a performance lower bound and an efficient algorithm for designing optimal MKH files in certain cases are presented. It is pointed out that in some cases the optimal MKH file for ORQs in a two-disk system using the DM allocation method is identical to the optimal MKH file for ORQs in a single-disk system and the optimal average response time in a two-disk system is slightly greater than one half of that in a single-disk system.     

Locality-preserving hash functions for general purpose parallel computation

Consider the problem of efficiently simulating the shared-memory parallel random access machine (PRAM) model on massively parallel architectures with physically distributed memory. To prevent network congestion and memory bank contention, it may be advantageous to hash the shared memory address space. The decision on whether or not to use hashing depends on (1) the communication latency in the network and (2) the locality of memory accesses in the algorithm.We relate this decision directly to algorithmic issues by studying the complexity of hashing in the Block PRAM model of Aggarwal, Chandra, and Snir, a shared-memory model of parallel computation which accounts for communication locality. For this model, we exhibit a universal family of hash functions having optimal locality. The complexity of applying these hash functions to the shared address space of the Block PRAM (i.e., by permuting data elements) is asymptotically equivalent to the complexity of performing a square matrix transpose, and this result is best possible for all pairwise independent universal hash families. These complexity bounds provide theoretical evidence that hashing and randomized routing need not destroy communication locality, addressing an open question of Valiant.This work was started when the author was a student at Oxford University, supported by a National Science Foundation Graduate Fellowship and a Rhodes Scholarship. Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the author and do not necessarily reflect the views of the National Science Foundation or the Rhodes Trust.     

稀疏差分网络和多监督哈希用于高效图像检索

针对基于深度哈希的图像检索中卷积神经网络（CNN）特征提取效率较低和特征相关性利用不充分的问题,提出一种融合稀疏差分网络和多监督哈希的新方法SDNMSH（sparse difference networks and multi-supervised hashing）,并将其用于高效图像检索。SDNMSH以成对的图像作为训练输入,通过精心设计的稀疏差分卷积神经网络和一个监督哈希函数来指导哈希码学习。稀疏差分卷积神经网络由稀疏差分卷积层和普通卷积层组成。稀疏差分卷积层能够快速提取丰富的特征信息,从而实现整个网络的高效特征提取。同时,为了更加充分地利用语义信息和特征的成对相关性,以促进网络提取的特征信息能够更加有效地转换为具有区分性的哈希码、进而实现SDNMSH的高效图像检索,采用一种多监督哈希（MSH）函数,并为此设计了一个目标函数。在MNIST、CIFAR-10和NUS-WIDE三个广泛使用的数据集上进行了大量的对比实验,实验结果表明,与其他先进的深度哈希方法相比,SDNMSH取得了较好的检索性能。     

一种具有平衡标签预测能力的在线哈希算法

针对传统离线哈希算法训练模型耗时、占用内存大和不易更新模型的问题,以及现实图像集的标签存在大量损失的现象,提出了一种能够平衡标签预测的在线哈希算法（BLPOH）。BLPOH通过标签预测模块生成预测标签,并融合残缺的真实标签,能够有效缓解因标签损失导致的模型性能下降。观察到标签存在分布不平衡现象,提出标签类别相似性平衡算法并应用于标签预测模块,提升标签预测的准确性。将旧数据的信息加入哈希函数的在线更新过程,提升模型对旧数据的兼容性。通过在两个广泛使用的数据集上进行实验,并和一些当前先进的算法进行对比,结果证实了BLPOH的优越性。     

一种等量约束聚类的无监督蒸馏哈希图像检索方法

为了进一步降低无监督深度哈希检索任务中的伪标签噪声,提出了一种等量约束聚类的无监督蒸馏哈希图像检索方法。该方法主要分为两个阶段,在第一阶段中,主要对无标签图像进行软伪标签标注,用于第二阶段监督哈希特征学习,通过所提等量约束聚类算法,在软伪标签标注过程中可以有效降低伪标签中的噪声;在第二阶段中,主要对学生哈希网络进行训练,用于提取图像哈希特征。通过所提出的无监督蒸馏哈希方法,利用图像软伪标签指导哈希特征学习,进一步提高了哈希检索性能,实现了高效的无监督哈希图像检索。为了评估所提方法的有效性,在CIFAR-10、FLICKR25K和EuroSAT三个公开数据集上进行了实验,并与其他先进方法进行了比较。在CIFAR-10数据集上,与TBH方法相比,所提方法检索精度平均提高12.7%;在FLICKR25K数据集上,与DistillHash相比,所提方法检索精度平均提高1.0%;在EuroSAT数据集上,与ETE-GAN相比,所提方法检索精度平均提高16.9%。在三个公开数据集上进行的实验结果表明,所提方法能够实现高性能的无监督哈希检索,且对各类数据均有较好的适应性。     

基于优先级队列的分布式多主题爬虫

在分布式环境中,为提高资源利用率和网页抓取效率,提出一种基于优先级队列的分布式多主题爬虫调度算法PQ‐MCSA。利用基于缓存的扩展式哈希算法对整体任务集进行切割,按照URL逻辑二级节点哈希映射法,将分割后的子任务集均匀地分配到各处理节点中;利用单处理节点的计算能力结合构建的任务优先级队列进行不同主题任务的调度。该算法改善了传统分布式爬虫对单节点的处理资源调度不充分、多主题任务爬取不均匀等缺点。实际项目的应用结果表明,使用该方法能够有效地提高各主题爬取结果的均衡度,具有较强的实用性。     

Key-dependent 3D model hashing for authentication using heat kernel signature

Multimedia-based hashing is considered an important technique for achieving authentication and copy detection in digital contents. However, 3D model hashing has not been as widely used as image or video hashing. In this study, we develop a robust 3D mesh-model hashing scheme based on a heat kernel signature (HKS) that can describe a multi-scale shape curve and is robust against isometric modifications. We further discuss the robustness, uniqueness, security, and spaciousness of the method for 3D model hashing. In the proposed hashing scheme, we calculate the local and global HKS coefficients of vertices through time scales and 2D cell coefficients by clustering HKS coefficients with variable bin sizes based on an estimated L² risk function, and generate the binary hash through binarization of the intermediate hash values by combining the cell values and the random values. In addition, we use two parameters, bin center points and cell amplitudes, which are obtained through an iterative refinement process, to improve the robustness, uniqueness, security, and spaciousness further, and combine them in a hash with a key. By evaluating the robustness, uniqueness, and spaciousness experimentally, and through a security analysis based on the differential entropy, we verify that our hashing scheme outperforms conventional hashing schemes.     

Robust video hashing based on representative-dispersive frames

This study proposes a robust video hashing for video copy detection.The proposed method,which is based on representative-dispersive frames(R-D frames),can reveal the global and local information of a video.In this method,a video is represented as a graph with frames as vertices.A similarity measure is proposed to calculate the weights between edges.To select R-D frames,the adjacency matrix of the generated graph is constructed,and the adjacency number of each vertex is calculated,and then some vertices that represent the R-D frames of the video are selected.To reveal the temporal and spatial information of the video,all R-D frames are scanned to constitute an image called video tomography image,the fourth-order cumulant of which is calculated to generate a hash sequence that can inherently describe the corresponding video.Experimental results show that the proposed video hashing is resistant to geometric attacks on frames and channel impairments on transmission.     

用于图像Hash的视觉相似度客观评价测度

由于评价图像Hash性能时,要求对两幅图像是否在感知上相似做出判断,因此针对这一需求,提出了一种衡量感知相似程度的评价测度。该测度的确定是先对图像进行低通滤波,再进行图像重叠分块;然后运用相关系数检测法计算每一对分块的相似程度,并对相似系数归一化,再分别计算若干个最小和最大的归一化相似系数的乘积;最后用最小相似系数乘积与最大相似系数乘积的比值作为感知相似性的测度。实验结果表明,该测度不仅可有效反映图像视觉质量的变化,而且能较好地区分两幅图像是否存在重要的视觉差异,其对感知相似进行评价的性能优于峰值信噪比。     

基于特征融合的无监督跨模态哈希

已有的无监督跨模态哈希（UCMH）方法主要关注构造相似矩阵和约束公共表征空间的结构,忽略了2个重要问题：一是它们为不同模态的数据提取独立的表征用以检索,没有考虑不同模态之间的信息互补;二是预提取特征的结构信息不完全适用于跨模态检索任务,可能会造成一些错误信息的迁移。针对第一个问题,提出一种多模态表征融合结构,通过对不同模态的嵌入特征进行融合,从而有效地综合来自不同模态的信息,提高哈希码的表达能力,同时引入跨模态生成机制,解决检索数据模态缺失的问题;针对第二个问题,提出一种相似矩阵动态调整策略,在训练过程中用学到的模态嵌入自适应地逐步优化相似矩阵,减轻预提取特征对原始数据集的偏见,使其更适应跨模态检索,并有效避免过拟合问题。基于常用数据集Flickr25k和NUS-WIDE进行实验,结果表明,通过该方法构建的模型在Flickr25k数据集上3种哈希位长检索的平均精度均值较DGCPN模型分别提高1.43%、1.82%和1.52%,在NUS-WIDE数据集上分别提高3.72%、3.77%和1.99%,验证了所提方法的有效性。