基于一致性Hash的分布式海量分子检索模型

针对大数据环境下,传统通用图匹配检索低效、折射率数据无法快速定位的问题,建立了基于一致性Hash的分布式海量分子检索模型。模型结合分子特点,将连续的折射率通过等宽算法离散化建立高速Hash索引,实现分布式海量分子检索系统,有效减小了参与计算的分子数据规模,并根据分子访问频次处理冲突从而提高分子检索效率。实验结果表明,在包含20万个分子的数据中,该方法平均检索耗时约为通用图匹配平均检索耗时的5%,模型性能稳定,具有高可扩展性;对于海量数据环境下依据折射率检索高频次分子较为适用。

Distributed massive molecule retrieval model based on consistent Hash

In view of the problems that the traditional general graph matching search is inefficient, and refractive index data cannot be positioned fast in large data environment, a distributed massive molecular retrieval model based on consistent Hash function was established. Combined with the characteristics of molecular storage structures, to improve retrieval efficiency of molecules, the continuous refractive index was discretized by fixed width algorithm to establish high-speed Hash index, and the distributed massive retrieval system was realized. The size of dataset was effectively reduced, and Hash collision was handled according to the visiting frequency. The experimental results show that, in the chemical data containing 200 thousand structures of molecules, the average time of this method is about five percent of the traditional general graph matching search. Besides, the model has the steady performance with high scalability. It is applicable to retrieve high-frequency molecules in accordance with refractive index under the environment of massive data.