基于键值存储的分布式时序相似性搜索方法

时序相似性搜索是时序数据分析最基本的操作之一,具有广泛的应用场景.针对现有分布式算法无法应对维度增长、扫描范围过大和相似性计算耗时的问题,提出一种面向键值存储的分布式时序相似性搜索方法KV-Search.首先对时序数据分块,并设计其键值存入键值数据库,解决了时序数据维度高且不断增长的问题;其次,基于切比雪夫距离计算其下界,并利用键值范围扫描提前过滤无效数据,减少了数据传输;最后,利用基于分块的时序表示计算距离下界,避免了更高维度真实数据的计算,加快了查询效率.使用HBase实现了KV-Search,并利用真实的大规模数据集做了大量实验.实验结果表明, KV-Search算法在效率和扩展性方面均优于基准实验.

Distributed Time Series Similarity Search Method Based on Key-value Data Stores

Time series similarity search is one of the most basic operations for temporal data analysis, which has various application scenarios. Existing distributed methods face the problems of dimension explosion, too large scan range, and time-consuming similarity calculation. To this end, this paper proposes a novel distributed time series similarity search algorithm KV-Search. First, time series are segmented into blocks and stored in the key-value database, which solves the problem of high and growing dimension. Second, the lower bound is calculated based on Chebyshev distance, and the invalid data is filtered out in advance using key value range scanning, which reduce the data transmission and calculation overhead. Third, a block-based time series representation is used to calculate the lower bound of distance, which avoids the calculation of higher dimensional real data. We implement KV-Search based on HBase, and conduct a set of extensive experiments using both real and synthetic time series data. The results show that our KV-Search is superior to benchmark experiment in efficiency and scalability.