基于分布式操作系统的数据灾备与拓扑连接挖掘方法

在传统的分布式操作系统灾备处理过程的基础上,结合已有分布式跨边界片段连接优化方法,提出基于空间片段拓扑连接优化的关系代数转换原则,通过利用等价转换规则,进一步简化经过数据本地化处理后的查询树。然后引入连接归并树和执行计划树等概念,并利用相应归并和优化算法将全局空间查询转化为各个场地局部空间数据库的具体执行计划。     

分布式空间数据库集成访问技术

提出了采用网格中间件技术解决面向分布式空间数据库的查询问题,设计并实现了一个网格中间件系统OGSA-SDQP.给出了该系统的设计思想,重点研究了其中的空间数据类型转换、空间数据集成、空间操作函数扩展、空间数据查询流程等关键技术,给出了系统实现及查询性能测试.实验结果表明,OGSA-SDQP能够高效处理网格环境下的分布式空间数据查询.     

形状自适应的二维水印方法

通过对空间数据的拓扑一致性问题的分析,给出了一种基于拓扑的空间数据模型,在此基础上提出了一种空间拓扑规则发现机制.利用所得的空间拓扑规则,判断空间数据是否具有拓扑不一致性,进而采取相应的策略进行拓扑一致性维护,以提高空间数据的质量.     

基于规则的空间一致性维护

通过对空间数据的拓扑一致性问题的分析，给出了一种基于拓扑的空间数据模型，在此基础上提出了一种空间拓扑规则发现机制。利用所得的空间拓扑规则，判断空间数据是否具有拓扑不一致性，进而采取相应的策略进行拓扑一致性维护，以提高空间数据的质量。     

基于网格服务的半连接查询优化算法研究

随着网格计算技术的快速发展,其应用领域在不断扩大,然而,跨网络分布式数据的联合查询往往成为性能的瓶颈,因此文中从提高分布式数据的联合查询的效率出发,以网格计算这一新型web体系结构为技术平台,研究基于网格服务的查询优化目标、查询优化对象和查询优化策略。文中采用了应用非常广泛的启发式搜索算法来缩小策略空间这一优化策略,并在该策略的基础上提出了一种基于网格服务的2-way半连接查询优化算法,最后在网格计算环境下对该算法与一般的全连接算法做了实验对比,证明基于网格服务的2-way半连接查询优化算法可大大缩短查询响应时间。     

集群上一种面向空间连接聚集的并行计算模型

单机运行环境难以满足海量空间数据的连接聚集操作对时空开销的需求,集群上的并行计算是高效处理海量空间数据的连接聚集操作的关键. Map-Reduce是云计算中一种应用于大规模集群进行大规模数据处理的分布式并行编程模型,分析发现,Map-Reduce并不直接支持以既高效又自然的方式来处理具有二次归约特征的并行空间连接聚集操作.因此,提出了一种并行计算模型——Map-Reduce-Combine（MRC）来有效地处理大规模空间数据的连接聚集操作.MRC在Map-Reduce 模型上增加一个Combine阶段,有效地合并分散在各个Reducer的部分聚集结果.针对并行任务划分中空间对象的单分配问题,提出了过滤优化算法,提高了MRC下处理空间连接聚集查询的效率.实验验证所提出的并行计算模型在处理空间连接聚集查询时具有良好的效率、有效性、可扩展性和简单性.     

基于Kd树递归区域划分的分布式空间连接查询

随着空间信息应用需求的不断增长,分布式空间查询处理已经成为空间数据库领域一个重要的研究问题,其中应用最广也是最复杂的一类查询是分布式空间连接查询,分布式空间连接操作的计算代价与传输代价都非常高。目前处理该问题的策略大都要求空间数据集上存在索引并且对数据分布敏感,然而在某些情况下,这个前提并不存在。面对这个问题,本文提出一种基于Kd树递归区域划分的分布式空间连接策略,该策略以最小化网络数据传输代价为目标,基于任务分治的思想对连接区域进行递归划分。实验表明,该策略在不同数据分布情况下均优于传统查询策略,能有效地减小网络传输代价,表现出较好的性能。     

基于GML矢量图层分割的空间数据分布式协同处理的研究

在对传统集中式的GIS的弊端的研究之上,提出了一种基于地理标识语言（GML）矢量图层分割的空间数据分布式协同处理方式。以GML对空间数据进行建模,利用GML基于XML的特点,对GML文档进行解析,并设计了对GML中空间数据进行分割的算法。同时设计相应的分布式空间数据库和空间元数据库。分布式数据库采用按地域分片的策略,用空间元数据库来协助对空间数据的存储管理和查找定位,设计了全局协同模块处理空间数据的发布和查询处理,用来协同对不同地区的空间数据的存放和获取,并用数据加锁的方式来处理多用户并发。     

分布式下MongoDB对激光点云的存储和处理研究

近年来,激光点云数据的应用急剧增加,如何对其进行高效存储和快速处理成为当前的一个重要研究方向。点云数据包含着丰富的地理信息,属于空间数据范畴。传统的关系型数据库对海量空间数据的存储和处理相对薄弱,分布式环境下非关系型数据库的应用为此提供了一个新的研究视角。Sharding技术是数据库水平扩展的一种解决方案,在分布式环境下搭建MongoDB的Sharding集群,通过范围分片和哈希分片对大量激光点云数据进行分布式存储、空间查询和MapReduce运算测试,充分体现了分布式下MongoDB在空间数据的存储和处理方面的巨大优势。     

列存储数据查询中的连接策略优化方法

列存储数据查询优化的重点是列的连接策略.现有的列存储系统通过存储的改变来简化列的连接,致使列的连接缺少查询优化处理,策略单一且无法满足复杂查询.在剖析现有连接选择策略的基础上,提出一种新的连接策略优化方法,即首先利用基于规则的优化方法为列存储数据查询制定优化规则,过滤不可能产生最优计划的候选计划;然后设计了基于代价的优化算法,根据动态Huffman树和左深连接树原理对查询执行顺序进行改进,进一步减少候选计划的规模;根据列存储数据的特点将候选计划中每个连接节点的执行策略归纳为串行连接和并行连接两类,并在此基础上提出代价估计模型,进而可针对这两种连接策略进行代价估计和策略选择.最后在SSB数据集上通过实验证明了方法在列存储数据查询中的有效性.     

The query clustering problem: a set partitioning approach

In this research, we address the query clustering problem which involves determining globally optimal execution strategies for a set of queries. The need to process a set of queries together often arises in deductive database systems, scientific database systems, large bibliographic retrieval systems and several other database applications. We address the optimization problem from the perspective of overlaps in data requirements, and model the batched operations using a set-partitioning approach. In this model, we first consider the case of m queries each involving a two-way join operation. We develop a recursive methodology to determine all the processing strategies in this case. Next, we establish certain dominance properties among the strategies, and develop exact as well as heuristic algorithms for selecting an appropriate strategy. We extend this analysis to a clustering approach, and outline a framework for optimizing multiway joins. The results show that the proposed approach is viable and efficient, and can easily be incorporated into the query processing component of most database systems     

Semantic query optimization for tree and chain queries

Semantic query optimization, or knowledge-based query optimization, has received increasing interest in recent years. The authors provide an effective and systematic approach to optimizing queries by appropriately choosing semantically equivalent transformations. Basically, there are two different types of transformations: transformations by eliminating unnecessary joins, and transformations by adding/eliminating redundant beneficial/nonbeneficial selection operations (restrictions). A necessary and sufficient condition to eliminate a single unnecessary join is provided. We prove that it is 𝒩𝒫-𝒞omplete to eliminate as many unnecessary joins as possible for various types of acyclic queries with the exception of the closure chain queries whose query graphs are chains and all equi-join attributes are distinct. An algorithm is provided to minimize the number of joins in tree queries. This algorithm has an important property that, when applied to a closure chain query, it will yield an optimal solution with the time complexity O(n*m), where n is the number of relations referenced in the chain query, and m is the time complexity of a restriction closure computation     

基于R树的方向关系查询处理

方向关系描述了对象间的空间顺序关系.近年来,方向关系查询处理逐渐受到空间数据挖掘和地理信息系统等空间数据库应用领域研究者的关注.方向关系查询处理需要执行方向连接操作,目前有关空间连接的研究主要集中在拓扑关系和距离关系方面,而较少考虑方向关系.研究了基于R树的方向关系查询处理方法,定义了四元组模型表示对象MBR间的方向关系,提出了基于R树的处理方向关系查询过滤(filter)步骤的方法,并将提炼(refinement)步骤细化为3种不同的操作.所提出的方法能够高效处理任意对象间的方向关系查询.考虑到空间数据挖掘中方向关系查询通常是在满足一定距离约束条件的对象之间进行,还提出了一种同时利用方向和距离约束限制R树搜索空间的查询处理算法.实验证明,与不利用R树的方向关系查询处理方法相比,所提出的方法在I/O开销和CPU开销两方面都具有很高的性能.     

基于最小生成树的数据流窗口连接优化算法

与传统关系数据库不同,数据流管理系统主要处理并发的连续查询.由于查询可能随时增删,所以其主要关注适合查询增删的并发连续查询优化,而不是单条查询优化.提出适合频繁增删查询环境下的数据流窗口连接优化算法.对于新注册的查询以类似最小生成树算法写出数据流的探测序列,然后在不更改其他查询探测序列顺序的情况下尽量合并,减少重复计算.注册或删除查询并不影响其他的查询计划,不需要执行繁琐的查询计划迁移.理论分析和实验证明,该算法简单,优化性能在可接受的范围内,尤其适合查询更新频率较高的系统.     

Symbolic Intersect Detection: A Method for Improving Spatial Intersect Joins

Due to the increasing popularity of spatial databases, researchers have focused their efforts on improving the query processing performance of the most expensive spatial database operation: the spatial join. While most previous work focused on optimizing the filter step, it has been discovered recently that, for typical GIS data sets, the refinement step of spatial join processing actually requires a longer processing time than the filter step. Furthermore, two-thirds of the time in processing the refinement step is devoted to the computation of polygon intersections. To address this issue, we therefore introduce a novel approach to spatial join optimization that drastically reduces the time of the refinement step. We propose a new approach called Symbolic Intersect Detection (SID) for early detection of true hits. Our SID optimization eliminates most of the expensive polygon intersect computations required by a spatial join by exploiting the symbolic topological relationships between the two candidate polygons and their overlapping minimum bounding rectangle. One important feature of our SID optimization is that it is complementary to the state-of-the-art methods in spatial join processing and therefore can be utilized by these techniques to further optimize their performance. In this paper, we also develop an analytical cost model that characterizes SIDs effectiveness under various conditions. Based on real map data, we furthermore conduct an experimental evaluation comparing the performance of the spatial joins with SID against the state-of-the-art approach. Our experimental results show that SID can effectively identify more than 80% of the true hits with negligible overhead. Consequently, with SID, the time needed for resolving polygon intersect in the refinement step is improved by over 50% over known techniques, as predicted by our analytical model.     

Slot index spatial join

Efficient processing of spatial joins is very important due to their high cost and frequent application in spatial databases and other areas involving multidimensional data. This paper proposes slot index spatial join (SISJ), an algorithm that joins a nonindexed data set with one indexed by an R-tree. We explore two optimization techniques that reduce the space requirements and the computational cost of SISJ and we compare it, analytically and experimentally, with other spatial join methods for two cases: 1) when the nonindexed input is read from disk and 2) when it is an intermediate result of a preceding database operator in a complex query plan. The importance of buffer splitting between consecutive join operators is also demonstrated through a two-join case study and a method that estimates the optimal splitting is proposed. Our evaluation shows that SISJ outperforms alternative methods in most cases and is suitable for limited memory conditions.     

分布式数据流连接查询算法

分布式处理是数据流管理系统发展的必然趋势。文章研究了分布式数据流的连接查询，提出DM3Join算法，它由2部分组成：一是通过分解并发的连接请求，合并相同的连接谓词，形成分布式查询操作算子；二是数据流在各分布式代理(Agent)中流转实现部分连接，并在查询引擎处组合成最终结果。DM3Join算法采用了一种类似路由表的结构执行窗口连接，由于可以共享中间结果，算法只需扫描数据1遍。分析和实验证明，该连接算法是高效的。     

Large join optimization on a hypercube multiprocessor

Optimizing large join queries that consist of many joins has been recognized as NP-hard. Most of the previous work focuses on a uniprocessor environment. In a multiprocessor, the location of each join adds another dimension to the complexity of the problem. In this paper, we examine the feasibility of exploiting the inherent parallelism in optimizing large join queries on a hypercube multiprocessor. This includes using the multiprocessor not only to answer the large join query but also to optimize it. We propose an algorithm to estimate the cost of a parallel large join plan. Three heuristics are provided for generating an initial solution, which is further optimized by an iterative local-improvement method. The entire process of parallel query optimization and execution is simulated on an Intel iPSC/2 hypercube machine. Our experimental results show that the performance of each heuristic depends on the characteristics of the query