数据仓库系统中一种改进的维层次聚集Cube存储结构

提出利用Cube中的维层次（dimension hierarchy）聚集技术来创建高性能的维层次聚集Cube（dimension hierarchy aggregate cube,DHAC）．充分利用DHAC已保存的维层次信息,对Cube中多维数据的查询和更新效率进行了优化,并且支持Cube的上探、下钻等语义操作．在DHAC中进行数据插入和删除等数据更新时,由下向上用更新前后的差值对受到更新结点影响的所有祖先结点进行增量更新．实现了在插入新维或维层次时不需要重新构建聚集Cube就可以实现Cube的模式更新．对维层次聚集Cube与传统Cube进行了算法性能分析和比较,理论分析和实验结果都表明,所提出的DHAC性能最佳．     

基于维层次的语义Cube存储与增量更新技术研究*

利用维的层次性为每一个维建立一个索引,同时保存相应的层次信息和预聚集数据,提出了基于维层次的语义Cube.在进行数据更新时,使用更新前后的差值自下而上对受到更新单元影响的祖先节点进行增量更新,在进行模式更新时,无须重构Cube,即可实现增量更新.由于其存储结构的灵活性,在高效完成增量更新的同时实现了Cube上进行上探、下钻等语义操作.理论分析和实验结果均表明,提出的基于维层次的语义Cube与传统Cube相比,性能显著提高.     

一种P2P网络环境下的OLAP模式匹配方案

P2P网络环境下,每个OLAP服务器上都有一套完整的数据解决方案。当OLAP服务器之间进行协同工作时,如何将Data Cube中的模式进行匹配以完成联合OLAP查询成为一个关键问题。在对Data Cube中的维及维层次链进行了定义后,提出了一种维层次链的匹配规则,能较好地优化P2P网络环境下执行联合OLAP查询。     

DHMC:一种有效的高维Cube并行分布式存储结构

在数据仓库系统中，数据立方体（Cube）及其预聚集处理在OLAP起到非常重要的作用．对于一d个d维的dataCube可以生成2d个聚集Cuboids和d∏i=1（｜Di｜＋1）个聚集数据单元，但对于一个高维Cube，要创建这些所有聚集Cuboids是不现实的．提出通过共享分段立方体Mini．Cube的高维Cube并行分布式存储结构（DHMC），将高维Cube划分成若干个低维共享分段立方体Mini-Cube，利用并行分布式处理技术来创建这些分割的分段共享Mini—Cube及其聚集Cuboids，来实现高维Cube的并行创建和增量更新维护，从而解决高维OLAP聚集海量数据的存储与查询问题．理论分析与实验结果均表明DHMC性能最佳．     

一种基于维层次编码的OLAP聚集查询算法

联机分析处理(OLAP)查询往往需在海量数据上进行即席的复杂分组聚集查询，在其SQL语句中通常包含多表连接和分组聚集操作，因而减少多表连接和压缩关键字，以及对查询数据进行有效地分组聚集操作，成为ROLAP查询处理的关键问题。提出了一种基于维层次编码的新型预分组聚集算法DHEPGA．DHEPGA算法充分利用了编码长度较小的维层次编码及其前缀，来快速检索出与查询关键字相匹配的维层次编码，求得维层次属性的查询范围，减少了I／O开销，提高了OLAP查询效率。理论分析和实验结果表明，DHEPGA算法性能是非常有效的。     

HDC结构的数据更新算法实现

数据更新是数据仓库上支持联机分析处理的一种重要操作。增量更新是一种有效的数据更新方法。实现了二维层次式数据立方体(Cube)存储结构HDC的建立以及基于此结构的数据增量更新算法。     

基于层次聚簇的Cube存储结构的研究

数据仓库中的维数据通常都是有层次的,基于维层次路径的聚簇能有效地在物理空间上将关联数据组织到一起,减少查询访问磁盘的次数。而现在的Cube存储结构都关注于Cube操作的计算和存储,忽视了这一特点。论文提出基于维层次聚簇的Cube存储结构HC(HierarchicallyClustered)Cube及相关算法,解决了目前存在的问题。     

基于ROLAP层次维计算技术的研究

在研究BUC算法的基础上探讨了维层次数据的计算方法,提出一种改进的雏层次计算方法,减小其排序开销,加快计算速度,从而提高聚集查询处理的性能.     

基于层次有序维的分组聚集算法

在联机分析处理(OLAP)中,有效地维度模型对海量数据的即席复杂分组聚集查询起着关键的作用.在偏序和映射的基础上,通过定义层次有序维,提出一种基于层次有序维的分组聚集算法.该算法利用维属性之间的聚集关系,通过约束层次链中的元素次序,实现了分组聚集计算中多表连接转换为维范围的查询,提高了连接和聚集效率.最后,实验结果验证了该算法的有效性.     

基于KDG-tree的数据库多维索引技术

针对目前网格索引(Grid index)的冗余数据及KD-tree等多维索引的维度灾难等问题,提出一种将网格索引与二叉搜索树结合起来的高效索引结构KDG-tree。KDG-tree通过纵横向指针将结点链接起来构成二叉索引树,树中的结点分为中间索引结点和叶子结点,所有数据对象只存于叶子结点。创建索引时分别从高维到低维按结点索引值顺序插入,查找对象时逐维搜索。实验分析表明,KDG-tree避免了Grid index的数据冗余,又改进了KD-tree与KDB-tree的性能,是一种适合高维海量数据的多维索引。     

概念格的快速渐进式构造算法

概念格作为形式概念分析理论中的核心数据结构，已经在知识工程和软件工程等领域得到了广泛的应用。概念格的快速构造在其应用过程中具有重要的意义，研究人员已经提出了一系列构造概念格的算法，其中渐进式算法是很有前途的一类。该文通过对概念格渐进式构造过程的分析，识别出要解决的基本问题，提出了采用树结构对概念格节点进行组织，研究了基于这种树状组织的概念格快速渐进式算法，并给出了算法的伪码。概念格节点的树结构组织有利于识别出格节点的类型以及约束新生格节点的父节点和子节点的搜索范围，从而可以有效地减少算法的执行时间。实验结果表明，基于这种树状索引的渐进式构造算法的时间性能要明确优于著名的Godin算法。     

Efficient Incremental Maintenance for Distributive and Non-Distributive Aggregate Functions

Data cube pre-computation is an important concept for supporting OLAP （Online Analytical Processing） and has been studied extensively. It is often not feasible to compute a complete data cube due to the huge storage requirement. Recently proposed quotient cube addressed this issue through a partitioning method that groups cube cells into equivalence partitions. Such an approach not only is useful for distributive aggregate functions such as SUM but also can be applied to the maintenance of holistic aggregate functions like MEDIAN which will require the storage of a set of tuples for each equivalence class. Unfortunately, as changes are made to the data sources, maintaining the quotient cube is non-trivial since the partitioning of the cube cells must also be updated. In this paper, the authors design incremental algorithms to update a quotient cube efficiently for both SUM and MEDIAN aggregate functions. For the aggregate function SUM, concepts are borrowed from the principle of Galois Lattice to develop CPU-efficient algorithms to update a quotient cube. For the aggregate function MEDIAN, the concept of a pseudo class is introduced to further reduce the size of the quotient cube, Coupled with a novel sliding window technique, an efficient algorithm is developed for maintaining a MEDIAN quotient cube that takes up reasonably small storage space. Performance study shows that the proposed algorithms are efficient and scalable over large databases.     

一种并行处理多维连接和聚集操作的有效方法

随着并行计算算法的完善和廉价、功能强大的多处理机系统的成熟，使得采用多处理机系统来并行处理多维数据仓库的连接和聚集操作成为当前有效提高OLAP查询处理性能的首选技术．为此，提出一种降低连接和聚集操作开销的并行算法PJAMDDC(parallel join and aggregation for multi-dimensional data cube)．算法充分考虑了多维数据立方体的存储机制和多处理机分布系统的结构特点，在原有聚集计算多维数据立方体的搜索点阵逻辑结构的基础上，采用多维数据仓库的层次联合代理(hierarchy combined surrogate)和对立方体的搜索点阵进行加权的方法，使得立方体数据在多个处理机间的分配达到最佳的状态，从而在分割多维数据的同时，提高了并行处理多维连接和聚集操作的效率．算法实验评估表明，PJAMDDC算法并行处理多维数据仓库的连接和聚集操作是有效的．     

PMC: Select Materialized Cells in Data Cubes

QC-Tree is one of the most storage-efficient structures for data cubes in an MOLAP system. Although QC-Tree can achieve a high compression ratio, it is still a fully materialized data cube. In this paper, an improved structure PMC is presented allowing us to materialize only a part of the cells in a QC-Tree to save more storage space. There is a notable difference between our partially materialization algorithm and traditional materialized views selection algorithms. In a traditional algorithm, when a view is selected, all the cells in this view are to be materialized. Otherwise, if a view is not selected, all the cells in this view will not be materialized. This strategy results in the unstable query performance. The presented algorithm, however, selects and materializes data in cell level, and, along with further reduced space and update cost, it can ensure a stable query performance. A series of experiments are conducted on both synthetic and real data sets. The results show that PMC can further reduce storage space occupied by the data cube, and can shorten the time to update the cube.     

Communication and memory optimal parallel data cube construction

Data cube construction is a commonly used operation in data warehouses. Because of the volume of data that is stored and analyzed in a data warehouse and the amount of computation involved in data cube construction, it is natural to consider parallel machines for this operation. This paper addresses a number of algorithmic issues in parallel data cube construction. First, we present an aggregation tree for sequential (and parallel) data cube construction, which has minimally bounded memory requirements. An aggregation tree is parameterized by the ordering of dimensions. We present a parallel algorithm based upon the aggregation tree. We analyze the interprocessor communication volume and construct a closed form expression for it. We prove that the same ordering of the dimensions in the aggregation tree minimizes both the computational and communication requirements. We also describe a method for partitioning the initial array and prove that it minimizes the communication volume. Finally, in the cases when memory may be a bottleneck, we describe how tiling can help scale sequential and parallel data cube construction. Experimental results from implementation of our algorithms on a cluster of workstations show the effectiveness of our algorithms and validate our theoretical results.     

Parallel H-Tree Based Data Cubing on Graphics Processors

Graphics processing units (GPUs) have an SIMD architecture and have been widely used recently as powerful general-purpose co-processors for the CPU. In this paper, we investigate efficient GPU-based data cubing because the most frequent operation in data cube computation is aggregation, which is an expensive operation well suited for SIMD parallel processors. H-tree is a hyper-linked tree structure used in both top-k H-cubing and the stream cube. Fast H-tree construction, update and real-time query response are crucial in many OLAP applications. We design highly efficient GPU-based parallel algorithms for these H-tree based data cube operations. This has been made possible by taking effective methods, such as parallel primitives for segmented data and efficient memory access patterns, to achieve load balance on the GPU while hiding memory access latency. As a result, our GPU algorithms can often achieve more than an order of magnitude speedup when compared with their sequential counterparts on a single CPU. To the best of our knowledge, this is the first attempt to develop parallel data cubing algorithms on graphics processors.     

一种改进的关联规则的增量式更新算法

增量关联规则挖掘的主要思想是在原有规则的基础上,去除那些不满足条件的旧规则,发现满足条件的新规则,目的是尽量减少计算量.增量规则算法主要解决两类问题,即最小支持度的更新和数据库的更新.目前大多数算法对上述两个条件只更新其中一个,另一个保持不变,而实际应用中往往需要两者都更新.通过对数据挖掘中的IUA算法和FUP算法的分析和研究,提出IFU算法,用于解决数据库和最小支持度均发生改变时关联规则的增量式更新问题.相对于IUA算法和FUP算法以及基于他们改进的算法,该算法不仅扩展了更新条件,而且减少了对事务数据库和新增数据库的扫描次数.模拟实验表明IFU算法提高了更新效率.