大规模多维网络数据分析框架的研究与实现

随着互联网的快速发展和计算机应用的不断增加,大量的图数据特别是社会网络数据不断生成。多维信息网络已经成为表示这些图数据的通用方式。但是在多维信息网络中,节点的类型多种多样,节点的属性也不尽相同,如何对多维信息网络数据进行多角度多粒度的分析,挖掘其中的隐藏信息,成为人们关注的焦点。图联机分析处理技术(graph online analytical processing,GraphOLAP)可以对图数据进行快速的联机分析以及查询操作。借助于GraphOLAP的现有成果,针对多维信息网络的特点,提出了新的数据立方体框架。引入主节点的概念来指导元路径的生成,通过元路径指导网络的上卷下钻,提出属性转化和同质转化来丰富OLAP操作。最后讨论了优化的物化策略,使用并行计算框架Spark来实现算法,通过多个数据集验证了框架的有效性和高效性。     

基于Apriori算法和OLAP的关联规则挖掘模型设计

通过分析Apriori算法的特点，提出一种有针对性的联机分析处理（OLAP）关联规则挖掘模型。该模型在数据立方体维度和度量值设计上充分考虑了Apriori算法的特点，使数据立方体物化更多算法所需要的中间数据；同时适当改进算法使之适应所设计的数据立方体。研究表明，该设计方法在灵活性和效率方面都有较好的表现。     

基于DBLP数据的多维异质网络Graph OLAP设计与实现

为了分析DBLP数据中的多种类型的实体信息, 挖掘其中特定的知识, 首先根据异质网络Graph OLAP（图联机分析处理）模型, 建立相应的数据仓库模型; 然后依据实体维的概念, 构建多维异质图立方模型; 最后针对 Graph OLAP处理异质网络能力不足的问题, 补充了旋转和拉伸操作, 并完善了Graph OLAP原型系统Liter Miner。实证表明设计的原型系统可以有效地对DBLP数据中的多维异质网络进行分析, 挖掘出研究人员需要的知识。     

Spark环境下并行立方体计算方法

针对传统联机分析处理(OLAP)处理大数据时实时响应能力差的问题,研究基于分布式内存计算框架Spark加速的数据立方体计算方法,设计基于Spark内存集群的自底向上构造(BUC)算法——BUCPark,来提高BUC的并行度和大数据适应能力。在此基础上,为避免内存中迭代的立方体单元膨胀,基于内存重复利用和共享的思想设计改进的BUCPark算法——LBUCPark。实验结果表明:LBUCPark算法性能优于BUC算法和BUCPark算法,能够胜任大数据背景下的快速数据立方体计算任务。     

多维数据立方体模型及其基于关系的实现

数据立方体模型是多维数据库和以多维分析为基础的联机分析处理技术的核心机制,本文参照当前对多维数据立方体模型的最新研究成果,提出了一种新的模型并且应用于OLAP服务器基于关系的实现。并且提出了实现的难点和改进的方向。     

基于数据立方体的多维关联规则的挖掘方法

总结了现有基于数据立方体的多维关联规则挖掘算法，在此基础上将联机分析处理(OLAP)的钻取操作引入关联规则挖掘过程，提出Apriori_cubc算法的改进算法。通过动态调整立方体的维层次，来挖掘出用户感兴趣的关联规则。     

多维数据立方体模型及其基于关系的实现

数据立方体模型是多维数据库和以多维分析为基础的联机分析处理技术的核心机制。本文参照当前对鑫维数据立方体模型的最新研究成果,提出了一种新的模型并且应用于OLAP服务器基于关系的实现。并且提出了实现的难点和改进的方向。     

OLAP性能测试方法研究与实现

随着商业智能市场的逐步扩大,联机分析处理(OLAP)系统的使用质量评估已经成为数据库应用的研究热点.作为效用特性的OLAP系统性能评估需要一个性能基准.以OLAP委员会推出的APB-1性能基准为基础,首先设计了面向多维数据库的立方体(Cube)模型以及相应的多维表达式(MDX)查询模板,在Cube模型设计的过程中修改了APB-1基准ROLAP星型模型的不足之处;接着在测试数据一致和测试参数一致的前提下,通过对设计的MOLAP模型查询结果与ROLAP模型查询结果进行对比分析,证明了MOLAP模型及MDX查询模板设计的正确性;然后给出了OLAP性能测试流程,描述了支持ROLAP和MOLAP性能测试的工具框架及其主要模块.最后使用该测试框架在商业数据库管理系统上对ROLAP和MOLAP进行并发查询实践,验证了框架的有效性.提出的方法及技术实现为未来OLAP产品性能的测试和评价提供多维数据模型、业务模型和工具的支持.     

一种高效的多维关联规则挖掘算法研究

对经典关联规则挖掘算法进行深入研究的基础上,结合数据立方体的结构特点和OLAP技术,给出了一种高效的多维关联规则挖掘算法,并对不同数据立方体下的算法的性能进行了分析比较.     

基于社区划分的联机分析处理查询优化方案

针对P2P环境下的联机分析处理（OLAP）查询节点数目不断增加时,易造成网络拥塞、查询效率降低的问题,提出一种基于社区划分的OLAP查询优化方案。该方案构建一个虚拟的社区网,并在此结构上设计了一种基于社区划分的多维数据集(CPDS)的OLAP查询优化算法。实验结果表明,该算法能有效避免因网络节点数目递增而导致的网络负载加剧问题,能有效地减少网络拥塞,优化了OLAP的查询效率,进一步提高P2P环境下OLAP的决策分析性能。     

澜沧江数据仓库中空间度量的物化选取

空间数据仓库技术是将数据仓库和联机分析处理技术应用到空间信息领域,以有效地支持空间数据分析和决策。空间度量的物化是提高空间联机分析处理响应速度的关键。该文在介绍澜沧江空间数据仓库基础上,介绍了空间度量的物化选取策略。     

Graph OLAP: a multi-dimensional framework for graph data analysis

Databases and data warehouse systems have been evolving from handling normalized spreadsheets stored in relational databases, to managing and analyzing diverse application-oriented data with complex interconnecting structures. Responding to this emerging trend, graphs have been growing rapidly and showing their critical importance in many applications, such as the analysis of XML, social networks, Web, biological data, multimedia data and spatiotemporal data. Can we extend useful functions of databases and data warehouse systems to handle graph structured data? In particular, OLAP (On-Line Analytical Processing) has been a popular tool for fast and user-friendly multi-dimensional analysis of data warehouses. Can we OLAP graphs? Unfortunately, to our best knowledge, there are no OLAP tools available that can interactively view and analyze graph data from different perspectives and with multiple granularities. In this paper, we argue that it is critically important to OLAP graph structured data and propose a novel Graph OLAP framework. According to this framework, given a graph dataset with its nodes and edges associated with respective attributes, a multi-dimensional model can be built to enable efficient on-line analytical processing so that any portions of the graphs can be generalized/specialized dynamically, offering multiple, versatile views of the data. The contributions of this work are three-fold. First, starting from basic definitions, i.e., what are dimensions and measures in the Graph OLAP scenario, we develop a conceptual framework for data cubes on graphs. We also look into different semantics of OLAP operations, and classify the framework into two major subcases: informational OLAP and topological OLAP. Second, we show how a graph cube can be materialized by calculating a special kind of measure called aggregated graph and how to implement it efficiently. This includes both full materialization and partial materialization where constraints are enforced to obtain an iceberg cube. As we can see, due to the increased structural complexity of data, aggregated graphs that depend on the underlying “network” properties of the graph dataset are much harder to compute than their traditional OLAP counterparts. Third, to provide more flexible, interesting and informative OLAP of graphs, we further propose a discovery-driven multi-dimensional analysis model to ensure that OLAP is performed in an intelligent manner, guided by expert rules and knowledge discovery processes. We outline such a framework and discuss some challenging research issues for discovery-driven Graph OLAP.     

High Performance OLAP and Data Mining on Parallel Computers

On-Line Analytical Processing (OLAP) techniques are increasingly being used in decision support systems to provide analysis of data. Queries posed on such systems are quite complex and require different views of data. Analytical models need to capture the multidimensionality of the underlying data, a task for which multidimensional databases are well suited. Multidimensional OLAP systems store data in multidimensional arrays on which analytical operations are performed. Knowledge discovery and data mining requires complex operations on the underlying data which can be very expensive in terms of computation time. High performance parallel systems can reduce this analysis time. Precomputed aggregate calculations in a Data Cube can provide efficient query processing for OLAP applications. In this article, we present algorithms for construction of data cubes on distributed-memory parallel computers. Data is loaded from a relational database into a multidimensional array. We present two methods, sort-based and hash-based for loading the base cube and compare their performances. Data cubes are used to perform consolidation queries used in roll-up operations using dimension hierarchies. Finally, we show how data cubes are used for data mining using Attribute Focusing techniques. We present results for these on the IBM-SP2 parallel machine. Results show that our algorithms and techniques for OLAP and data mining on parallel systems are scalable to a large number of processors, providing a high performance platform for such applications.     

OLAP数据挖掘引擎算法的设计与实现

基于OLAP的数据挖掘，是数据挖掘的一个新的发展方向。对于如何把OLAP(联机分析处理技术)和DM(数据挖掘)统一起来，从而在数据库或数据仓库的不同层次进行挖掘，提出了OLAP数据挖掘系统的结构。通过研究数据挖掘方法和OLAP操作的特点，以及数据立方的构建和物化，对传统的DM算法进行了改进，设计并实现了更能适应OLAP数据挖掘引擎的算法。     

超大型压缩数据仓库上的CUBE算法

数据压缩是提高多维数据仓库性能的重要途径,联机分析处理是数据仓库上的主要应用,Cube操作是联机分析处理中最常用的操作之一.压缩多维数据仓库上的Cube算法的研究是数据库界面临的具有挑战性的重要任务.近年来,人们在Cube算法方面开展了大量工作,但却很少涉及多维数据仓库和压缩多维数据仓库.到目前为止,只有一篇论文提出了一种压缩多维数据仓库上的Cube算法.在深入研究压缩数据仓库上的Cube算法的基础上,提出了产生优化Cube计算计划的启发式算法和3个压缩多维数据仓库上的Cube算法.所提出的Cube算法直     

Object-based selective materialization for efficient implementationof spatial data cubes

With a huge amount of data stored in spatial databases and the introduction of spatial components to many relational or object-relational databases, it is important to study the methods for spatial data warehousing and OLAP of spatial data. In this paper, we study methods for spatial OLAP, by integrating nonspatial OLAP methods with spatial database implementation techniques. A spatial data warehouse model, which consists of both spatial and nonspatial dimensions and measures, is proposed. Methods for the computation of spatial data cubes and analytical processing on such spatial data cubes are studied, with several strategies being proposed, including approximation and selective materialization of the spatial objects resulting from spatial OLAP operations. The focus of our study is on a method for spatial cube construction, called object-based selective materialization, which is different from cuboid-based selective materialization (proposed in previous studies of nonspatial data cube construction). Rather than using a cuboid as an atomic structure during the selective materialization, we explore granularity on a much finer level: that of a single cell of a cuboid. Several algorithms are proposed for object-based selective materialization of spatial data cubes, and a performance study has demonstrated the effectiveness of these techniques     

Efficient aggregation algorithms for compressed data warehouses

Aggregation and cube are important operations for online analytical processing (OLAP). Many efficient algorithms to compute aggregation and cube for relational OLAP have been developed. Some work has been done on efficiently computing cube for multidimensional data warehouses that store data sets in multidimensional arrays rather than in tables. However, to our knowledge, there is nothing to date in the literature describing aggregation algorithms on compressed data warehouses for multidimensional OLAP. This paper presents a set of aggregation algorithms on compressed data warehouses for multidimensional OLAP. These algorithms operate directly on compressed data sets, which are compressed by the mapping-complete compression methods, without the need to first decompress them. The algorithms have different performance behaviors as a function of the data set parameters, sizes of outputs and main memory availability. The algorithms are described and the I/O and CPU cost functions are presented in this paper. A decision procedure to select the most efficient algorithm for a given aggregation request is also proposed. The analysis and experimental results show that the algorithms have better performance on sparse data than the previous aggregation algorithms     

属性维概念及其操作的研究

Dimension member attribute is used to describe the property of dimension members. It is not fully understood and well defined in OLAP research area.This paper focuses on a special kind of dimension member attributes,which can be used as dimensions by themselves. We call them attribute dimensions.In order to facilitate this kind of necessity of multidimensional data modeling in many real-world applications,the traditional multidimensional structure is extended and a group of operations are given to formulate corresponding multidimebsuibak qyerues.     

一种基于实时语义维层次链的OLAP查询算法

目前,P2P环境下的OLAP查询策略都是基于从客户端获取查询结果集,如DSCD算法和DQDC算法等主要是研究怎样快速地从客户端获取查询结果集,由于客户端的Data Cube的实时数据更新效率低,易导致查询结果失真,从而影响OLAP的查询效率。为了提高P2P网络中OLAP的实时查询效率,提出了一种RTOS（Real-time Semantic OLAP Search,实时语义的OLAP查询）算法,并结合查询速度和失真率两方面的实验证明,该算法能有效地提高P2P环境下OLAP的决策分析性能。