一种基于多维数据库的OLAP实现方法ADT：数据仓库的工具应用讨论

“数据仓库”是近数年来兴起的一种数据处理技术,其特点是对数据的长期保留并作出分析的支持决策,在其应用的工具中,国外新兴了一门针对特定问题的联机数据访问和分析的软件技术,称之为OLAP(On_Line Analytical Process-ing).OLAP的两种实现技术MOLAP(多维数据库OLAP)和ROLAP(关系型OLAP)现在正处于激烈的竞争之中.事实上,这两种技术各有特色,各有利弊,本文结合了ROLAP与MOLAP两种技术的特点,阐述了采用MOLAP ADT方法来实现的OLAP技术,相信研究数据库和数据仓库的读者会对此有一定的兴趣.     

MOLAP ROLAP可以兼得

OLAP 有三种流派 MOLAP,ROLAP,HOLAP。MOLAP 国内经常称其为“多维数据库”技术。当进行大数据量的复杂查询时,利用多维数据库技术可大大加快查询(与传统数据库相比)的速度。但他的限制是,多维数据库不能支撑太大的数据量(通常300GB 就已经是上限了)。ROLAP 实际上就是用传统的关系型数据库来模拟多维数据库。其特点是可以支撑大数据量,但缺陷是查询速度表较慢。真是鱼和熊掌不能兼得。HOLAP 中文称为“混合型 OLAP”。其工作时把访问较频繁的汇总数据存储在多维数据库中,把大量的原始数据存放于传统的关系型数据库中。查询时 HOLAP引擎需要自动判断从何处取得查询数据并集中计算,返     

基于缓存机制的ROLAP系统改进方法研究

ROLAP相对于MOLAP在处理多维数据性能方面有很多不足.给出ROLAP的体系结构,研究了改进ROLAP系统性能的若干方法,如实视图技术、索引技术以及并行处理技术,重点分析了如何在SQL生成器中处理查询优化,并提出在ROLAP设计和应用中引入缓存机制,进一步提高ROLAP系统的查询性能.     

提高0LAP系统性能的方法研究

1.引言随着市场竞争的日趋激烈,近年来企业更加强调决策的及时性和准确性,这使得以支持决策管理分析为主要目的的应用迅速崛起,这类应用被称为联机分析处理(OLAP),OLAP应用主要是指通过各种即席复杂查询,对数据仓库中存储的数据进行各种统计分析的应用。由于各种OLAP查询涉及大量的数据,非常复杂,并要求比较快的响应速度,因此除了研究如何形式化描述OLAP查询与操作外,还需要研究各种OLAP查询和处理的有效方法。本文首先介绍OLAP的概念和分类,然后研究了提高OLAF系统应用性能的主要的技术。文章的第四部分分析ROLAP和MOLAP两种方式的优缺点,给出了一个结合ROLAP和MOLAP的查询解决方案。最后给出结论和进一步的研究方向。     

一种适应GPU的混合OLAP查询处理模型

通用GPU因其强大的并行计算能力成为新兴的高性能计算平台,并逐渐成为近年来学术界在高性能数据库实现技术领域的研究热点.但当前GPU数据库领域的研究沿袭的是ROLAP(relational OLAP)多维分析模型,研究主要集中在关系操作符在GPU平台上的算法实现和性能优化技术,以哈希连接的GPU并行算法研究为中心.GPU拥有数千个并行计算单元,但其逻辑控制单元较少,相对于CPU具有更强的并行计算能力,但逻辑控制和复杂内存管理能力较弱,因此并不适合需要复杂数据结构和复杂内存管理机制的内存数据库查询处理算法直接移植到GPU平台.提出了面向GPU向量计算特性的混合OLAP多维分析模型semi-MOLAP,将MOLAP(multidimensionalOLAP)模型的直接数组访问和计算特性与ROLAP模型的存储效率结合在一起,实现了一个基于完全数组结构的GPU semi-MOLAP多维分析模型,简化了GPU数据管理,降低了GPU semi-MOLAP算法复杂度,提高了GPU semi-MOLAP算法的代码执行率.同时,基于GPU和CPU计算的特点,将semi-MOLAP操作符拆分为CPU和GPU平台的协同计算,提高了CPU和GPU的利用率以及OLAP的查询整体性能.     

联机分析处理的实现及其性能调优

本文首先对联机分析处理(OLAP)技术进行了简要的介绍．接下来提出了联机分析处理的三种实现方式．关系型联机分析处理(ROLAP)．多维联机分析处理(MOLAP)．混合联机分析处理(HOLAP)．其中对多维联机分析处理的存储方式进行了重点描述．并阐述了基于多维数据组织的MOLAP的性能调优方法．用以提高联机分析处理工具的开发效率．推动计算机应用技术的进一步发展。     

一个面向OLAP的多维层次聚簇存储模式

文献[2]针对ROLAP提出的多维层次聚簇存储模式（MHC）,极大地提高了查询效率。然而与ROLAP相比,MOLAP往往具有更高的存储效率和查询效率。这让人自然地联想到：如果能构造一个集二者优点为一身的混合型OLAP系统,以实现MHC,也许能进一步提高系统性能。作为这一设想的探索性研究,本文利用ORDBMS的可扩展性实现了这一原形系统：多维数据按维层次分块聚簇,其中每个分块以数组ADT存储,分块间以B^＋树索引聚簇。实验表明,本文提出的MHC实现能有效减少存储空间,进一步提高查询性能。     

MD-SQL：一种基于MDX的多维数据查询语言

多维数据分析技术在建立企业级数据仓库的过程中起着关键的作用,同时也是面向联机分析处理(OLAP)的核心.本文在分析了传统多维数据查询语言MDX(Multi-Dimensional eXpression)不足的基础上,提出了一种改进的多维数据查询的语言模型MD-SQL,并给予实现,最后对模型提出了优化方法,通过相关实验证明MD-SQL语言性能优于MDX语言.     

论OLAP的三种实现技术

OLAP（On—Line Analytical Processing）是数据仓库系统的主要应用，它支持复杂的分析操作，侧重决策支持。文章从数据的存储方式，钻取操作，数据更新，和性能上研究了OLAP的三种实现技术ROLAP（Kehtional OLAP），MOLAP（Multidimensional OLAP）HOLAP（Hybrid OLAP）的不同和三种实现技术的适用范围。     

基于聚类的OLAP多维分析查询推荐方法研究

为提高用户在使用OLAP(联机分析处理)时的查询效率,改善用户体验,对OLAP中的MDX(多维表达式)查询语言进行了研究,提出了在MDX查询语言的基础上,对OLAP用户的历史查询记录进行聚类的方法,建立了查询操作的分级推荐机制,提供给当前用户候选查询作为推荐,帮助用户设计下一步查询操作.介绍了该方法的推荐机制构架,并表明了该方法的可行性.     

Spatial data warehouses and spatial OLAP come towards the cloud: design and performance

Cloud computing systems handle large volumes of data by using almost unlimited computational resources, while spatial data warehouses (SDWs) are multidimensional databases that store huge volumes of both spatial data and conventional data. Cloud computing environments have been considered adequate to host voluminous databases, process analytical workloads and deliver database as a service, while spatial online analytical processing (spatial OLAP) queries issued over SDWs are intrinsically analytical. However, hosting a SDW in the cloud and processing spatial OLAP queries over such database impose novel obstacles. In this article, we introduce novel concepts as cloud SDW and spatial OLAP as a service, and afterwards detail the design of novel schemas for cloud SDW and spatial OLAP query processing over cloud SDW. Furthermore, we evaluate the performance to process spatial OLAP queries in cloud SDWs using our own query processor aided by a cloud spatial index. Moreover, we describe the cloud spatial bitmap index to improve the performance to process spatial OLAP queries in cloud SDWs, and assess it through an experimental evaluation. Results derived from our experiments revealed that such index was capable to reduce the query response time from 58.20 up to 98.89 %.     

Processing online analytics with classification and association rule mining

Business performance measurements, decision support systems (DSS) and online analytical processing (OLAP) have a common goal i.e., to assist decision-makers during the decision-making process. Integrating DSS and OLAP into existing business performance measurements hopes to improve the accuracy of analysis and provide in-depth, multi-angle view of data. This paper describes a decision support system containing our methodology, Weighted and Layered workflow evaluation (WaLwFA), extended to incorporate business intelligence using C4.5 and association rule algorithms. C4.5 produces more comprehensible decision trees by showing only important attributes. Furthermore, C4.5 can be transformed into IF-THEN rules. However, association rules are preferred as data can be described in rules of multiple granularities. Sorting rules based on rules’ complexities permits OLAP to navigate through layers of complexities to extract rules of relevant sizes and to view data from multidimensional perspectives in each layer. Experimental results on an airline domain are presented.     

基于社区划分的联机分析处理查询优化方案

针对P2P环境下的联机分析处理（OLAP）查询节点数目不断增加时,易造成网络拥塞、查询效率降低的问题,提出一种基于社区划分的OLAP查询优化方案。该方案构建一个虚拟的社区网,并在此结构上设计了一种基于社区划分的多维数据集(CPDS)的OLAP查询优化算法。实验结果表明,该算法能有效避免因网络节点数目递增而导致的网络负载加剧问题,能有效地减少网络拥塞,优化了OLAP的查询效率,进一步提高P2P环境下OLAP的决策分析性能。     

列存储中的OLAP多查询优化方法

为了使列存储OLAP(on-line analytical processing)操作中I/O和CPU开销较大的扫描、连接、聚集操作实现有效的共享和复用,提出了一个多查询优化技术。根据列存储以及OLAP操作的特点,提出了一系列转换规则,为OLAP查询请求产生的一组相关查询语句生成一个单一全局查询计划。为了达到共享复用的目的,在全局计划中引入新的过滤结点、分组结点、合并结点和聚集结点。同时,借用MuGA(multiply group by algo-rithm)算法,通过分组结点、合并结点、连接结点实现维表及事实表元组的分组序号标记,从而实现列扫描、列连接的共享。并为聚集结点提出了一个多阶段聚集算法,结合最终生成的事实表复合分组序号,实现聚集操作的复用。在SSB(star schema benchmark)数据集上设计实验,证明了该多查询优化策略的有效性。     

基于比较策略的嵌入式系统性能基准测试研究

嵌入式系统受成本、功耗、芯片体积和开发周期等多种因素的制约,其性能往往难以满足应用需求,充分发挥嵌入式系统的潜能对嵌入式系统设计尤为重要。为此,需要对嵌入式系统进行性能基准测试,指导嵌入式系统设计的技术选型和决策,在满足应用需求的前提下,达到最佳性能/价格比。该文分析了嵌入式系统性能基准测试的基本原理和性能指标,介绍了几种常用的嵌入式系统性能评测基准及其适用范围,提出了一种运用对比策略进行嵌入式系统性能测试的测试方法和测试环境的构成,实现了一种嵌入式性能测试工具,并对相关的嵌入式系统及构件进行了性能测试。最后对嵌入式系统性能基准测试研究存在的主要问题进行了评价。     

大数据分析的分布式MOLAP技术

大数据的规模效应给数据存储、管理以及数据分析带来了极大的挑战,学界和业界广泛采用分布式文件系统和MapReduce编程模型来应对这一挑战.提出了大数据环境中一种基于Hadoop分布式文件系统（HDFS）和MapReduce编程模型的分布式MOLAP技术,称为DOLAP（distributed OLAP）.DOLAP采用一种特殊的多维模型完成维和度量的映射;采用维编码和遍历算法实现维层次上的上卷下钻操作;采用数据分块和线性化算法将维和度量保存在分布式文件系统中;采用数据块选择算法优化OLAP的性能;采用MapReduce编程模型实现OLAP操作.描述了DOLAP在科学数据分析的应用案例,并与主流的非关系数据库系统进行性能对比.实验结果表明,尽管数据装载性能略显不足,但DOLAP的性能要优于基于HBase,Hive,HadoopDB,OLAP4Cloud等主流非关系数据库系统实现的OLAP性能.     

Empirical performance evaluation of graphics recognition systems

Presents a methodology for evaluating graphics recognition systems operating on images that contain straight lines, circles, circular arcs, and text blocks. It enables an empirical comparison of vectorization software packages and uses practical performance evaluation methods that can be applied to complete vectorization systems. The methodology includes a set of matching criteria for pairs of graphical entities, a set of performance evaluation metrics, and a benchmark for the evaluation of graphics recognition systems. The benchmark was tested on three systems. The results are reported and analyzed in the paper     

A top-down approach for compressing data cubes under the simultaneous evaluation of multiple hierarchical range queries

A novel top-down compression technique for data cubes is introduced and experimentally assessed in this paper. This technique considers the previously unrecognized case in which multiple Hierarchical Range Queries (HRQ), a very useful class of OLAP queries, must be evaluated against the target data cube simultaneously. This scenario makes traditional data cube compression techniques ineffective, as, contrary to the aim of our work, these techniques take into consideration one constraint only (e.g., a given storage space bound). The result of our study consists in introducing an innovative multiple-objective OLAP computational paradigm, and a hierarchical multidimensional histogram, whose main benefit is meaningfully implementing an intermediate compression of the input data cube able to simultaneously accommodate an even large family of different-in-nature HRQ. A complementary contribution of our work is represented by a wide experimental evaluation of the performance of our technique against both benchmark and real-life data cubes, also in comparison with state-of-the-art histogram-based compression techniques.     

Application of OLAP to a PDM database for interactive performance evaluation of in-progress product development

This paper applies online analytic processing (OLAP), a widely accepted database analysis technique, to a product data management (PDM) database to evaluate the performance of in-progress product development. This paper introduces a set of processing key performance indicators (KPIs) that can measure ongoing product development. To convert and analyze operational data in a PDM database with interactive OLAP operations, this study proposes a multidimensional data model specified by product facts with associated dimensions. The model is implemented using a commercial OLAP engine, and applied to a database supported by a prototype PDM system. The OLAP engine allows analysts to interactively evaluate the performance of in-progress product development in a multidimensional data space. This is a far more flexible and efficient approach than other result-oriented static evaluation approaches.     

Development of an Intelligent Data-Mining System for a Dispersed Manufacturing Network

Recent advances related to on-line analytical processing (OLAP) have resulted in a significant improvement in data analysis efficiency by virtue of its multidimensional database structure and pre-computing operations of measuring data. However, the research related to the design and implementation of OLAP, particularly in the support of dispersed manufacturing networks in terms of 'intelligent decision making', has yet to be considered as remarkable. Research studies indicate that the level of intelligence of decision support systems can be enhanced with the incorporation of computational intelligence techniques such as case-based reasoning or rule-based reasoning. This paper describes the development of an intelligent data-mining system using a rule-based OLAP approach which can be adopted to support dispersed manufacturing networks in terms of performance enhancement. In this paper, the techniques, methods and infrastructure for the development of such a data-mining system, which possesses certain intelligent features, are presented. To validate the feasibility of this approach, a case example related to the testing of the approach in an emulated industrial environment is covered.