一种OLAP应用系统的设计和实现

通过对数据仓库和OLAP概念及体系结构的分析，描述了一种OLAP应用系统的设计方案，并介绍了它的具体实现方法。基于数据仓库的查询，一般都是及时特定查询，要在严格的响应时间内执行复杂的查询，遍历百万上亿的记录，同时进行可能很复杂的搜索、连接和汇总的操作。查询的数据吞吐量和响应时间是判断数据仓库性能的重点。CUBE的计算是OLAP及时查询的基础，提高查询的速度需要对OLAP进行预先的计算。文中系统比较了一些计算立方体的算法，并运用到具体的系统当中。     

动态更新实物化视图以提高OLAP查询效率

在数据仓库系统中,OLAP查询一般都涉及多表连接和分组聚集两部分操作,提高这些查询的性能成为提高OLAP响应速度的关键。利用实物化视图,可以准确地计算并保存表连接或聚集等耗时较多的操作的结果。研究基于查询频率的实物化视图的更新算法,可以使实物化视图得到最大效率的使用,明显地缩短查询的响应时间,从而提高OLAP的查询效率。     

查询代价视图选择问题的研究及实现

实视图选择问题是数据仓库研究的重要问题之一。数据仓库存储实视图主要为OLAP查询,用户查询响应时间是首要考虑的问题,提出了查询代价视图选择问题,给出了其代价模型。提出了对查询代价视图选择问题利用遗传算法来解决的方法和策略。经实验证明,该算法达到了良好的效果,效率高。     

在Informix上实现移动网管的联机分析处理

联机分析处理（LOAP）用于对企业大量数据进行分析,以支持复杂查询和决策。分析比较了联机分析处理系统的两种不同的体系结构,介绍了基于informix MetaCube的OLAP思想并给出了一个针对移动通信网络管理的基于数据仓库的OLAP实例。     

数据仓库中的一种立方体数据模型

数据仓库和联机分析处理(OLAP)是商业数据处理领域中的两个最重大的新技术。OLAP应用要求对数据仓库中存储的大量数据进行分析,用标准关系数据库技术来实现非常复杂的查询是相当困难的。所以,在数据仓库中,数据被组织成立方体数据模型。该文提出了一种简单、直观的数据立方体模型以及在这个立方体上支持OLAP操作的代数。为复杂的查询提供了简要的表述方法。     

基于关系数据库的ROLAP中实视图的选择

联机分析处理（OLAP）是伴随着数据仓库出现的一种数据分析处理技术,其特点是使分析人员能够更充分地利用数据仓库中的数据资源,从多种角度、多个层次,快速地构建易为用户理解的并全面反映企业行为特征的数据快照,从而可使用户更加深入地了解企业的发展状况和趋势。ROLAP是OLAP中使用最广泛的一种类型。文中对影响ROLAP查询效率的关键技术进行了讨论,提出了一个改进的实视图动态选择算法。该算法从存储空间、查询频率、更新代价三个方面综合评价每个实视图,有效地保证了ROLAP查询的响应时间。     

基于动态镜像的实时数据仓库存取预处理技术研究

实时数据仓库是数据仓库技术的重要分支,而实时数据查询和实时数据导入引发的查询竞争问题一直是实时数据仓库技术研究的重点之一。查询竞争问题严重影响了查询分析的精度和效率,还降低了数据仓库的性能。提出了一种在数据仓库外部构建动态存储区域的方法,它采用动态镜像技术,有效地缓解查询竞争问题。同时,为了提高实时OLAP上的查询分析操作的性能,提出了蝇量级物化方法及蝇量级物化下的表连接算法FWMJoin(Fly-Weight Materialization Join)。基于TPC-H基准的实时数据仓库测试系统,针对动态镜像技术下的动态存储区域的OLAP性能进行分析与评估,并对实验结果进行总结。     

一种负载敏感的OLAP查询结果缓存管理技术

OLAP(On Line Analysis Processing)是数据仓库的典型应用,在数据仓库中频繁并发地执行涉及较大数据量的OLAP查询时,其查询处理效率易于逐渐降低。缓存技术是一种有效降低OLAP查询处理延时的方法。在现有的缓存数据存储、淘汰策略等研究工作的基础上,结合OLAP任务的负载特性、OLAP任务的结果集大小等因素对性能的影响,提出了一种负载敏感的OLAP查询缓存管理技术Workload-LRU,并实现了一个ROLAP(Relational OLAP)原型系统。实验证明,Workload-LRU技术获得了较好的性能提升效果。     

基于关系数据库的ROLAP中实视图的选择

联机分析处理(OLAP)是伴随着数据仓库出现的一种数据分析处理技术,其特点是使分析人员能够更充分地利用数据仓库中的数据资源,从多种角度、多个层次,快速地构建易为用户理解的并全面反映企业行为特征的数据快照,从而可使用户更加深入地了解企业的发展状况和趋势。ROLAP是OLAP中使用最广泛的一种类型。文中对影响ROLAP查询效率的关键技术进行了讨论,提出了一个改进的实视图动态选择算法。该算法从存储空间、查询频率、更新代价三个方面综合评价每个实视图,有效地保证了ROLAP查询的响应时间。     

OLAP系统基于查询结构的用户浏览引导

联机分析处理（OLAP）系统是数据仓库主要的前端支持工具,在OLAP系统中用户以浏览的方式进行数据访问。通常,OLAP系统用户一般会有相对稳定的信息需求,而OLAP系统中查询的结构一定程度上反映了用户所关心的信息内容,因此,用户执行查询的结构也会保持一定的稳定性。以查询结构为基础,对OLAP系统用户的查询行为进行了分析,提出了一种建立OLAP系统用户轮廓文件的方法,并对如何根据轮廓文件对用户的行为进行预测,并进一步对用户的浏览进行引导的方法进行了探讨。以此为基础,当OLAP系统用户进行信息浏览时,可以在OLAP系统前端,对用户可能感兴趣的地方做出一定的标识,引导用户将要进行的浏览动作,使用户能轻松的完成信息搜索的工作。     

dmGQL：一种新的数据立方梯度查询语言

现有数据立方梯度查询语言CubegradeQL主要是针对非实例化数据立方的，实际上，为了提高OLAP查询效率，数据仓库中往往保存了大量实例化的数据立方。本文我们改进了CubegradeQL语言，给出了一个新的查询语言dmGQL，dmGQL能够支持实例化／非实例化数据立方中的梯度查询，最后，我们讨论了dmGQL的查询处理。     

Compressed Data Cube for Approximate OLAP Query Processing

Approximate query processing has emerged as an approach to dealing with the huge data volume and complex queries in the environment of data warehouse.In this paper,we present a novel method that provides approximate answers to OLAP queries.Our method is based on building a compressed (approximate) data cube by a clustering technique and using this compressed data cube to provide answers to queries directly,so it improves the performance of the queries.We also provide the algorithm of the OLAP queries and the confidence intervals of query results.An extensive experimental study with the OLAP council benchmark shows the effectiveness and scalability of our cluster-based approach compared to sampling.     

Optimization in Data Cube System Design

The design of an OLAP system for supporting real-time queries is one of the major research issues. One approach is to use data cubes, which are materialized precomputed multidimensional views of data in a data warehouse. We can derive a set of data cubes to answer each frequently asked query directly. However, there are two practical problems: (1) the maintenance cost of the data cubes, and (2) the query cost to answer those queries. Maintaining a data cube requires disk storage and CPU computation, so the maintenance cost is related to the total size as well as the total number of data cubes materialized. In most cases, materializing all data cubes is impractical. The maintenance cost may be reduced by merging some data cubes. However, the resulting larger data cubes will increase the query cost of answering some queries. If the bounds on the maintenance cost and the query cost are too strict, we help the user decide which queries to be sacrificed and not taken into consideration. We have defined an optimization problem in data cube system design. Given a maintenance-cost bound, a query-cost bound and a set of frequently asked queries, it is necessary to determine a set of data cubes such that the system can answer a largest subset of the queries without violating the two bounds. This is an NP-hard problem. We propose approximate Greedy algorithms GR, 2GM and 2GMM, which are shown to be both effective and efficient by experiments done on a census data set and a forest-cover-type data set.     

Approximate Query Processing in Cube Streams

Data cubes have become important components in most data warehouse systems and decision support systems. In such systems, users usually pose very complex queries to the online analytical processing (OLAP) system, and systems usually have to deal with a huge amount of data because of the large dimensionality of the sets; thus, approximating query processing has emerged as a viable solution. Specifically, the applications of cube streams handle multidimensional data sets in a continuous manner in contrast to the traditional cube approximation. Such an application collects data events for cube streams online, generates snapshots with limited resources, and keeps the approximated information in a synopsis memory for further analysis. Compared to the OLAP applications, applications of cube streams are subject to many more resource constraints on both the processing time and the memory and cannot be dealt with by existing methods due to the limited resources. In this paper, we propose the DAWA algorithm, which is a hybrid algorithm of discrete cosine transform (DCT) for data and the discrete wavelet transform (DWT), to approximate cube streams. Our algorithm combines the advantages of the high compression rate of DWT and the low memory cost of DCT. Consequently, DAWA requires much smaller working buffer and outperforms both DWT-based and DCT-based methods in execution efficiency. Also, it is shown that DAWA provides a good solution for an approximate query processing of cube streams with a small working buffer and a short execution time. The optimality of the DAWA algorithm is theoretically proved and empirically demonstrated by our experiments.     

数据方体系统设计中的优化问题

支持实时查询的联机分析处理系统的设计是当前一个很重要的研究问题。其中常用的方法是使用数据方体来实现。对于出现频率较高的查询,可以给出对应的数据方体集,使得每个查询都可以直接得到回答。但是在设计基于方体的系统时,需要考虑以下两个问题:(1)数据方体的维护成本,(2)回答频繁查询的响应时间。在用户给出了维护成本上限和响应时间上限后,需要对数据方体集进行优化,使得系统能够满足用户的要求,并回答尽可能多的查询。文章给出了数据方体系统设计优化问题的定义,这是一个NP完全问题,并提出了贪心删除和贪心合并的近似算法。实验表明了算法的有效性。     

用于聚集值近似查询的基于密度的树索引结构

如何快速有效地对数据立方体上的聚集查询给出近似的回答,是数据挖掘和数据仓库研究领域中的核心问题之一。现有大多数聚集查询算法在同一个数据立方体上只能支持某种特定的而非多种类型的聚集查询。本文给出了一种新的框架AdenTS,即基于密度的自适应树结构,它可以回答同一数据立方体上的各类聚集查询,也提出了一些近似和启发式技术,改善了查询结果和精度。实验结果表明,这种方法在支持的查询种类和性能上是更好的。     

Policies for Caching OLAP Queries in Internet Proxies

The Internet now offers more than just simple information to the users. Decision makers can now issue analytical, as opposed to transactional, queries that involve massive data (such as, aggregations of millions of rows in a relational database) in order to identify useful trends and patterns. Such queries are often referred to as On-Line-Analytical Processing (OLAP). Typically, pages carrying query results do not exhibit temporal locality and, therefore, are not considered for caching at Internet proxies. In OLAP processing, this is a major problem as the cost of these queries is significantly larger than that of the transactional queries. This paper proposes a technique to reduce the response time for OLAP queries originating from geographically distributed private LANs and issued through the Web toward a central data warehouse (DW) of an enterprise. An active caching scheme is introduced that enables the LAN proxies to cache some parts of the data, together with the semantics of the DW, in order to process queries and construct the resulting pages. OLAP queries arriving at the proxy are either satisfied locally or from the DW, depending on the relative access costs. We formulate a cost model for characterizing the respective latencies, taking into consideration the combined effects of both common Web access and query processing. We propose a cache admittance and replacement algorithm that operates on a hybrid Web-OLAP input, outperforming both pure-Web and pure-OLAP caching schemes.     

一种P2P网络环境下的OLAP查询方案

传统网络环境和P2P环境中,客户端向OLAP服务器提交OLAP查询,并从服务器获取查询结果,OLAP服务器的负载将随着客户端的增加而急剧增加。设计了一种基于P2P（Peer-to-Peer,点对点技术）技术的DQDC（Distributed Query Data Cube,多维数据集的分布式查询）算法,实现P2P网络中语义级的多节点Data Cube数据共享,从而提高系统整体的决策分析性能。     

一种高度浓缩和语义保持的数据立方

Quotient Cube和QC-tree试图在浓缩一个数据立方尺寸的同时,保持该数据立方蕴涵的语义,但是,前者没有语义关系的存储,后者存储的语义关系是晦涩模糊的.为此提出了下钻立方结构,首次从语义角度考虑数据立方存储,存储的不是类的内容,而是类之间的直接下钻关系.下钻立方不仅能够极大地减小数据立方的存储尺寸,而且可以清晰地表达原数据立方蕴涵的下钻语义.此外,下钻立方具有较高的查询响应性能,这一点在范围查询中表现得尤其显著.实验和分析表明,下钻立方在存储尺寸和查询响应方面明显优于QC-tree,适于用来组织和存储数据立方.