动态更新实物化视图以提高OLAP查询效率

在数据仓库系统中,OLAP查询一般都涉及多表连接和分组聚集两部分操作,提高这些查询的性能成为提高OLAP响应速度的关键。利用实物化视图,可以准确地计算并保存表连接或聚集等耗时较多的操作的结果。研究基于查询频率的实物化视图的更新算法,可以使实物化视图得到最大效率的使用,明显地缩短查询的响应时间,从而提高OLAP的查询效率。     

基于语义分解的联机分析处理查询并行优化方案

利用联机分析处理(OLAP)查询中存在的语义关联,对聚集关系与语义分解关系进行了形式化描述,并基于这些关系定义了查询与查询集之间的补集关系,在执行OLAP查询集时,可以利用这些关系尽可能地识别查询集中查询的公共部分,并且可以在查询时从多个角度来采取并行优化措施。实验验证表明采用并行优化方案后,系统的整体效率得到了提高。     

Finding an efficient rewriting of OLAP queries using materialized views in data warehouses

OLAP queries involve a lot of aggregations on a large amount of data in data warehouses. To process expensive OLAP queries efficiently, we propose a new method to rewrite a given OLAP query using various kinds of materialized views which already exist in data warehouses. We first define the normal forms of OLAP queries and materialized views based on the selection and aggregation granularities, which are derived from the lattice of dimension hierarchies. Conditions for usability of materialized views in rewriting a given query are specified by relationships between the components of their normal forms. We present a rewriting algorithm for OLAP queries that can effectively utilize materialized views having different selection granularities, selection regions, and aggregation granularities together. We also propose an algorithm to find a set of materialized views that results in a rewritten query which can be executed efficiently. We show the effectiveness and performance of the algorithm experimentally.     

数据流管理系统中适应性查询机制的研究

介绍了数据流技术的发展现状,然后讨论了适应性查询在数据管理中的发展演变,特别是在数据流管理中的特殊性。最后,在此基础上,提出了一个支持适应性查询的数据流管理系统RealStream,并详细介绍了其适应性查询处理机制。     

无线传感器中数据管理技术的探讨

无线传感器网络是一门获取和处理信息的新兴技术,它以数据为中心,提供数据采集、处理和查询功能,其根本任务是准确获取物理世界的有价值信息。数据存储和数据查询是无线传感器网络研究中的重点和热点问题。本文探讨了无线传感器网络的数据存储与查询技术。     

数据流管理系统研究与进展

综述了数据流管理系统的研究现状及相关的技术,包括相关的基本概念的阐述、现有实验系统的回顾、流式查询中存在的问题及相关的解决方案,并就今后如何进行数据流管理系统的研究提出了一些新的看法。     

一个面向OLAP应用的多维数据查询语言及其在对象关系数据库中的实现

该文提出了一个面向联机分析处理(OLAP)的多维查询语言,为多维查询的方便表达提供了一个直观的高层界面。这一查询语言的独特之处在于能够很好地支持具有继承关系的维层次结构,这种维层次结构在传统的关系型的ROLAP中是难于用星型/雪花模式表示的。文章将这一查询语言实现于对象关系数据库之上。并通过实例进行说明,对象关系数据库所特有的基本类型扩展能力、复杂对象的表示以及继承机制,使得对多维数据的描述更加灵活、多维操作更为高效。     

A cubic-wise balance approach for privacy preservation in data cubes

A data warehouse stores current and historical records consolidated from multiple transactional systems. Securing data warehouses is of ever-increasing interest, especially considering areas where data are sold in pieces to third parties for data mining practices. In this case, existing data warehouse security techniques, such as data access control, may not be easy to enforce and can be ineffective. Instead, this paper proposes a data perturbation based approach, called the cubic-wise balance method, to provide privacy preserving range queries on data cubes in a data warehouse. This approach is motivated by the following observation: analysts are usually interested in summary data rather than individual data values. Indeed, our approach can provide a closely estimated summary data for range queries without providing access to actual individual data values. As demonstrated by our experimental results on APB benchmark data set from the OLAP council, the cubic-wise balance method can achieve both better privacy preservation and better range query accuracy than random data perturbation alternatives.     

数据立方体选择的改进遗传算法

数据立方体选择问题是一个NP完全问题。研究了利用遗传算法来解决立方体选择问题,提出了一个结合局部搜索机制的遗传算法。这一算法的核心思想在于,首先运用一个基于单位空间最大收益值的预处理算法来生成初始解,然后该初始解经结合了局部搜索机制的遗传算法进行提高。实验结果表明,该算法在寻优性能上优于启发式算法和经典遗传算法。     

基于“C藤”Pair Copula的高维OLAP查询建模方法研究

信息爆炸造成的数据仓库维度的急剧增加,大大影响了OLAP查询模型的精度和效率.首次将数理统计学中的“C藤”Pair Copula引入到OLAP查询建模的研究中,有效地解决了高维OLAP查询建模时的“维数灾难”问题,并设计了针对该模型的参数估计方法以提取数据概要知识.实验分析表明与传统方法相比,基于Pair Copula方法的模型可以在保证OLAP的查询精度的基础上减少数据立方体的存储空间,并且在高维数据环境下具有更高的查询效率.     

数据流中一种适应性查询优化方法

对数据流中的查询处理机制进行了深入的研究,从内存使用量和查询的实时性两方面进行综合考虑,提出了一种基于多因素的动态查询优化及调度策略MultiFactor,它根据各操作符消耗系统中元组数量的快慢来动态调整操作符调度次序,按查询的截止时间来确定各操作符调度时间,同时提出了多流连接查询的调度方法。给出了Multi-Factor适应性优化的时机以及调整策略。     

Extending OCL for OLAP querying on conceptual multidimensional models of data warehouses

The development of data warehouses begins with the definition of multidimensional models at the conceptual level in order to structure data, which will facilitate decision makers with an easier data analysis. Current proposals for conceptual multidimensional modelling focus on the design of static data warehouse structures, but few approaches model the queries which the data warehouse should support by means of OLAP (on-line analytical processing) tools. OLAP queries are, therefore, only defined once the rest of the data warehouse has been implemented, which prevents designers from verifying from the very beginning of the development whether the decision maker will be able to obtain the required information from the data warehouse. This article presents a solution to this drawback consisting of an extension to the object constraint language (OCL), which has been developed to include a set of predefined OLAP operators. These operators can be used to define platform-independent OLAP queries as a part of the specification of the data warehouse conceptual multidimensional model. Furthermore, OLAP tools require the implementation of queries to assure performance optimisations based on pre-aggregation. It is interesting to note that the OLAP queries defined by our approach can be automatically implemented in the rest of the data warehouse, in a coherent and integrated manner. This implementation is supported by a code-generation architecture aligned with model-driven technologies, in particular the MDA (model-driven architecture) proposal. Finally, our proposal has been validated by means of a set of sample data sets from a well-known case study.     

基于WEB的数据仓库技术在电信网络管理系统中的研究

本文讨论了WEB数据仓库的体系结构、OLAP技术及数据挖掘等相关数据仓库技术,并描述了当前电信网络管理系统的现状及存在问题,以存在问题为出发点,提出了创建以WEB数据仓库技术为基础的综合网管系统,为网管存在问题提供解决方案,它的建立对网管现状的改善有积极意义。     

Answering ad hoc aggregate queries from data streams using prefix aggregate trees

In some business applications such as trading management in financial institutions, it is required to accurately answer ad hoc aggregate queries over data streams. Materializing and incrementally maintaining a full data cube or even its compression or approximation over a data stream is often computationally prohibitive. On the other hand, although previous studies proposed approximate methods for continuous aggregate queries, they cannot provide accurate answers. In this paper, we develop a novel prefix aggregate tree (PAT) structure for online warehousing data streams and answering ad hoc aggregate queries. Often, a data stream can be partitioned into the historical segment, which is stored in a traditional data warehouse, and the transient segment, which can be stored in a PAT to answer ad hoc aggregate queries. The size of a PAT is linear in the size of the transient segment, and only one scan of the data stream is needed to create and incrementally maintain a PAT. Although the query answering using PAT costs more than the case of a fully materialized data cube, the query answering time is still kept linear in the size of the transient segment. Our extensive experimental results on both synthetic and real data sets illustrate the efficiency and the scalability of our design. Moonjung Cho is a Ph.D. candidate in the Department of Computer Science and Engineering at State University of New York at Buffalo. She obtained her Master from same university in 2003. She has industry experiences as associate researcher for 4 years. Her research interests are in the area of data mining, data warehousing and data cubing. She has received a full scholarship from Institute of Information Technology Assessment in Korea. Jian Pei received the Ph.D. degree in Computing Science from Simon Fraser University, Canada, in 2002. He is currently an Assistant Professor of Computing Science at Simon Fraser University, Canada. In 2002–2004, he was an Assistant Professor of Computer Science and Engineering at the State University of New York at Buffalo, USA. His research interests can be summarized as developing advanced data analysis techniques for emerging applications. Particularly, he is currently interested in various techniques of data mining, data warehousing, online analytical processing, and database systems, as well as their applications in bioinformatics. His current research is supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC) and National Science Foundation (NSF). He has published over 70 papers in refereed journals, conferences, and workshops, has served in the program committees of over 60 international conferences and workshops, and has been a reviewer for some leading academic journals. He is a member of the ACM, the ACM SIGMOD, and the ACM SIGKDD. Ke Wang received Ph.D from Georgia Institute of Technology. He is currently a professor at School of Computing Science, Simon Fraser University. Before joining Simon Fraser, he was an associate professor at National University of Singapore. He has taught in the areas of database and data mining. Ke Wang's research interests include database technology, data mining and knowledge discovery, machine learning, and emerging applications, with recent interests focusing on the end use of data mining. This includes explicitly modeling the business goal (such as profit mining, bio-mining and web mining) and exploiting user prior knowledge (such as extracting unexpected patterns and actionable knowledge). He is interested in combining the strengths of various fields such as database, statistics, machine learning and optimization to provide actionable solutions to real life problems. Ke Wang has published in database, information retrieval, and data mining conferences, including SIGMOD, SIGIR, PODS, VLDB, ICDE, EDBT, SIGKDD, SDM and ICDM. He is an associate editor of the IEEE TKDE journal and has served program committees for international conferences including DASFAA, ICDE, ICDM, PAKDD, PKDD, SIGKDD and VLDB.     

Wavelet synopsis for hierarchical range queries with workloads

Synopses structures and approximate query answering have become increasingly important in DSS/ OLAP applications with stringent response time requirements. Range queries are an important class of problems in this domain, and have a wide variety of applications and have been studied in the context of histograms. However, wavelets have been shown to be quite useful in several scenarios and in fact their multi-resolution structure makes them especially appealing for hierarchical domains. Furthermore the fact that the Haar wavelet basis has a linear time algorithm for the computation of coefficients has made the Haar basis one of the important and widely used synopsis structures. Very recently optimal algorithms were proposed for the wavelet synopsis construction problem for equality/point queries. In this paper we investigate the problem of optimum Haar wavelet synopsis construction for range queries with workloads. We provide optimum algorithms as well as approximation heuristics and demonstrate the effectiveness of these algorithms with our extensive experimental evaluation using synthetic and real-life data sets. Research was supported in part by the Alfred P. Sloan Research Fellowship and NSF awards CCF-0430376, CCF-0644119. Research was supported by the Ministry of Information and Communication, Korea, under the College Information Technology Research Center Support Program, grant number IITA-2006-C1090-0603-0031.     

数据流管理系统分析与研究

综述了数据流管理系统的研究现状及相关的技术,包括基本概念的阐述、流式查询中存在的问题及其解决方案,并就今后如何进行数据流管理系统的研究提出了一些新的看法。     

基于数据仓库技术的空中交通流量管理系统

论文提出了把信息技术应用于空域流量管理的设想,简单介绍了数据仓库与决策支持系统的特点,并结合清华大学CIMS中心与华北空管局合作研制的流量管理原型系统,描述了在数据仓库上构建决策支持系统的几个技术问题和解决方案。     

Partitioned optimization of complex queries

Performing complex analysis on top of massive data stores is essential to most modern enterprises and organizations and requires significant aggregation over different attribute sets (dimensions) of the participating relations. Such queries may take hours or days, a time period unacceptable in most cases. As a result, it is important to study these queries and identify special frequent cases that can be evaluated with specialized algorithms. Understanding complex aggregate queries leads to better execution plans and, consequently, performance.