数据仓库自维护下视图分解系统的设计与实现

数据仓库自维护实质上是通过维护实化视图实现,然而现有的实化视图自维护策略不能有效的减少数据仓库集成端和数据源监视端的多余数据,从而影响数据仓库环境的整体响应速度.一种基于数据仓库自维护方法的视图分解系统改进了现有的视图分解模式,将全局定义的实化视图分解成局部定义的单源视图集来减少存在数据仓库中不必要的数据,实现了现有实化视图自维护策略的分解和重写,提高数据仓库自维护效率.     

数据仓库中基于实体化辅助视图的视图增量维护

为了加快对大量数据的查询处理速度，通常在数据仓库以实视图方式存储数据，当基础数据发生变化时，这些实视图也必须随着更新，因而视图自维护和一致性维护成为数据仓库的重要问题。本文提出利用视图计算的中间结果创建辅助视图，在数据仓库中进行实体化，采用有效的增量维护算法计算实视图的精确变化，实现数据仓库视图自维护。     

基于源视图增量的在线实化视图自维护

借鉴传统的基于基表变化的数据仓库维护方法Strobe，提出一种基于源视图增量的在线实化视图自维护方法，使实化视图的状态保持与底层数据源的一致性。这种方法不仅保持数据仓库数据的一致性，而且还能够加快实化视图维护的速度，减少底层信息源与数据仓库之间的网络通信负担。     

数据仓库实化视图的联机维护

数据仓库实化视图和联机维护是数据仓库系统维护的一项关键技术,采用这种技术,能够在不影响用户正常业务的情况下,实现数据仓库的实化视图数据的及时更新、联机分析处理（OLAP）作为数据仓库的一个主要应用,在数据仓库实化视图的联机维护的过程中会面临严重的数据不一致问题。为了解决这个问题,本文引入“维护库”（Maintaining Database)的概念,提出基于事务触发的视图维护算法TVM,通过消息应答机制实现实化视图与数据源的数据一致性。     

数据仓库实化视图的联机维护

数据仓库实化视图的联机维护是数据仓库系统维护的一项关键技术,采用这种技术,能够在不影响用户正常业务的情况下,实现数据仓库中实化视图数据的及时更新。但联机分析处理（OLAP）作为数据仓库的一个主要应用,在数据仓库实化视图的联机维护过程中会产生严重的数据不 一致问题。为了解决这个问题,引入“维护库”（Maintaining Database)的概念,提出基于事务触发的视图维护算法TVM,采取应答机制,达到数据的一致性。     

数据仓库中多视图环境下的联机维护

数据仓库的视图联机维护是指数数据仓库中的实体化视图实时地与信息源中的数据库仑保持一致,同时不影响前端用户对数据仓库的正常使用。为了解决多视图环境中视图联机维护与下钻查询的一致性问题,文中在数据仓库体系结构中引入了“基库”模型,并提出了相应的视图维护算法３ＶＰＡ。     

基于多维护策略的物化视图选择方法

物化视图是数据仓库环境中提高OLAP查询效率的重要手段，因此，物化视图的选择是数据仓库设计中重要的决策之一。本文提出的物化视图选择方法目标是选择合适的视图进行物化，使得查询处理的总代价和物化视图的维护代价最低，提出了物化视图收益模型，并在此基础上基于视图的多维护策略提出了物化视图选择的方法：基于增量和重计算的物化视图选择算法IRMVS、基于增量策略的物化视图选择算法IMVS和基于重计算策略的物化视图选择算法RMVs和基于增量策略的物化后代视图选择算法IMDVS，理论分析和实验表明这些算法是有效可行的。     

数据仓库中实现图的建立与维护

数据仓库技术是分布式异构数据库系统集成的一种较为先进的解决方法,实视图是数据仓库中存储的主要信息实体。实视图不仅是数据仓库中的数据的基本组织方式,而且采用实现图来定义和存储一些经过抽取及综合计算的数据,将有利于提高数据仓库的查询性能。实视图的建立和更新维护是其实现的主要技术问题,本文提出的实现图增量维护法和实现图版本链控制法,可以较好地满足不同种类实视图的实时更新维护。     

基于多实化视图增量维护的基库生成算法

数据仓库的维护是数据仓库应用中的一个十分重要的问题,近几年产生了很多的维护算法。已有的维护算法多是针对单个实化视图的维护;或只针对简单SPJ视图的维护;或只针对聚集函数的维护;而实际的数据仓库大多是由包含聚集函数的多个实化视图组成。因此综合考虑包含聚集函数的多个实化视图的维护问题是必然的。文章正是在此情况下提出了一种基于多实化视图增量维护的基库生成算法,在《基于基库的多实化视图增量维护算法》中提出了包含聚集函数的多实化视图的维护算法。     

数据仓库环境下的实化视图维护

本文通过示例说明数据仓库环境下实化视图维护存在的数据一致性问题，并分析了产生这一问题的根本原因．文中介绍了一些能解决数据一致性问题的具有代表性的实化视图维护算法，比较了它们之间的差异，最后描述了数据仓库环境下数据一致性程度的四个层次。     

数据仓库联机维护技术的研究与实现

针对数据仓库联机维护技术提出了一种三层维护体系结构TMA,在其中引进了“数据仓库基库”概念,利用版本控制思想提出了对单视图和多视图的联机维护算法,并实现了一个原型验证系统。     

基于网络的数据仓库动态增量维护算法与模型

提出数据仓库动态增量维护算法和模型．文中阐述了动态增量维护算法、模型以及利用该算法对数据仓库视图的维护技术，并以基于网络的数据仓库为例，描述了动态增量维护算法在数据仓库系统中的实现技术．本算法与技术对数据仓库技术的发展及应用有着重要的理论意义和实用价值。     

数据仓库相关技术研究综述

该文给出了数据仓库的定义,通过对相关文献的研究,给出了普通数据仓库和空间数据仓库的构建方法,研究了数据仓库的关键技术,包括数据仓库的粒度、查询、维护、集成等方法,文章还就数据仓库的应用进行了深入的研究,最后进行了总结与展望。     

一种改进的分布式ETL体系结构

在分析了分布式数据仓库数据一致性维护的重要性,以及目前分布式ETL(Extract,transform and load)中存在问题的基础上,针对传统ETL体系结构对分布式数据仓库一致性维护的不足,提出一种新的分布式ETL的体系结构ETLM,并详细描述了数据一致性维护模块(M模块)的设计与实现。ETLM的体系结构免去了分布式数据仓库需要专门进行一致性维护的额外负担,可以更加正确、快捷、高效地支持OLAP。     

Multiagent immediate incremental view maintenance for data warehouses

Data warehouse systems typically designate downtime for view maintenance, ranging from tens of minutes to hours depending on the system size. We develop a multiagent system that achieves immediate incremental view maintenance (IIVM) for continuous updating of data warehouse views. We describe an IIVM system that processes updates as transactions are executed at the underlying data sources to eliminate view maintenance downtime for the data warehouse-a crucial requirement for internet applications. The use of a multiagent framework provides considerable process speed improvement when compared with other IIVM systems. Since agents are used to delegate the data sources and warehouse views, it is easy to reorganize the components of the system. Through the use of cooperative agents, the data consistency of IIVM can be easily maintained. The test results from this research show that the proposed system increases the availability of the data warehouse while preserving a stringent requirement on data consistency.     

Parallel multisource view maintenance

In a distributed environment, materialized views are used to integrate data from different information sources and then store them in some centralized location. In order to maintain such materialized views, maintenance queries need to be sent to information sources by the data warehouse management system. Due to the independence of the information sources and the data warehouse, concurrency issues are raised between the maintenance queries and the local update transactions at each information source. Recent solutions such as ECA and Strobe tackle such concurrent maintenance, however with the requirement of quiescence of the information sources. SWEEP and POSSE overcome this limitation by decomposing the global maintenance query into smaller subqueries to be sent to every information source and then performing conflict correction locally at the data warehouse. Note that all these previous approaches handle the data updates one at a time. Hence either some of the information sources or the data warehouse is likely to be idle during most of the maintenance process. In this paper, we propose that a set of updates should be maintained in parallel by several concurrent maintenance processes so that both the information sources as well as the warehouse would be utilized more fully throughout the maintenance process. This parallelism should then improve the overall maintenance performance. For this we have developed a parallel view maintenance algorithm, called PVM, that substantially improves upon the performance of previous maintenance approaches by handling a set of data updates at the same time. The parallel handling of a set of updates is orthogonal to the particular maintenance algorithm applied to the handling of each individual update. In order to perform parallel view maintenance, we have identified two critical issues that must be overcome: (1) detecting maintenance-concurrent data updates in a parallel mode and (2) correcting the problem that the data warehouse commit order may not correspond to the data warehouse update processing order due to parallel maintenance handling. In this work, we provide solutions to both issues. For the former, we insert a middle-layer timestamp assignment module for detecting maintenance-concurrent data updates without requiring any global clock synchronization. For the latter, we introduce the negative counter concept to solve the problem of variant orders of committing effects of data updates to the data warehouse. We provide a proof of the correctness of PVM that guarantees that our strategy indeed generates the correct final data warehouse state. We have implemented both SWEEP and PVM in our EVE data warehousing system. Our performance study demonstrates that a manyfold performance improvement is achieved by PVM over SWEEP.Received: 12 November 2001, Accepted: 18 December 2002, Published online: 31 July 2003This work was supported in part by the NSF NYI grant IIS-979624 and NSF CISE Instrumentation grant IRIS 97-29878 and NSF grant IIS-9988776.     

Efficient approaches for materialized views selection in a data warehouse

View materialization is an effective method to increase query efficiency in a data warehouse and improve OLAP query performance. However, one encounters the problem of space insufficiency if all possible views are materialized in advance. Reducing query time by means of selecting a proper set of materialized views with a lower cost is crucial for efficient data warehousing. In addition, the costs of data warehouse creation, query, and maintenance have to be taken into account while views are materialized. In this paper, we propose efficient algorithms to select a proper set of materialized views, constrained by storage and cost considerations, to help speed up the entire data warehousing process. We derive a cost model for data warehouse query and maintenance as well as efficient view selection algorithms that effectively exploit the gain and loss metrics. The main contribution of our paper is to speed up the selection process of materialized views. Concurrently, this will greatly reduce the overall cost of data warehouse query and maintenance.     

数据仓库系统中的元数据管理

人们对数据分析的要求的不断提高导致了数据仓库的发展,而在建设数据仓库的过程中元数据管理起着至关重要的作用。详细而准确的元数据对于数据仓库的创建、数据加载、运行维护、清理脏数据等工作都必不可少。文章在对数据仓库系统中的元数据以及元数据管理进行全面分析的基础上,较为详细地介绍了笔者自行开发的数据仓库系统SEUwarehouse中的元数据及其管理的设计与实现。