数据仓库中物化视图选择策略

为了提高决策支持和OLAP查询的响应效率,数据仓库多采用物化视图的思想.因此,物化视图的选择策略是数据仓库研究的重要问题之一.其目标是选出一组存储、维护代价与查询代价的总和为最小的物化视图.提出一个以MVPP(multi-view processing plan)为视图选择的搜索空间的物化视图选择新算法--VSMF(views selection base on multi-factor)算法.该算法在存储空间约束下同时实现多查询最优化和视图维护最优化.     

数据仓库中物化视图选择的一种混合算法

物化视图是数据仓库中提高查询效率的有效方法,物化视图选择问题是数据仓库设计时期最重要的决定之一。通过研究和实验,提出了一种结合遗传算法和模拟退火算法的混合算法,用于解决物化视图的选择。理论分析和实验结果表明,该混合算法的搜索性能优于传统的遗传算法,能够提供更高质量的解。     

Efficient approaches for materialized views selection in a data warehouse

View materialization is an effective method to increase query efficiency in a data warehouse and improve OLAP query performance. However, one encounters the problem of space insufficiency if all possible views are materialized in advance. Reducing query time by means of selecting a proper set of materialized views with a lower cost is crucial for efficient data warehousing. In addition, the costs of data warehouse creation, query, and maintenance have to be taken into account while views are materialized. In this paper, we propose efficient algorithms to select a proper set of materialized views, constrained by storage and cost considerations, to help speed up the entire data warehousing process. We derive a cost model for data warehouse query and maintenance as well as efficient view selection algorithms that effectively exploit the gain and loss metrics. The main contribution of our paper is to speed up the selection process of materialized views. Concurrently, this will greatly reduce the overall cost of data warehouse query and maintenance.     

XML materialized views and schema evolution in VIREX

Web-based databases are gaining increased popularity. This has positively influenced the availability of structured and semi-structured databases for access by a variety of users ranging from professionals to naive users. The number of users accessing online databases will continue to increase if the visual tools connected to web-based databases are flexible and user-friendly enough to meet the expectations of naive users and professionals. Further, XML is accepted as the standard for platform independent data exchange. This motivated for the development of the conversion tools between structured databases and XML. Realizing that such a need has not been well handled by the available tools, including Clio from IBM, we developed VIREX as a visual tool for converting relational databases into XML, and since then has been empowered with further capabilities to manipulate the produced XML schema including the maintenance of materialized views and schema evolution functions. VIREX provides an interactive approach for querying and integrating relational databases to produce XML documents and the corresponding XML schema(s). VIREX supports VRXQuery as a visual naive users-oriented query language that allows users to specify queries and define views directly on the interactive diagram as a sequence of mouse clicks with minimum keyboard input. As the query result, VIREX displays on the screen the XML schema that satisfies the specified characteristics and generates colored (easy to read) XML document(s). The main contribution described in this paper is the novel approach for turning query results into materialized views which are maintained to remain consistent with the underlying database. VIREX supports deferred update of XML views by keeping an ordered summary of the necessary and sufficient information required for the process. Each view has a corresponding marker in the ordered summary to indicate the start of the information to be reflected onto the view when it is accessed. When a view is accessed, its marker moves to the head of the list to mark for the next update. In addition, VIREX supports some basic schema evolution functions include renaming, adding and dropping of elements and attributes, among others. The supported schema evolution functions add flexibility to the view maintenance and materialization process.     

Maintaining materialized views without accessing base data

Access to a database through a user view can be serviced quickly when the view is materialized, i.e. the transformed data is explicitly stored. In the presence of database updates, however, the materialized view can become costly to maintain; often it must be completely rederived from the base data using the view definition. Under some conditions the view can be updated directly given only the view definition, the current contents of the materialized view, and the update operation (still expressed against the base data), without accessing the base data itself. In this paper, we consider relational views defined by projection, selection, and join. We present necessary and sufficient conditions on the view definition, contents, and update operations for insertions and deletions to be reflected in the view without reference to base data. Because the possibility of such view-based updating is dependent on the current contents of view, we call the update conditionally autonomously computable.     

数据仓库中新型动态实视图选择调整算法

现有的静态实视图选择算法存在搜索空间太大、时间复杂度高以及未考虑查询的概率和分布等诸多缺点,并且当源数据发生变化时,这种变化不能立刻反映到数据仓库,不适合在线运行。针对上述问题在候选视图生成算法和IGA算法的基础上,对算法进行了动态调整,从而得出了新型物化视图动态调整算法CNUMV。经实验证明该算法降低了视图的搜索空间和时间复杂度,更重要的是该算法考虑到了各视图之间相互依赖关系对视图收益的影响,从而使算法能够动态地在线调整,并且用实验证明了CNUMV算法的优越性,达到了预期的目的。     

Applying evolutionary algorithms to materialized view selection in a data warehouse

Effective analysis of genome sequences and associated functional data requires access to many different kinds of biological information. A data warehouse [14,16] plays an important role for storage and analysis for genome sequence and functional data. A data warehouse stores lots of materialized views to provide an efficient decision-support or OLAP queries. The view-selection problem addresses to select a fittest set of materialized views from a variety of MVPPs 0 forms a challenge in data warehouse research. In this paper, we present genetic algorithm to choose materialized views. We also use experiments to demonstrate the power of our approach. We would like to thank the authors, i.e. J. Yang, K. Karlapalem, and Q. Li, of the paper [15]. In this study, we borrow their mathematical model of the work in [15].     

Construction and distribution of materialized views in Non-binary data space

Innovations in Systems and Software Engineering - Materialized views are heavily used to speed up the query response time of any data centric application. In the literature, the construction and...     

Updating distributed materialized views

The problem of updating materialized views in distributed database systems is discussed. An architecture and detailed procedures for updating a collection of remote views with arbitrary refresh times by using a single differential file are described. The efficiency of the update procedure is enhanced by adopting s multiquery optimization approach and by introducing a powerful prescreening procedure to eliminate differential tuples. It is shown that even for a single remote view, there are many instances where the update procedure performs better (with respect to total I/O and communication costs) than a base table approach     

一种实化视图的合并算法

对于拥有大量实化视图的实际数据库应用系统,提出了视图合并的方法以减少整个视图 的数量,缩减实化视图的搜索空间;还提出了归并树和基于归并树的快速有效的合并算法。实验表 明,实化视图的合并是快速寻找可能响应查询的实化视图的一种有效途径,可以显著改进查询处理的 性能。     

Selection of views to materialize in a data warehouse

A data warehouse stores materialized views of data from one or more sources, with the purpose of efficiently implementing decision-support or OLAP queries. One of the most important decisions in designing a data warehouse is the selection of materialized views to be maintained at the warehouse. The goal is to select an appropriate set of views that minimizes total query response time and the cost of maintaining the selected views, given a limited amount of resource, e.g., materialization time, storage space, etc. In This work, we have developed a theoretical framework for the general problem of selection of views in a data warehouse. We present polynomial-time heuristics for a selection of views to optimize total query response time under a disk-space constraint, for some important special cases of the general data warehouse scenario, viz.: 1) an AND view graph, where each query/view has a unique evaluation, e.g., when a multiple-query optimizer can be used to general a global evaluation plan for the queries, and 2) an OR view graph, in which any view can be computed from any one of its related views, e.g., data cubes. We present proofs showing that the algorithms are guaranteed to provide a solution that is fairly close to (within a constant factor ratio of) the optimal solution. We extend our heuristic to the general AND-OR view graphs. Finally, we address in detail the view-selection problem under the maintenance cost constraint and present provably competitive heuristics.     

Finding an efficient rewriting of OLAP queries using materialized views in data warehouses

OLAP queries involve a lot of aggregations on a large amount of data in data warehouses. To process expensive OLAP queries efficiently, we propose a new method to rewrite a given OLAP query using various kinds of materialized views which already exist in data warehouses. We first define the normal forms of OLAP queries and materialized views based on the selection and aggregation granularities, which are derived from the lattice of dimension hierarchies. Conditions for usability of materialized views in rewriting a given query are specified by relationships between the components of their normal forms. We present a rewriting algorithm for OLAP queries that can effectively utilize materialized views having different selection granularities, selection regions, and aggregation granularities together. We also propose an algorithm to find a set of materialized views that results in a rewritten query which can be executed efficiently. We show the effectiveness and performance of the algorithm experimentally.     

多版本数据仓库中版本进化的实现

多版本数据仓库中模式进化将产生新的数据仓库版本,直接设计基本的模式进化操作算法来实现数据仓库版本进化会使得算法中包含大量冗余代码,并产生多个重复版本对象。设计进化元操作来实现数据仓库版本的进化,给出了数据仓库版本的一种形式定义以及基于版本对象的三类元操作定义,并用案例说明了通过执行元操作实现数据仓库版本进化的算法思想。进化元操作的设计降低了版本进化算法的复杂性,有利于版本的管理与维护。     

Locking protocols for materialized aggregate join views

The maintenance of materialized aggregate join views is a well-studied problem. However, to date the published literature has largely ignored the issue of concurrency control. Clearly, immediate materialized view maintenance with transactional consistency, if enforced by generic concurrency control mechanisms, can result in low levels of concurrency and high rates of deadlock. While this problem is superficially amenable to well-known techniques, such as fine-granularity locking and special lock modes for updates that are associative and commutative, we show that these previous high concurrency locking techniques do not fully solve the problem, but a combination of a "value-based" latch pool and these previous high concurrency locking techniques can solve the problem.     

Using partial information to update materialized views

This paper lays the theoretical foundations for identifying classes of views and data partitioning strategies that allow efficient incremental view maintenance using “partial information” about the underlying base relations. We present necessary and sufficient conditions for determining how a materialized select-project-join view can be updated using only the view definition, the current view materialization, and the update (i.e., no base relations); and also how to update views using only the view definition, the update, and the updated base relation (i.e., not the other base relations). We generalize the above results to use an arbitrary subset of the base relations and the current view materialization. Our results are especially useful in distributed databases, in disconnected and mobile computing environments where the underlying database is not always accessible, and in data warehousing applications. Our techniques can be used to minimize remote data access and often to completely avoid remote access. This paper gives the intuition and theory necessary for identifying and designing views that are efficiently maintainable using partial information.     

Queries and materialized views on probabilistic databases

We review in this paper some recent yet fundamental results on evaluating queries over probabilistic databases. While one can see this problem as a special instance of general purpose probabilistic inference, we describe in this paper two key database specific techniques that significantly reduce the complexity of query evaluation on probabilistic databases. The first is the separation of the query and the data: we show here that by doing so, one can identify queries whose data complexity is #P-hard, and queries whose data complexity is in PTIME. The second is the aggressive use of previously computed query results (materialized views): in particular, by rewriting a query in terms of views, one can reduce its complexity from #P-complete to PTIME. We describe a notion of a partial representation for views, and show that, once computed and stored, this partial representation can be used to answer subsequent queries on the probabilistic databases. evaluation.     

Answering queries using materialized views with minimum size

In this paper, we study the following problem. Given a database and a set of queries, we want to find a set of views that can compute the answers to the queries, such that the amount of space, in bytes, required to store the viewset is minimum on the given database. (We also handle problem instances where the input has a set of database instances, as described by an oracle that returns the sizes of view relations for given view definitions.) This problem is important for applications such as distributed databases, data warehousing, and data integration. We explore the decidability and complexity of the problem for workloads of conjunctive queries. We show that results differ significantly depending on whether the workload queries have self-joins. Further, for queries without self-joins we describe a very compact search space of views, which contains all views in at least one optimal viewset. We present techniques for finding a minimum-size viewset for a single query without self-joins by using the shape of the query and its constraints, and validate the approach by extensive experiments. Part of this article was published elsewhere [Chirkova, R., Li, C.: Materializing views with minimal size to answer queries. PODS (2003)]. In addition to the prior materials, this article contains new theoretical results, as well as new results on how to efficiently implement the proposed techniques (Sects. 5 and 5.4)     

数据仓库自维护下视图分解系统的设计与实现

数据仓库自维护实质上是通过维护实化视图实现,然而现有的实化视图自维护策略不能有效的减少数据仓库集成端和数据源监视端的多余数据,从而影响数据仓库环境的整体响应速度.一种基于数据仓库自维护方法的视图分解系统改进了现有的视图分解模式,将全局定义的实化视图分解成局部定义的单源视图集来减少存在数据仓库中不必要的数据,实现了现有实化视图自维护策略的分解和重写,提高数据仓库自维护效率.