数据仓库中实体化视图的一致性维护问题的研究

实现实体化视图是提高系统响应时间的一个关键技术和有效的解决方案,但在具体的实现过程中面临着实体化视图的一致性维护问题。由于源数据的不稳定性,其结构和数据的变化必须及时传播到实体化视图中,以保持实体化视图与源数据的变化一致性,否则会降低实体化视图中数据的新鲜度,并影响OLAP查询结果的真实性和有效性。为此本文针对视图的一致性维护问题,介绍了视图中数据的维护方法,同时还就实体化视图结构的维护,提出了使视图重计算代价最小化的解决方案和基本算法。     

数据仓库性能优化之索引和物化视图耦合方法

为了进一步提高数据仓库的性能, 通过分析数据仓库中性能优化技术的特点, 提出了索引和物化视图耦合的性能优化技术。通过数据挖掘自动选择候选索引和物化视图, 减少查询的扫描范围; 然后研究在物化视图上建立索引的空间高效存储方法, 以提高查询速率; 最后利用成本模型对耦合情况进行分析, 验证了耦合方法可以极大提高单一索引查询或者物化视图的性能。     

基于时间戳实化视图动态维护技术的研究

由于源数据的不稳定性,其结构和数据的变化必须及时传播到实体化视图中,以保持实体化视图与源数据变化的一致性,否则会降低实体化视图中数据的新鲜度,并影响OLAP查询结果的真实性和有效性.为此,本文提出了基于时间戳的动态视图维护技术.该技术采用版本链控制技术,通过时间戳的控制进一步使视图更新和查询的同步进行,有效地解决了由于OLTP更新事务和OLAP事务同时访问数据所发生冲突的问题,在满足视图联机实时维护的同时,更好的提高了数据仓库的新鲜度和OLAP的查询效率.     

数据仓库中基于实体化辅助视图的视图增量维护

为了加快对大量数据的查询处理速度，通常在数据仓库以实视图方式存储数据，当基础数据发生变化时，这些实视图也必须随着更新，因而视图自维护和一致性维护成为数据仓库的重要问题。本文提出利用视图计算的中间结果创建辅助视图，在数据仓库中进行实体化，采用有效的增量维护算法计算实视图的精确变化，实现数据仓库视图自维护。     

基于浓缩数据立方的内存实化小方的动态选择

OLAP通常使用预计算数据立方的方法提高可能的聚集查询的响应速度,在内存实化预计算的数据,可以更进一步加快响应的速度,但是受到内存空间的限制。在浓缩数据立方的环境中,动态地选择一定的数据小方在内存实化,加快响应速度,并更好地适应不同的查询模式。给出了在动态选择模型中,特定存储方式下的查询分解和响应算法。     

基于X-RESTORE查询XML视图

基于转换 XML文档到关系数据库中进行存储与查询的策略 ,研究了 XML视图查询的有效计算问题 .提出了XML 视图查询的合成重写技术 ,它能够消除视图查询中所有在视图结构上的路径导航操作 ,并将视图查询中所有在原文档结构上的路径导航操作以及所有谓词操作下推到视图定义中去 ,与视图定义中的路径导航操作相结合 ,形成统一的在原文档结构上的路径导航操作 .视图合成重写不仅避免了对视图中不出现在最后结果中的中间 XML 片段的构造 ,而且允许将查询中的所有内存密集型或数据密集型操作尽量下推到关系引擎中去执行 ,从而提高 XQuery查询的执行性能     

一种用于搜索可能响应查询的候选实化视图的索引

1 引言实化视图是存储了实际数据的视图,在响应查询时如果能直接利用实化视图,那么就可以避免相应的重新计算,从而提高查询性能。对一个复杂查询,一个实化视图往往只能替代查询的一部分(子表达式),在构造一个重写查询时,这种视图替代过程将被多次调用。当实化视图的数量很多时,如果每次视图替代都要把所有视图检查一遍,那将花费大量时间,从而降低利用视图响应查询的效果。     

基于兴趣视图子集的流立方体计算方法

流立方体计算是流式数据多维分析的重要基础,然而流式数据的动态性、无限性、突发性等特征使其面临巨大的挑战.在实际应用中,用户的兴趣通常集中在部分视图上,基于这个特点提出了一种基于兴趣视图子集的计算方法,依据用户历史查询信息确定兴趣视图子集与兴趣路径,同时定义了Stream-Tree结构用于在主存中物化存储兴趣视图子集所包含的数据单元,在运行过程中依据多层次时间窗口约束不断更新和维护Stream-Tree中存储的数据单元,而对于稀疏数据单元仅保留高层次的聚集值.实验和分析表明,该方法能够在有限的主存空间中维持流立方体当前窗口内的数据单元,同时能够支持快速更新维护存储结构和响应用户查询.     

实体化视图动态选择预处理算法的研究

实体化视图是数据仓库中提高查询效率的有效手段，数据仓库运行期间，需要对其中的实体化视图进行维护，从而保证用户查询的响应时间较短。针对用于实体化视图动态选择的遗传算法收敛速度慢，运行时间长的问题，提出一种预处理算法来计算动态选择实体化视图时遗传算法的初始群体。理论分析和宴验结果表明，该算法可以有效地提高实体化视图动态选择时的寻优收敛速度。     

使用MapReduce构建列存储数据的索引

大数据的存储与分析是近年来数据库领域研究的热点,高效的索引技术是提高大数据查询分析性能的重要技术手段。在现有的数据存储模型及索引技术研究基础上,提出使用MapReduce构建列存储数据的索引。该索引技术结合MapReduce编程模型,先在Map阶段完成数据划分,然后在Reduce阶段完成数据的排序,最后在数据有序的Reduce节点上创建RB+树索引,从而减少索引创建时因为RB+树内部节点递归分裂而产生的昂贵代价和树的高度,提高数据查询的性能。通过在真实数据集上进行实验,验证了所提出方法的有效性。     

一种基于对象关系模型的时空数据库管理系统体系结构

时空数据库的关键与难点在于其实现技术．本文提出了一种基于对象关系模型的优化型时空数据库管理系统体系结构，该体系结构采用时空数据类型扩展和时空操作扩展技术对数据库管理系统的内核进行扩充，使其具有内建的时空数据管理能力，同时以时空查询优化层实现时空查询的逻辑优化，解决了底层数据库管理系统的查询优化问题。     

Optimizing complex queries based on similarities of subqueries

As database technology is applied to more and more application domains, user queries are becoming increasingly complex (e.g. involving a large number of joins and a complex query structure). Query optimizers in existing database management systems (DBMS) were not developed for efficiently processing such queries and often suffer from problems such as intolerably long optimization time and poor optimization results. To tackle this challenge, we present a new similarity-based approach to optimizing complex queries in this paper. The key idea is to identify similar subqueries that often appear in a complex query and share the optimization result among similar subqueries in the query. Different levels of similarity for subqueries are introduced. Efficient algorithms to identify similar queries in a given query and optimize the query based on similarity are presented. Related issues, such as choosing good starting nodes in a query graph, evaluating identified similar subqueries and analyzing algorithm complexities, are discussed. Our experimental results demonstrate that the proposed similarity-based approach is quite promising in optimizing complex queries with similar subqueries in a DBMS.     

数据库服务器与WWW之间的一种集成设计

本文根据WWW用户和数据库用户各自对数据访问的特点不同,提出了一种将数据库服务器与WWW集成的结构。该集成结构中既提供了基于超文本的导航浏览功能访问数据库系统,又支持用户直接利用数据库系统本身的查询界面,快速查询数据对象。     

Comparing columnar,row and array DBMSs to process recursive queries on graphs

Analyzing graphs is a fundamental problem in big data analytics, for which DBMS technology does not seem competitive. On the other hand, SQL recursive queries are a fundamental mechanism to analyze graphs in a DBMS, whose processing and optimization are significantly harder than traditional SPJ queries. Columnar DBMSs are a new faster class of database system, with significantly different storage and query processing mechanisms compared to row DBMSs, still the dominating technology. With that motivation in mind, we study the optimization of recursive queries on a columnar DBMS focusing on two fundamental and complementary graph problems: transitive closure and adjacency matrix multiplication. From a query processing perspective we consider the three fundamental relational operators: selection, projection and join (SPJ), where projection subsumes SQL group-by aggregation. We present comprehensive experiments comparing recursive query processing on columnar, row and array DBMSs to analyze large graphs with different shape and density. We study the relative impact of query optimizations and we compare raw speed of DBMSs to evaluate recursive queries on graphs. Results confirm classical query optimizations that keep working well in a columnar DBMS, but their relative impact is different. Most importantly, a columnar DBMS with tuned query optimization is uniformly faster than row and array systems to analyze large graphs, regardless of their shape, density and connectivity. On the other hand, there is no clear winner between the row and array DBMSs.     

面向数据流离线分析的无共享并行查询中间件研究

通过分析网络监控离线分析处理的负载特征,给出了一种面向数据流离线分析处理的并行多策略查询中间件,并利用多策略及DBMS实现了局部结果的汇总,对需后处理查询的系统扩展性、不需后处理查询的系统扩展性分别进行了评价分析。评价分析结果表明:提出的基于面向数据流离线分析处理的并行多策略查询中间件,不但做到了中间件的轻量级,实现了查询内部的并行化;同时,能利用多策略及DBMS实现子节点间并行查询,能提高查询的响应时间,从而在多节点时保持较好的扩展比,避免了系统过载,提高了资源利用率。     

基于VFP的数据库Rushmore技术分析

Visual FoxPro中的Rushmore技术虽然在一般教程中极少讲述,但在实用中却每有奇效,它也构成了Visual FoxPro独有的特色。该文从Visual FoxPro之索引、查询和优化等侧面,详细剖析了Rushmore技术的相关细节,对数据库开发者有一定的参考和借鉴作用。     

PARADISE: Big data analytics using the DBMS tightly integrated with the distributed file system

There has been a lot of research on MapReduce for big data analytics. This new class of systems sacrifices DBMS functionality such as query languages, schemas, or indexes in order to maximize scalability and parallelism. However, as high functionality of the DBMS is considered important for big data analytics as well, there have been a lot of efforts to support DBMS functionality in MapReduce. HadoopDB is the only work that directly utilizes the DBMS for big data analytics in the MapReduce framework, taking advantage of both the DBMS and MapReduce. However, HadoopDB does not support sharability for the entire data since it stores the data into multiple nodes in a shared-nothing manner—i.e., it partitions a job into multiple tasks where each task is assigned to a fragment of data. Due to this limitation, HadoopDB cannot effectively process queries that require internode communication. That is, HadoopDB needs to re-load the entire data to process some queries (e.g., 2-way joins) or cannot support some complex queries (e.g., 3-way joins). In this paper, we propose a new notion of the DFS-integrated DBMS where a DBMS is tightly integrated with the distributed file system (DFS). By using the DFS-integrated DBMS, we can obtain sharability of the entire data. That is, a DBMS process in the system can access any data since multiple DBMSs are run on an integrated storage system in the DFS. To process big data analytics in parallel, our approach use the MapReduce framework on top of a DFS-integrated DBMS. We call this framework PARADISE. In PARADISE, we employ a job splitting method that logically splits a job based on the predicate in the integrated storage system. This contrasts with physical splitting in HadoopDB. We also propose the notion of locality mapping for further optimization of logical splitting. We show that PARADISE effectively overcomes the drawbacks of HadoopDB by identifying the following strengths. (1) It has a significantly faster (by up to 6.41 times) amortized query processing performance since it obviates the need to re-load data required in HadoopDB. (2) It supports query types more complex than the ones supported by HadoopDB.     

一种外联接优化的设计与实现

一、引言关系代数中联接运算将两个关系中相匹配的元组结合起来,生成新的元组对。如果,一个关系中的元组在另一个关系中没有相匹配的元组,那么这个元组中的数据就不会出现在联接结果中。Codd在1979年提出了关系数据库的扩展模型,引进了外联接(outerjoin)的概念,作为普通联接的补充。外联接保存了进行联接运算的两个关系中的一个或两个中的所需要的信息。外联接已被列入ISO-ANSI的SQL92标准。目前流行的关系数据库管理系统如Sybase,SQL Server,ORACLE等均支持这种联接方式。在我们自行研制开发的数据库管理系统DM3中,也实现了这种联接方式,并研究了其优化方法。     

基于XML的汽车零部件查询系统的研究和设计

在一个汽车零部件信息查询系统的基础，从一个侧面对基于XML存储的具有语义特征的Web信息的存储、索引和查询进行了研究，通过把XML和传统的DBMS技术结合起来，提出了一种新的实现方法。     

A foundation for conventional and temporal query optimizationaddressing duplicates and ordering

Most real-world databases contain substantial amounts of time-referenced, or temporal, data. Recent advances in temporal query languages show that such database applications may benefit substantially from built-in temporal support in the DBMS. To achieve this, temporal query representation, optimization, and processing mechanisms must be provided. This paper presents a foundation for query optimization that integrates conventional and temporal query optimization and is suitable for both conventional DBMS architectures and ones where the temporal support is obtained via a layer on top of a conventional DBMS. This foundation captures duplicates and ordering for all queries, as well as coalescing for temporal queries, thus generalizing all existing approaches known to the authors. It includes a temporally extended relational algebra to which SQL and temporal SQL queries may be mapped, six types of algebraic equivalences, concrete query transformation rules that obey different equivalences, a procedure for determining which types of transformation rules are applicable for optimizing a query, and a query plan enumeration algorithm