复杂立方体聚集依赖及其查询技术

针对复杂立方体查询中可能存在的3种聚集依赖(完全依赖、部分依赖和互斥依赖),分别提出了3种基于Cache重用技术的解决方法:完全Cache重用、部分Cache重用以及反Cache重用机制,并相应地给出了计算方法和算法.在模拟和真实数据集上的实验结果表明,不同数据集下改进算法均比基本算法的效率有明显提高,特别地,数据量越大,Cache重用技术的优越性越明显.     

数据集成系统中的动态查询处理

为实现数据集成查询我们会用到查询优化器,而传统的查询优化器生成的执行计划会由于以下几个原因产生不良的结果：成本估计不正确,运行时可用的内存不足和数据传输率无法预测,所有这些问题都要求助于动态策略来修正静态的查询执行计划。介绍了一个动态的查询处理框架和这个框架用到的动态策略。     

Microsoft OLAP Services数据安全性设计

介绍了OLAP系统的基本结构和对Microsoft OLAP Services系统的安全体系进行了全面的分析，最后给出了OLAP数据安全性设计的方案。     

Diversification on big data in query processing

Recently, in the area of big data, some popular applications such as web search engines and recommendation systems, face the problem to diversify results during query processing. In this sense, it is both significant and essential to propose methods to deal with big data in order to increase the diversity of the result set. In this paper, we firstly define the diversity of a set and the ability of an element to improve the overall diversity. Based on these definitions, we propose a diversification framework which has good performance in terms of effectiveness and efficiency. Also, this framework has theoretical guarantee on probability of success. Secondly, we design implementation algorithms based on this framework for both numerical and string data. Thirdly, for numerical and string data respectively, we carry out extensive experiments on real data to verify the performance of our proposed framework, and also perform scalability experiments on synthetic data.     

超大型压缩数据仓库的查询研究

查询速度是联机分析处理中的一个关键性能指标,人们通过事先生成所有可能的聚集来提高查询速度,然而这样的完全物化是以存储空间为代价的.针对数据立方体数据分布特点和结合压缩技术,本文介绍如何最大化节省存储空间来进行完全物化,然后在此基础上对查询进行了研究,以达到最小存储空间以及较好的查询速度的目的.     

基于“C藤”Pair Copula的高维OLAP查询建模方法研究

信息爆炸造成的数据仓库维度的急剧增加,大大影响了OLAP查询模型的精度和效率.首次将数理统计学中的“C藤”Pair Copula引入到OLAP查询建模的研究中,有效地解决了高维OLAP查询建模时的“维数灾难”问题,并设计了针对该模型的参数估计方法以提取数据概要知识.实验分析表明与传统方法相比,基于Pair Copula方法的模型可以在保证OLAP的查询精度的基础上减少数据立方体的存储空间,并且在高维数据环境下具有更高的查询效率.     

Hierarchical clustering for OLAP: the CUBE File approach

This paper deals with the problem of physical clustering of multidimensional data that are organized in hierarchies on disk in a hierarchy-preserving manner. This is called hierarchical clustering. A typical case, where hierarchical clustering is necessary for reducing I/Os during query evaluation, is the most detailed data of an OLAP cube. The presence of hierarchies in the multidimensional space results in an enormous search space for this problem. We propose a representation of the data space that results in a chunk-tree representation of the cube. The model is adaptive to the cube’s extensive sparseness and provides efficient access to subsets of data based on hierarchy value combinations. Based on this representation of the search space we formulate the problem as a chunk-to-bucket allocation problem, which is a packing problem as opposed to the linear ordering approach followed in the literature. We propose a metric to evaluate the quality of hierarchical clustering achieved (i.e., evaluate the solutions to the problem) and formulate the problem as an optimization problem. We prove its NP-Hardness and provide an effective solution based on a linear time greedy algorithm. The solution of this problem leads to the construction of the CUBE File data structure. We analyze in depth all steps of the construction and provide solutions for interesting sub-problems arising, such as the formation of bucket-regions, the storage of large data chunks and the caching of the upper nodes (root directory) in main memory. Finally, we provide an extensive experimental evaluation of the CUBE File’s adaptability to the data space sparseness as well as to an increasing number of data points. The main result is that the CUBE File is highly adaptive to even the most sparse data spaces and for realistic cases of data point cardinalities provides hierarchical clustering of high quality and significant space savings.     

向量计算Array OLAP查询处理技术

多核和众核处理器成为新的具有强大并行处理能力的大内存计算平台的主流配置。多核处理器遵循以LLC（Last Level Cache,最后一级cache）大小为中心的优化技术,而众核处理器,如Phi、GPU协处理器,则采用较小的cache并以更多的硬件级线程来掩盖内存访问延迟的设计。随着处理核心数量的增长,计算框架更倾向于面向大规模处理核心的、代码执行效率高并且扩展性强的设计思想。提出了一种基于数组存储和向量处理的内存分析处理框架Array OLAP,简化OLAP的存储模型和查询处理模型。在Array OLAP计算框架中,维表规范化为基于向量的维过滤器,事实表规范化为带有多维索引的度量属性。通过多维索引计算,一个多维查询被简化为事实表上的向量索引扫描并根据度量表达式进行聚集计算。规范化的向量查找和向量索引扫描具有较好的代码执行效率,并且阶段化的处理模型更好地适应不同的计算平台,将计算阶段分配给最适合的计算平台。同时,Array OLAP是一种面向数据仓库模式特点的设计,向量处理模型设计简单,对于数据仓库维表较小且增长缓慢的特点具有较好的效率。描述了在不同平台上的Array OLAP计算框架并且通过基准测试评估Array OLAP的性能,通过与当前的内存分析型数据库的性能对比,Array OLAP性能超过主流的内存分析型数据库并且可以平滑地迁移到新的硬件平台。     

OLAP数据仓库在电网调度决策中的研究与应用

以某电力系统为研究背景,在对原有的数据源进行分析和重新组织的基础上,构建电网调度数据仓库,并建立多维雪花模式的数据立方体。运用OLAP和数据挖掘技术,从多角度、多层次快速地分析和查询数据仓库的数据,实现负荷预估和调度的科学化,并说明OLAP数据仓库能够为电网调度管理人员提供有效的决策信息。     

一种面向分布式数据库的多并发OLAP型查询性能预测方法

数据库查询性能预测即预测查询的执行延迟时间。数据库查询延迟预测技术能够用于查询调度和查询进度显示等。当前的查询性能预测技术都具有一定的局限性,不能准确预测分布式数据库中多并发查询导致CPU、I/O资源争用情形下的查询延迟。针对这一问题,该论文提出了一种在分布式数据库中预测并发OLAP型查询延迟的技术,该技术通过建立查询干扰度和查询敏感度模型来衡量资源（I/O,网络）竞争的激烈程度,并据此来预测不同执行环境下的查询性能。TPC-DS的实验结果表明,分析型查询预测的误差率在25%以下,说明该技术能够较准确地预测查询执行时间。     

面向网络数据管理的并行查询处理

随着互联网的迅猛发展,监控网络的所产生的海量数据对查询处理提出挑战。根据数据明显分为大量的事件数据和少量、稳定的配置数据的特点,提出了一种基于单机DBMS的并行查询处理方法。从关系代数的角度,将任意查询分解成对水平数据分区的子查询和汇总中间结果的后处理查询。借助DBMS提供的数据库链路,在不改动DBMS的情况下,方便地构造查询处理器。用真实负载的测试表明：在中间结果集不很大的情况下,能获得接近线性的扩展比。     

两层传感器网络中隐私保护的等区间近似查询算法

隐私保护已经成为拓展无线传感器网络（WSN）应用的关键因素,是当前的研究热点。针对传感器网络中感知数据的安全性问题,提出了两层传感器网络中隐私保护的等区间近似查询（PEIAQ）算法。首先,将传感器节点编号及其采集的数据等信息隐藏在随机向量中;然后,基站根据接收到的向量信息构造线性方程组,从而得到包含全局统计信息的直方图;最后,根据直方图完成近似查询。此外,PEIAQ利用数据扰动技术和传感器节点与基站共享密钥的方式来对感知数据进行加密,保证了感知数据的隐私性。仿真实验显示,PEIAQ的通信量在查询阶段明显低于隐私保护通用近似查询（PGAQ）的通信量,约节省60%,因此,该PEIAQ具有低能耗、高效率等特点。     

一种OLAP应用系统的设计和实现

通过对数据仓库和OLAP概念及体系结构的分析,描述了一种OLAP应用系统的设计方案,并介绍了它的具体实现方法。基于数据仓库的查询,一般都是及时特定查询,要在严格的响应时间内执行复杂的查询,遍历百万上亿的记录,同时进行可能很复杂的搜索、连接和汇总的操作。查询的数据吞吐量和响应时间是判断数据仓库性能的重点。CUBE的计算是OLAP及时查询的基础,提高查询的速度需要对OLAP进行预先的计算。文中系统比较了一些计算立方体的算法,并运用到具体的系统当中。     

基于语义分解的联机分析处理查询并行优化方案

利用联机分析处理(OLAP)查询中存在的语义关联,对聚集关系与语义分解关系进行了形式化描述,并基于这些关系定义了查询与查询集之间的补集关系,在执行OLAP查询集时,可以利用这些关系尽可能地识别查询集中查询的公共部分,并且可以在查询时从多个角度来采取并行优化措施。实验验证表明采用并行优化方案后,系统的整体效率得到了提高。     

一种P2P网络环境下的OLAP查询方案

传统网络环境和P2P环境中,客户端向OLAP服务器提交OLAP查询,并从服务器获取查询结果,OLAP服务器的负载将随着客户端的增加而急剧增加。设计了一种基于P2P（Peer-to-Peer,点对点技术）技术的DQDC（Distributed Query Data Cube,多维数据集的分布式查询）算法,实现P2P网络中语义级的多节点Data Cube数据共享,从而提高系统整体的决策分析性能。     

无线传感器网络中数据查询处理算法研究

提出一种改进的定向扩散路由,将传感器网络分簇,查询兴趣由sink节点发,只在各簇头节点扩散,簇头以广播的方式在簇内发散兴趣消息,簇成员将感知数据传送到簇头节点,簇头负责将收到的数据进行融合后传到sink节点。仿真结果表明,改进后的查询路由比典型的查询路由定向扩散具有更高的能量有效性和更低的时延,能较好地延长网络的生命周期,提高了传感器网络数据查询处理效率。     

基于社区划分的联机分析处理查询优化方案

针对P2P环境下的联机分析处理（OLAP）查询节点数目不断增加时,易造成网络拥塞、查询效率降低的问题,提出一种基于社区划分的OLAP查询优化方案。该方案构建一个虚拟的社区网,并在此结构上设计了一种基于社区划分的多维数据集(CPDS)的OLAP查询优化算法。实验结果表明,该算法能有效避免因网络节点数目递增而导致的网络负载加剧问题,能有效地减少网络拥塞,优化了OLAP的查询效率,进一步提高P2P环境下OLAP的决策分析性能。     

OLAP query reformulation in peer-to-peer data warehousing

Inter-business collaborative contexts prefigure a distributed scenario where companies organize and coordinate themselves to develop common and shared opportunities, but traditional business intelligence systems do not provide support to this end. To fill this gap, in this paper we envision a peer-to-peer data warehousing architecture based on a network of heterogeneous peers, each exposing query answering functionalities aimed at sharing business information. To enhance the decision making process, an OLAP query expressed on a peer needs to be properly reformulated on the local multidimensional schemata of the other peers. To this end, we present a language for the definition of mappings between the multidimensional schemata of peers and we introduce a query reformulation framework that relies on the translation of mappings, queries, and multidimensional schemata onto the relational level. Then, we formalize a query reformulation algorithm and prove two properties: correctness and closure, that are essential in a peer-to-peer setting. Finally, we discuss the main implementation issues related to the reformulation setting proposed, with specific reference to the case in which the local multidimensional engines hosted by peers use the standard MDX language.     

Extending metric index structures for efficient range query processing

Databases are getting more and more important for storing complex objects from scientific, engineering, or multimedia applications. Examples for such data are chemical compounds, CAD drawings, or XML data. The efficient search for similar objects in such databases is a key feature. However, the general problem of many similarity measures for complex objects is their computational complexity, which makes them unusable for large databases. In this paper, we combine and extend the two techniques of metric index structures and multi-step query processing to improve the performance of range query processing. The efficiency of our methods is demonstrated in extensive experiments on real-world data including graphs, trees, and vector sets.     

Foundations for object-oriented query processing

A minimal framework for an object-oriented query language standard should (1) include a formal definition of a high-level data model and the syntax and semantics of associated query languages, (2) provide the functionality of relational query languages, and (3) support proofs of correctness of transformations for logical query optimization. In this paper, a high-level conceptual model for object-oriented query processing is discussed; the model includes widely-used structural abstractions such as the isa relationship, associations (properties) between complex objects and complex objects/values, and inheritance of properties. A formal, algebraic query language for the model, inspired by relational algebra, is presented. Operators of the algebra allow queries based on values, queries that manipulate entire objects, and queries that construct new objects from existing objects/values. All queries retain connections to existing database objects, providing logical access paths to data. Each query result is a class, so the algebra has the closure property. The intensional and extensional results of query operators are summarized. Two forms of logical query optimization supported by the query algebra are outlined: algebraic transformations and classifier-based optimizations (optimizations which employ inclusion and exclusion dependencies between classes).