基于代价的XML路径索引构建

处理路径表达式是XML查询技术中的难点和热点．在本实验室提出的XML路径表达式索引-结构化映射的基础上．为了降低构建索引所需的空间开销，本文提出了构建路径索引的代价模型，并设计了基于给定查询负载，有选择地构建路径索引的相应算法，为给定查询负载自动选择近最优索引模式（NOIS）．本文还提出了当查询效率发生变化时，系统对索引模式进行自适应调整的策略．实验研究表明：使用本文方法，系统可在不影响路径表达式处理效率的前提下，大大降低路径索引的空间开销，取得查询收益和空间开销的较佳权衡．     

ALERT:基于Radix Tree的工作负载自适应学习型索引

学习型索引通过学习数据分布可以准确地预测数据存取的位置,在保持高效稳定的查询下,显著降低索引的内存占用.现有的学习型索引主要针对只读查询进行优化,而对插入和更新支持不足.针对上述挑战,设计了一种基于Radix Tree的工作负载自适应学习型索引ALERT.ALERT使用Radix Tree来管理不定长的分段,段内采用具有最大误差界的线性插值模型进行预测.同时,ALERT使用一种高效的插入缓冲来降低数据插入更新的代价.针对点查询和范围查询提出两种自适应重组优化方法,通过对工作负载进行感知,动态地调整插入缓冲的组织结构.经实验验证,ALERT与业界流行的学习型索引相比,构建时间平均降低了81%,内存占用平均降低了75%,在保持了优秀读性能的同时,使插入延迟平均降低了50%;此外,ALERT使用自适应重组优化能有效感知查询工作负载特征,与不使用自适应重组优化相比,查询延迟平均降低了15%.     

基于性能需求的自适应中间件框架

分析影响服务质量的构件资源需求与资源依赖关系模型,提出一种自适应中间件框架。该框架能动态感知负载变化,自适应调整服务器配置参数,确保应用的服务质量。采用回溯算法搜索最优配置,以满足性能需求。以一个信息查询系统作为测试用例进行实验,结果表明,该框架可以提高应用程序的性能。     

人工智能赋能的数据管理技术研究

大数据时代,数据规模庞大、数据管理应用场景复杂,传统数据库和数据管理技术面临很大的挑战.人工智能技术因其强大的学习、推理、规划能力,为数据库系统提供了新的发展机遇.人工智能赋能的数据库系统通过对数据分布、查询负载、性能表现等特征进行建模和学习,自动地进行查询负载预测、数据库配置参数调优、数据分区、索引维护、查询优化、查询调度等,以不断提高数据库针对特定硬件、数据和负载的性能.同时,一些机器学习模型可以替代数据库系统中的部分组件,有效减少开销,如学习型索引结构等.分析了人工智能赋能的数据管理新技术的研究进展,总结了现有方法的问题和解决思路,并对未来研究方向进行了展望.     

一种支持高效XML 路径查询的自适应结构索引

提出了一种新的自适应结构索引:AS-Index(adaptive structural index),能够克服现有静态索引和自适应索引的缺陷,具备高效的查询和调整性能.AS-Index 建立在F&B-Index 的基础之上,其索引结构包括F&B-Index,Query-Table 和Part-Table.Query-Table 能够记录频繁查询,避免了查询过程中的冗余操作.并且,在Query-Table 的基础上提出了自底向上的查询处理过程,能够充分利用现有的频繁查询高效地回答非频繁查询.Part-Table 用于优化包含祖先后裔边的查询,进一步提高了查询性能.现有的自适应结构索引的调整粒度是XML 元素节点,调整过程往往需要遍历整个文档.而AS-Index 是基于F&B-Index 节点的增量调整,其过程是局部的,高效的,并且能够支持复杂分支查询的调整.实验结果表明,AS-Index 在查询和调整性能上优于现有的XML 结构索引.同时,相比于现有的自适应结构索引,AS-Index 针对大规模文档具有更加优良的可扩展性.     

基于增量信息索引的子图查询算法

当前图数据库中的子图同构查询算法主要是依赖倒排索引,然而处理那些具有庞大数据的数据库和复杂的查询愈发成为挑战。研究目的是设计一个算法,使用新的索引作为查询处理的核心,记录查询图的每一个细小改变,并使用一种特殊的数据结构来维护。先是引出一个索引算法,然后逐渐分析整个索引、查询过程,并利用该算法实现一个系统,最后在不同数据集和查询上进行实验。实验证明了该算法具有良好的时间、空间效率和扩展性。新的索引算法能够支持更大的查询图和更加灵活的查询。通过实现的系统和其他系统的对比实验,验证了算法的有效性。     

使用MapReduce构建列存储数据的索引

大数据的存储与分析是近年来数据库领域研究的热点,高效的索引技术是提高大数据查询分析性能的重要技术手段。在现有的数据存储模型及索引技术研究基础上,提出使用MapReduce构建列存储数据的索引。该索引技术结合MapReduce编程模型,先在Map阶段完成数据划分,然后在Reduce阶段完成数据的排序,最后在数据有序的Reduce节点上创建RB+树索引,从而减少索引创建时因为RB+树内部节点递归分裂而产生的昂贵代价和树的高度,提高数据查询的性能。通过在真实数据集上进行实验,验证了所提出方法的有效性。     

基于查询采样的高维数据混合索引

为了改进高维数据库查询的效率,通常需要根据数据分布来选择合适的索引策略.然而,经典的分布模型难以解决实际应用中图像、视频等高维数据复杂的分布估计问题.提出一种基于查询采样进行数据分布估计的方法,并在此基础上提出了一种支持最近邻查询的混合索引,即针对多媒体数据分布的不均匀性,自适应地对不同分布的数据使用不同的索引结构,建立统一的索引结构.为了实现混合索引,采用构造性方法:首先通过聚类分解分割数据并建立树状索引;然后使用查询采样算法,对数据实际分布进行估计;最后根据数据分布的特性,把稀疏数据从树状索引中剪裁出来,进行基于顺序扫描策略的索引,而分布比较密集的数据仍然保留在树状索引中.在4个真实的图像数据集上进行了充分的实验,结果显示,该索引方法明显优于iDistance,M-Tree等度量空间索引,在维数达到336时,查询效率仍高于顺序扫描.实验结果显示,该查询采样算法在采样数据量仅为N~(1/2)(N为数据量)的情况下即可获得满足索引需要的分布估计结果.     

DBMS负载自适应体系中系统监测部件的研究

数据库系统负载自适应体系是一个性能管理过程,在这一过程中自主计算数据库系统能够自动地控制和过滤负载、并有效地使用数据库系统资源来满足网络应用的性能要求。在数据库系统负载自适应体系框架中,系统监测部件负责监控系统资源的使用情况,并将监控到的信息反馈给相关部件。提出了系统监测部件应该具备的功能,给出了实现的途径,并研究了监测部件的性能要求,分析了如何降低监测部件本身对数据库的影响。最后基于Oracle数据库进行了验证。     

一种具有自适应机制的闪存数据库索引结构

针对闪存数据库系统索引技术中基于日志更新策略存在的检索效率低、日志空间分配不合理及合并带来的高昂更新代价等问题,提出一种具有自适应机制的索引结构LM-B+TREE。LM-B+TREE将索引的更新缓冲页映射于传统B+TREE的相应节点,并根据闪存索引的读写负载及读写代价差异,动态地分配缓冲更新区,自适应地调整索引架构。实验证明LM-B+TREE能够动态地调整索引架构来适应索引的读写负载代价,在减少索引更新代价的同时,有效地提高了索引的查询性能。     

The performance of disk arrays in shared-memory database machines

Disk arrays and shared-memory multiprocessors are new technologies that are rapidly becoming pervasive. They are complementary because disk arrays naturally balance the I/O workload by interleaving data across all disks while a shared-memory multiprocessor balances the processing workload across multiple processors. In this paper, we examine how disk arrays and shared memory multiprocessors lead to an effective method for constructing database machines for general-purpose complex query processing. We show that disk arrays can lead to cost-effective storage systems if they are configured from suitably small formfactor disk drives. We introduce the storage system metricdata temperature (IO/s/Gbyte) as a way to evaluate how well a disk configuration can sustain its workload, and we show that disk arrays can sustain the same data temperature as a more expensive mirrored-disk configuration. We use the metric to evaluate the performance of disk arrays in XPRS, an operational shared-memory multiprocessor database system being developed at the University of California, Berkeley.     

基于强化学习的数据库多属性索引推荐

传统关系型数据库通过人工方式进行索引推荐,已无法适应日益增长的数据需求,而机器学习技术可以有效地解决数据库索引选择问题。针对以往仅在静态数据库下进行索引推荐且无法及时更新索引配置的局限性,提出了一种基于强化学习算法实现为数据库数据动态变化情况下的一组工作负载推荐最佳多属性索引配置的方法(multi-attribute index intelligent recommendation approach, MIRA)。在公开的TPC-H数据集上的实验结果表明,该方法不仅能有效地为一组工作负载推荐最佳的索引配置,而且优于自定义的比较基线和相关强化学习方法。     

移动复制数据库系统冲突检测及消解策略

复制技术是提高移动库系统性能的一项关键技术，该文提出了一种新的移动复制数据库系统模型－事务级吉果集传递（TLRSP）移动复制模型，重点分析了该模型中的冲突检测及消解策略，并给出具体的实现算法，TLRSP移动复制模型允许移动用户在系统断连时存取数据库的本地副本并提交事务，重新连接时进行冲突的检测及消解，同时进行事务结果集的合并，最后通过增量刷新的方式进行同步处理，使得系统最终收敛于一致性的状态。此外，通过引入简化的事务日志，数据牌本号以及权限控制等概念，TLRSP模型有效地降低了移动数据库系统的资源消耗，保证了数据库的一致性，从而为移动数据库系统复制提供了一个可行的解决方案。     

Efficient histogram-based range query estimation for dirty data

In recent years, data quality issues have attracted wide attentions. Data quality problems are mainly caused by dirty data. Currently, many methods for dirty data management have been proposed, and one of them is entity-based relational database in which one tuple represents an entity. The traditional query optimizations are not suitable for the new entity-based model. Then new query optimizations need to be developed. In this paper, we propose a new query selectivity estimation strategy based on histogram, and focus on solving the overestimation which traditional methods lead to. We prove our approaches are unbiased. The experimental results on both real and synthetic data sets show that our approaches can give good estimates with low error.     

Workload management: a technology perspective with respect to self-* characteristics

Rapid growth in data, maximum functionality requirements and changing behavior in the database workload tends the workload management to be more complex. Organizations have complex type of workloads that are very difficult to manage by humans and even in some cases this management becomes impossible. Human experts take long time to get sufficient experience so that they can manage the workload efficiently. The versatility in workload due to huge data size and user requirements leads us towards the new challenges. One of the challenges is the identification of the problematic queries and the decision about these, i.e. whether to continue their execution or stop. The other challenge is how to characterize the workload, as the tasks such as configuration, prediction and adoption are fully dependent on the workload characterization. Correct and timely characterization leads managing the workload in an efficient manner and vice versa. In this scenario, our objective is to produce a workload management strategy or framework that is fully adoptive. The paper provides a summary of the structure and achievements of the database tools that exhibit Autonomic Computing or self-* characteristics in workload management. We have categorized the database workload tools to these self-* characteristics and identified their limitations. Finally the paper presents the research done in the database workload management tools with respect to the workload type and Autonomic Computing.     

Online probabilistic learning for fuzzy inference system

Online learning is a key methodology for expert systems to gracefully cope with dynamic environments. In the context of neuro-fuzzy systems, research efforts have been directed toward developing online learning methods that can update both system structure and parameters on the fly. However, the current online learning approaches often rely on heuristic methods that lack a formal statistical basis and exhibit limited scalability in the face of large data stream. In light of these issues, we develop a new Sequential Probabilistic Learning for Adaptive Fuzzy Inference System (SPLAFIS) that synergizes the Bayesian Adaptive Resonance Theory (BART) and Rule-Wise Decoupled Extended Kalman Filter (RDEKF) to generate the rule base structure and refine its parameters, respectively. The marriage of the BART and RDEKF methods, both of which are built upon the maximum a posteriori (MAP) principle rooted in the Bayes’ rule, offers a comprehensive probabilistic treatment and an efficient way for online structural and parameter learning suitable for large, dynamic data stream. To manage the model complexity without sacrificing its predictive accuracy, SPLAFIS also includes a simple procedure to prune inconsequential rules that have little contribution over time. The predictive accuracy, structural simplicity, and scalability of the proposed model have been exemplified in empirical studies using chaotic time series, stock index, and large nonlinear regression datasets.     

分布式数据库中一致性与可用性的关系

随着各类应用在数据量和业务量上的扩展,单机数据库系统越发难以应对现实需求。分布式数据库能够根据业务的需求动态地扩容,因此逐步开始受到应用的青睐。近年来,分布式数据库产品层出不穷,并在互联网应用中被大量投入使用。然而,分布式数据库的系统复杂度前所未有。为了让系统可用,设计者需要在多种属性中作合理选择和折中。这造成现有的数据库产品形态各异、优缺点对比分明。至今为止,尚未有人对分布式数据库的设计空间和折中方案进行过深入分析和整理。本文作者在对多个分布式数据库产品进行深入理解之后认识到：分布式数据库系统的设计方案可以通过三个属性进行基本刻画–操作一致性、事务一致性和系统可用性。虽然这三个属性并不新颖,但它们在数据库语境下的含义在文献中尚未得到充分澄清。本文对这三个属性进行澄清,并通过它们对典型数据库产品的格局进行概括、对现有的分布式数据库技术进行综述。此外,本文还对这三个属性之间的相互关系进行深入分析,以期帮助未来的开发者在分布式数据库的设计过程中作出合理选择。     

Personalized ranking in web databases: establishing and utilizing an appropriate workload

The emergence of the deep Web has given a new connotation to the concept of ranking database query results. Earlier approaches for ranking either resorted to analyzing frequencies of database values and query logs or establishing user profiles. In contrast, an integrated approach, based on the notion of a similarity model, for holistically supporting user- and query-dependent ranking has been recently proposed (Telang et al. in IEEE Transactions on Knowledge and Data Engineering (TKDE), 2011). An important component of this framework is a workload consisting of ranking functions, wherein each function represents an individual user’s preferences towards the results of a specific query. At the time of answering a query for which no prior ranking function exists, the similarity model is employed, and is expected to ensure a good quality of ranking as long as a ranking function for a very similar user-query pair exists in this workload. In this paper, we address the problem of determining an appropriate set of user-query pairs to form a workload of ranking functions to support user- and query-dependent ranking for Web databases. We propose a novel metric, termed workload goodness, that quantifies the notion of a “good” workload into an absolute value. The process of finding such a workload of optimal goodness is a combinatorially explosive problem; therefore, we propose a heuristic solution, and advance three approaches for determining an acceptable workload, in a static as well as a dynamic environment. We discuss the effectiveness of our proposal analytically as well as experimentally over two Web databases.