能效查询处理与优化可行性初探

随着数据管理需求的不断增长,降低与控制数据中心的能耗成为一个挑战性问题. DBMS 是数据中心核心软件,能效查询处理与优化是其中一个重要议题. 本文提出了新型的能耗代价评估模型,通过评估查询计划的时间和能耗代价,考察了不同优化目标在不同硬件条件下对查询处理的影响. 实验表明,传统硬件下面向性能的优化与面向能耗的优化结果是一致的;在新硬件条件下,两者结果则不同,可以改进数据库系统能效.     

一种适合多数据库系统的查询表示方法

1 引言随着分布计算和网络技术的不断发展,传统的数据库技术已越来越不能满足数据共享和互操作的需要。同时,已有的数据库系统又不可能全部丢弃.因而研制能同时访问和处理来自多个数据库中数据的多数据库系统已成为必然趋势。多数据库系统是解决已存的、异构的、分布的多个局部数据库系统之间数据共享和集成的问题。由于多数据库系统具有异构性、分布性和局部自治性的特点,使得多数据库查询处理与传统数据库查询处理有很大的不同。多数据库系统呈现给用户的是全局模式,用户使用全局查询语言提交对多数据库的查询,而所需的数据又必须从各局部数据库获得,所以必须将全局查询转换成与局部数据库对应的局部查询。在全局查询转换为局部查询的过程中,需要     

基于数据库负载的SQL能耗预测模型

传统数据库以性能(吞吐量、响应时间)为首要优化目标,忽略了数据库系统的能量消耗。在一味追求性能的同时,高能耗问题日益突出,为数据库负载构建能耗模型是构建绿色数据库的基础。通过量化查询负载执行过程中对系统资源(CPU与磁盘)的消耗,将资源消耗产生的时间代价和功耗代价转化为时间代价预测模型和功率代价预测模型,在单站点数据库服务器上实现了为数据库系统构建资源单位代价统一的能耗预测模型。采用多元线性回归工具拟合模型的重要参数,实验结果验证了能耗预测模型的可行性;并分别在静态与动态的系统环境下对系统不同类型查询负载的能耗进行预测与评价,验证了该模型的准确性,使得提出的能耗模型适合于构建能耗感知的绿色数据库。     

基于CPUfreq的DVFS节能技术的研究与实现

针对目前计算机系统普遍存在的功耗较大的问题,研究和实现了基于CPUfreq的DVFS节能软件;首先,分析和比较了计算机系统现有主要的功耗管理框架;然后,阐述了CPUfreq子系统及其框架结构,并基于CPUfreq子系统开发了DVFS节能软件,实现了对计算机CPU的动态电压频率调节(DVFS);最后,以720 p视频播放为应用实例,对使用DVFS节能软件后的计算机系统进行功耗测试;测试结果表明,DVFS节能软件可以实现对以PC为代表的计算机系统的初步节能。     

基于任务行为分析的DVFS机制

动态电压频率缩放（DVFS）技术是当前最有效的功耗调节手段之一．本文首先分析现有DVFS技术存在的不足,指出限制DVFS技术高效运用的核心因素;基于现有低效的方式我们提出一种基于任务行为分析的DVFS机制（TC-DVFS）．其具有三个层次：一、采集任务的系统调用信息;二、识别任务的关键系统调用,并以关键系统调用刻画任务行为;三、根据任务行为构建特征库,并以任务的特征库来指导DVFS．我们将TC-DVFS添加到linux内核中,并在intel-core2处理器平台上对不同类型的应用任务进行性能与功耗测试．结果显示TC-DVFS总体获得10％的性能提升,并降低5％调频失效率和5％的系统能耗．     

基于DVFS感知与虚拟机动态合并的云数据中心能效策略

为了解决云数据中心资源分配时能耗与性能间的均衡问题,提出了一种基于DVFS感知与虚拟机动态合并的能效优化策略。首先,策略通过新的DVFS管理算法（DVFS-perf）在不降低系统性能的同时降低了数据中心功耗,然后,通过频率感知的虚拟机VM部署合并算法（Frequency-aware Placement）在实现DVFS最优配置的同时最小化总体能耗,同时确保了虚拟机映射时的QoS保障。最后,通过真实云负载数据流构建仿真实验进行了性能分析。结果表明,在动态负载条件下,策略可以在不降低QoS和不增加SLA违例的情况下,降低虚拟机迁移次数和数据中心的总体能耗,更好地实现能耗与性能的均衡。     

交互式系统下基于任务重要性的DVFS技术

动态电压频率调节技术(DVFS)是从软件层面进行系统功耗管理的重要技术。本论文针对交互式系统的特点,首先分析了当前使用的 DVFS 策略的不足之处,然后提出并实现了一种适用于交互式系统的 DVFS 策略。该策略在保障用户体验的前提下对移动设备进行功耗优化。实验结果证明,对于大多数应用能达到10%以上的功耗优化效果,部分应用最高有超过30%的功耗降低。     

适用于云计算的面向查询数据库数据分布策略

为满足海量数据的处理需求,业界提出了多种解决方案.云计算是目前较为热门的一种,它主要用廉价PC组成超大规模集群服务器来进行数据存储和处理.随着云计算技术的发展,越来越多的应用将转移到云中,数据库系统也不例外.但数据库系统要求的ACID特性在数据分布存储时可能导致部分操作性能低下,如连接查询操作.为在数据分布存储下提高数据库系统的性能,提出了一种面向查询的数据分布策略(Selection Oriented Distribution,SOD),即根据数据库的查询情况确定数据的分布算法.该算法适用于云计算,能明显提高系统的查询性能.     

分布式数据库系统的查询优化技术研究

数据库数据量日益增多,造成了用户在使用数据库系统查询时费时费力,传统的查询优化方式已无法满足如今的数据查询要求,提高数据库系统优化的效率也成为计算机研究工作的热点。提出基于半连接算法的分布式查询处理技术对数据库系统进行查询优化,提出半连接操作的查询优化算法(SDD-1),并采用实验分析的方法进行验证,计算查询算法的代价。结果表明,基于半连接的研究策略的分布式数据库查询优化可以显著降低传输代价,使查询总效率得到有效提高。     

并行数据操作算法和查询优化技术

本文是并行数据库的查询处理并行化技术和物理设计方法”一文的续篇，继续综述并行数据库系统的另外两个重要研究领域：并行数据操作算法和并行数据库查询优化技术．最后，作为并行数据库系统研究与进展情况综述的结尾，本文将探讨并行数据库系统今后的研究方向和问题．     

Governing energy consumption in Hadoop through CPU frequency scaling: An analysis

With increasingly inexpensive storage and growing processing power, the cloud has rapidly become the environment of choice to store and analyze data for a variety of applications. Most large-scale data computations in the cloud heavily rely on the MapReduce paradigm and on its Hadoop implementation. Nevertheless, this exponential growth in popularity has significantly impacted power consumption in cloud infrastructures. In this paper, we focus on MapReduce processing and we investigate the impact of dynamically scaling the frequency of compute nodes on the performance and energy consumption of a Hadoop cluster. To this end, a series of experiments are conducted to explore the implications of Dynamic Voltage and Frequency Scaling (DVFS) settings on power consumption in Hadoop clusters. By enabling various existing DVFS governors (i.e., performance, powersave, ondemand, conservative and userspace) in a Hadoop cluster, we observe significant variation in performance and power consumption across different applications: the different DVFS settings are only sub-optimal for several representative MapReduce applications. Furthermore, our results reveal that the current CPU governors do not exactly reflect their design goal and may even become ineffective to manage the power consumption in Hadoop clusters. This study aims at providing a clearer understanding of the interplay between performance and power management in Hadoop clusters and therefore offers useful insight into designing power-aware techniques for Hadoop systems.     

User-Driven Frequency Scaling

We propose and evaluate user-driven frequency scaling (UDFS) for improved power management on processors that support dynamic voltage and frequency scaling (DVFS), e.g, those used in current laptop and desktop computers. UDFS dynamically adapts CPU frequency to the individual user and the workload through a simple user feedback mechanism, unlike currently-used DVFS methods which rely only on CPU utilization. Our UDFS algorithms dramatically reduce typical operating frequencies while maintaining performance at satisfactory levels for each user. We evaluated our techniques through user studies conducted on a Pentium M laptop running Windows applications. The UDFS scheme reduces measured system power by 22.1%, averaged across all our users and applications, compared to the Windows XP DVFS scheme     

Understanding the future of energy-performance trade-off via DVFS in HPC environments

DVFS is a ubiquitous technique for CPU power management in modern computing systems. Reducing processor frequency/voltage leads to a decrease of CPU power consumption and an increase in the execution time. In this paper, we analyze which application/platform characteristics are necessary for a successful energy-performance trade-off of large scale parallel applications. We present a model that gives an upper bound on performance loss due to frequency scaling using the application parallel efficiency. The model was validated with performance measurements of large scale parallel applications. Then we track how application sensitivity to frequency scaling evolved over the last decade for different cluster generations. Finally, we study how cluster power consumption characteristics together with application sensitivity to frequency scaling determine the energy effectiveness of the DVFS technique.     

Comparing columnar,row and array DBMSs to process recursive queries on graphs

Analyzing graphs is a fundamental problem in big data analytics, for which DBMS technology does not seem competitive. On the other hand, SQL recursive queries are a fundamental mechanism to analyze graphs in a DBMS, whose processing and optimization are significantly harder than traditional SPJ queries. Columnar DBMSs are a new faster class of database system, with significantly different storage and query processing mechanisms compared to row DBMSs, still the dominating technology. With that motivation in mind, we study the optimization of recursive queries on a columnar DBMS focusing on two fundamental and complementary graph problems: transitive closure and adjacency matrix multiplication. From a query processing perspective we consider the three fundamental relational operators: selection, projection and join (SPJ), where projection subsumes SQL group-by aggregation. We present comprehensive experiments comparing recursive query processing on columnar, row and array DBMSs to analyze large graphs with different shape and density. We study the relative impact of query optimizations and we compare raw speed of DBMSs to evaluate recursive queries on graphs. Results confirm classical query optimizations that keep working well in a columnar DBMS, but their relative impact is different. Most importantly, a columnar DBMS with tuned query optimization is uniformly faster than row and array systems to analyze large graphs, regardless of their shape, density and connectivity. On the other hand, there is no clear winner between the row and array DBMSs.     

SQL能耗建模及优化研究

IT系统能耗的节节攀升,使得设计新一代DBMS时必须考虑其能耗效率问题。由于SQL语句的执行过程大约消耗70%~90%的数据库资源,因此对SQL进行能耗建模及优化对提高数据库的能源使用效率具有重要的意义。在对SQL查询处理机制进行研究的基础上,构建了SQL能耗模型,并对一系列查询优化原则进行了实验,以表明不同优化原则对性能提升及能耗减少的有效性。实验及能耗数据分析表明:CPU利用率是影响系统功耗的最关键因素,SQL能耗优化方法可忽略内存优化且应该均衡考虑性能优化及功耗优化两方面,提出的SQL能耗模型及节能优化方法具有较强的应用价值。     

A framework for testing DBMS features

Testing a specific feature of a DBMS requires controlling the inputs and outputs of the operators in the query execution plan. However, that is practically difficult to achieve because the inputs/outputs of a query depend on the content of the test database. In this paper, we propose a framework to test DBMS features. The framework includes a database generator called QAGen so that the generated test databases are able to meet the test requirements defined on the test queries. The framework also includes a set of tools to automate test case constructions and test executions. A wide range of DBMS feature testing tasks can be facilitated by the proposed framework.     

Achieving energy efficiency during collective communications

Energy consumption has become a major design constraint in modern computing systems. With the advent of petaflops architectures, power‐efficient software stacks have become imperative for scalability. Techniques such as dynamic voltage and frequency scaling (called DVFS) and CPU clock modulation (called throttling) are often used to reduce the power consumption of the compute nodes. To avoid significant performance losses, these techniques should be used judiciously during parallel application execution. For example, its communication phases may be good candidates to apply the DVFS and CPU throttling without incurring a considerable performance loss. They are often considered as indivisible operations although little attention is being devoted to the energy saving potential of their algorithmic steps. In this work, two important collective communication operations, all‐to‐all and allgather, are investigated as to their augmentation with energy saving strategies on the per‐call basis. The experiments prove the viability of such a fine‐grain approach. They also validate a theoretical power consumption estimate for multicore nodes proposed here. While keeping the performance loss low, the obtained energy savings were always significantly higher than those achieved when DVFS or throttling were switched on across the entire application run. Copyright © 2012 John Wiley & Sons, Ltd.     

海量结构化数据存储管理系统OceanBase

数据库是现代社会十分关键的基础设施,一直以来,数据库的扩展以纵向模式(Scale-up)为主,数据库系统的容量和性能更多依赖于服务器单机硬件(CPU/内存/磁盘/网络等) 的提升,难以满足当今信息化社会对海量结构化数据的高性能、低成本的存储和处理能力的需求。本文设计和实现了一个基于横向扩展(Scale-out)模式的数据库系统 OceanBase,该系统支持事务(ACID)和范围查询等关系数据库和SQL语言的主要功能分库分表不再需要,服务器可以在线地添加和移除。OceanBase已用于阿里巴巴的多个线上系统,每天提供数十亿次的实时读写访问服务。     

Automatic runtime frequency-scaling system for energy savings in parallel applications

Although high-performance computing has always been about efficient application execution, both energy and power consumption have become critical concerns owing to their effect on operating costs and failure rates of large-scale computing platforms. Modern processors provide techniques, such as dynamic voltage and frequency scaling (DVFS) and CPU clock modulation (called throttling), to improve energy efficiency on-the-fly. Without careful application, however, DVFS and throttling may cause a significant performance loss due to system overhead. This paper proposes a novel runtime system that maximizes energy saving by selecting appropriate values for DVFS and throttling in parallel applications. Specifically, the system automatically predicts communication phases in parallel applications and applies frequency scaling considering both the CPU offload, provided by the network-interface card, and the architectural stalls during computation. Experiments, performed on NAS parallel benchmarks as well as on real-world applications in molecular dynamics and linear system solution, demonstrate that the proposed runtime system obtaining energy savings of as much as 14 % with a low performance loss of about 2 %.