Shared-nothing并行数据库系统查询优化技术

查询优化是并行数据库系统的核心技术。该文介绍作者自行研制的一个Ｓｈａｒｅｄ－ｎｏｔｈｉｎｇ并行数据库系统ＰＢＡＳＥ／２中独特的两阶段优化策略。为了缩减并行相称优化庞大的搜索空间,ＰＢＡＳＥ／２将并行查询优化划分为顺序优化和并行化两个在阶段。在顺序优化阶段对并行化后的通信代价进行预先估算,将通信开销加入顺序优化的代价模型,同时对动态规划搜索算法进行了修正和扩展,保证了顺序优化阶段得到的最小代价计划在     

Accomplishing Deterministic XML Query Optimization

As the popularity of XML (extensible Markup Language) keeps growing rapidly, the management of XML compliant structured-document databases has become a very interesting and compelling research area. Query optimization for XML structured-documents stands out as one of the most challenging research issues in this area because of the much enlarged optimization (search) space, which is a consequence of the intrinsic complexity of the underlying data model of XML data. We therefore propose to apply deterministic transformations on query expressions to most aggressively prune the search space and fast achieve a sufficiently improved alternative (if not the optimal) for each incoming query expression. This idea is not just exciting but practically attainable. This paper first provides an overview of our optimization strategy, and then focuses on the key implementation issues of our rule-based transformation system for XML query optimization in a database environment. The performance results we obtained from experimentation show that our approach is a valid and effective one.     

基于LBT树的并行数据库查询优化方法

本文提出了一种基于LBT(Linear-Bushy-Tree)树的查询优化方法，它是对基于浓密树(Bushy-Tree)查询优化方法的一种改进。这种优化方法大大地缩减了查询执行计划空间，确保了并行查询执行计划的优化性。这种优化方法已经在我们自主研制的并行数据库管理系统PDBMS^[1,4]中得到实现。     

并行数据库系统PBASE/2的查询优化机制

并行执行计划的搜索空间呈指数级增长,如何高效地裁剪搜索空间是并行查询优化的关键所在,ＰＢＡＳＥ／２以流机制为基础,采用基于代价估算和启发式规则的改良的两阶段优化,有交地解决了这一问题。     

基于禁忌GEP的分布式数据库查询优化算法

为了更好地解决分布式查询优化问题,论文在传统GEP算法的基础上,结合禁忌搜索策略,提出了基于禁忌GEP的分布式数据库查询优化算法（DistributeddatabasequeryoptimizationalgorithmbaseduponTabu-GEP,DDQO-TGEP）。仿真实验表明,随着查询关系个数的增加,DDQO-TGEP算法执行查询时所需要的查询优化时间和最优查询策略生成时间都比传统的GA和GP算法显著下降,其中查询优化时间最大下降约42．16％,最优查询策略生成时间最大下降约36．8％。     

一个并行查询优化器的设计与实现

并行查询优化器的目标是缩减庞大的计划搜索空间,获得优化的查询规划。为此,并行实时数据库PRTD-BASE查询优化器针对无共享结构（SN）,充分考虑通信开销,采用两阶段 优化方法,依据代价估计模型先对查询树进行基于代价估计的顺序优化,然后利用启发式规则对顺序优化的查询计划进行并行化,充分利用了多处理机的并行性,获得了较快的查 询响应时间。     

列存储数据查询中的连接策略选择方法

列的连接策略优化是列存储数据查询中的重要问题。现有的列存储系统中,列的连接存在策略单一,缺少优化处理,无法满足复杂查询等缺陷。针对这些问题,提出一种连接策略选择方法。该方法首先定义简单规则过滤代价过大的查询计划,生成候选查询计划树。进而根据动态Huffman树原理提出动态优化树算法,对候选查询计划树中的查询执行顺序进行改进。根据列存储数据的特点,候选计划中每个连接节点的执行策略被归纳为两种:串行连接和并行连接。在此基础上构建代价估计模型,集中针对这两种连接策略进行代价估计和策略选择,从而以较小的时间复杂度获得优化的查询执行策略。     

Efficient processing of label-constraint reachability queries in large graphs

In this paper, we study a variant of reachability queries, called label-constraint reachability (LCR) queries. Specifically, given a label set S and two vertices u₁ and u₂ in a large directed graph G, we check the existence of a directed path from u₁ to u₂, where edge labels along the path are a subset of S. We propose the path-label transitive closure method to answer LCR queries. Specifically, we t4ransform an edge-labeled directed graph into an augmented DAG by replacing the maximal strongly connected components as bipartite graphs. We also propose a Dijkstra-like algorithm to compute path-label transitive closure by re-defining the “distance” of a path. Comparing with the existing solutions, we prove that our method is optimal in terms of the search space. Furthermore, we propose a simple yet effective partition-based framework (local path-label transitive closure+online traversal) to answer LCR queries in large graphs. We prove that finding the optimal graph partition to minimize query processing cost is a NP-hard problem. Therefore, we propose a sampling-based solution to find the sub-optimal partition. Moreover, we address the index maintenance issues to answer LCR queries over the dynamic graphs. Extensive experiments confirm the superiority of our method.     

基于多重加权树的并行数据库查询优化方法

本文提出了一种基于多重加权树的查询优化方法，包括多重加权树并行查询计划模型、并行查询计划的复杂性模型和查询优化处工法。     

Similarity query support in big data management systems

Similarity query processing is becoming increasingly important in many applications such as data cleaning, record linkage, Web search, and document analytics. In this paper we study how to provide end-to-end similarity query support natively in a parallel database system. We discuss how to express a similarity predicate in its query language, how to build indexes, how to answer similarity queries (selections and joins) efficiently in the runtime engine, possibly using indexes, and how to optimize similarity queries. One particular challenge is how to incorporate existing similarity join algorithms, which often require a series of steps to achieve a high efficiency, including collecting token frequencies, finding matching record id pairs, and reassembling result records based on id pairs. We present a novel approach that uses existing runtime operators to implement such complex join algorithms without reinventing the wheel; doing so positions the system to automatically benefit from future improvements to those operators. The approach includes a technique to transform a similarity join plan into an efficient operator-based physical plan during query optimization by using a template expressed largely in the system’s user-level query language; this technique greatly simplifies the specification of such a transformation rule. We use Apache AsterixDB, a parallel Big Data management system, to illustrate and validate our techniques. We conduct an experimental study using several large, real datasets on a parallel computing cluster to assess the similarity query support. We also include experiments involving three other parallel systems and report the efficacy and performance results.     

A novel two-archive strategy for evolutionary many-objective optimization algorithm based on reference points

Current evolutionary many-objective optimization algorithms face two challenges: one is to ensure population diversity for searching the entire solution space. The other is to ensure quick convergence to the optimal solution set. In this paper, we propose a novel two-archive strategy for evolutionary many-objective optimization algorithm. The uniform archive strategy, based on reference points, is used to keep population diversity in the evolutionary process, and to ensure that an evolutionary algorithm is able to search the entire solution space. The single elite archive strategy is used to ensure that individuals with the best single objective value are able to evolve into the next generation and have more opportunities to generate offspring. This strategy aims to improve the convergence rate. Then this novel two-archive strategy is applied to improving the Non-dominated Sorting Genetic Algorithm (NSGA-III). Simulation experiments are conducted on benchmark test sets and experimental results show that our proposed algorithm with the two-archive strategy has a better performance than other state-of-art algorithms.     

A hybrid approach using genetic and fruit fly optimization algorithms for QoS-aware cloud service composition

This paper addresses the QoS-aware cloud service composition problem, which is known as a NP-hard problem, and proposes a hybrid genetic algorithm (HGA) to solve it. The proposed algorithm combines two phases to perform the evolutionary process search, including genetic algorithm phase and fruit fly optimization phase. In genetic algorithm phase, a novel roulette wheel selection operator is proposed to enhance the efficiency and the exploration search. To reduce the computation time and to maintain a balance between the exploration and exploitation abilities of the proposed HGA, the fruit fly optimization phase is incorporated as a local search strategy. In order to speed-up the convergence of the proposed algorithm, the initial population of HGA is created on the basis of a heuristic local selection method, and the elitism strategy is applied in each generation to prevent the loss of the best solutions during the evolutionary process. The parameter settings of our HGA were tuned and calibrated using the taguchi method of design of experiment, and we suggested the optimal values of these parameters. The experimental results show that the proposed algorithm outperforms the simple genetic algorithm, simple fruit fly optimization algorithm, and another recently proposed algorithm (DGABC) in terms of optimality, computation time, convergence speed and feasibility rate.     

基于Greenplum数据库的查询优化

针对分布式数据库查询效率随着数据规模的增大而降低的问题,以Greenplum分布式数据库为研究对象,从优化查询路径的角度提出一个基于代价的最优查询计划生成方法。首先,该方法设计一种有效的代价模型来估算查询代价;然后,采用并行最大最小蚁群算法来搜索具有最小查询代价的连接顺序,即最优连接顺序;最后,根据Greenplum数据库对查询计划中不同操作的默认最优选择得到最优查询计划。采用该方法在自主生成的数据集与事务处理性能理事会测试基准（TPC-H）的标准数据集上进行了多组实验。实验结果表明,所提出的优化方法能有效地搜索出最优解,获得最优的查询计划,从而提升Greenplum数据库的查询效率。     

Runtime Shader Simplification via Instant Search in Reduced Optimization Space

Traditional automatic shader simplification simplifies shaders in an offline process, which is typically carried out in a context‐oblivious manner or with the use of some example contexts, e.g., certain hardware platforms, scenes, and uniform parameters, etc. As a result, these pre‐simplified shaders may fail at adapting to runtime changes of the rendering context that were not considered in the simplification process. In this paper, we propose a new automatic shader simplification technique, which explores two key aspects of a runtime simplification framework: the optimization space and the instant search for optimal simplified shaders with runtime context. The proposed technique still requires a preprocess stage to process the original shader. However, instead of directly computing optimal simplified shaders, the proposed preprocess generates a reduced shader optimization space. In particular, two heuristic estimates of the quality and performance of simplified shaders are presented to group similar variants into representative ones, which serve as basic graph nodes of the simplification dependency graph (SDG), a new representation of the optimization space. At the runtime simplification stage, a parallel discrete optimization algorithm is employed to instantly search in the SDG for optimal simplified shaders. New data‐driven cost models are proposed to predict the runtime quality and performance of simplified shaders on the basis of data collected during runtime. Results show that the selected simplifications of complex shaders achieve 1.6 to 2.5 times speedup and still retain high rendering quality.     

一种自底向上的静态语义查询优化算法

演绎数据库语义查询优化是运用数据库中的语义知识，即完整性约束条件，将用户提交的一种查询转换为能有效执行，并与原查询等价的查询的一种优化方法．至今在这一领域已有了许多的算法，但大多是基于自顶向下的查询计算模式．而本文提出的静态语义查询优化算法及其改进算法是在优化“并”和“连接”操作的过程中进行自底向上的查询计算，因此相对自顶向下的计算方式更有效地提高了查询执行效率．     

基于查询优化器的分布式空间查询优化方法

为了实现分布式空间数据库之间的互操作,需要对分布式查询进行优化处理,这种查询处理指的是在任何一个数据处理语句中它访问的是各个节点的数据而不是仅仅对发起查询的节点。提出了一种查询优化器的体系结构,针对上述查询最优化做了详细的讨论,着重讨论包含空间选择和连接的复杂空间查询。建立了典型的空间数据库的案例程序,通过分析表明,带有过滤和修正的查询优化器在时间与空间上的效率优势比较明显,获得了具有参考价值的结果。     

Hybridizing electromagnetism-like mechanism algorithm with migration strategy for layout and size optimization of truss structures with frequency constraints

The optimal design of a truss structure with dynamic frequency constraints is a highly nonlinear optimization problem with several local optimums in its search space. In this type of structural optimization problems, the optimization methods should have a high capability to escape from the traps of the local optimums in the search space. This paper presents hybrid electromagnetism-like mechanism algorithm and migration strategy (EM–MS) for layout and size optimization of truss structures with multiple frequency constraints. The electromagnetism-like mechanism (EM) algorithm simulates the attraction and repulsion mechanism between the charged particles in the field of the electromagnetism to find optimal solutions, in which each particle is a solution candidate for the optimization problem. In the proposed EM–MS algorithm, two mechanisms are utilized to update the position of particles: modified EM algorithm and a new migration strategy. The modified EM algorithm is proposed to effectively guide the particles toward the region of the global optimum in the search space, and a new migration strategy is used to provide efficient exploitation between the particles. In order to test the performance of the proposed algorithm, this study utilizes five benchmark truss design examples with frequency constraints. The numerical results show that the EM–MS algorithm is an alternative and competitive optimizer for size and layout optimization of truss structures with frequency constraints.     

Evolutionary Fuzzy‐based gravitational search algorithm for query optimization in crowdsourcing system to minimize cost and latency

Crowdsourcing is an environment where a group of users collaborates together to exchange information and to find answers for complex problems (queries). Query optimization is the task of selecting the best query strategy with less cost associated with it. The crowdsourcing cost can be determined by selecting the best plan from the set of options available and the best plan considerably reduce the cost for the inquiry configuration. As one of the center tasks in information recovery, the investigation of top‐k queries with crowdsourcing, to be specific group empowered top k inquiries is depicted. This issue is defined with three key variables, latency, money related expense, and nature of answers. The fundamental point is to plan a novel system that limits financial cost when the latency is compelled. In this article, we used a heuristic search algorithm named as Evolutionary Fuzzy‐based Gravitational Search algorithm (EFGSA) that produces an optimal query feature selection results with minimizing cost and latency. EFGSA‐based crowdsourcing framework gives a better balance between latency and cost while generating query plans. The performance analysis of proposed EFSGA for optimal query plan is evaluated in terms of running time, accuracy, monetary cost, and so on. From the experimental results, the proposed method achieved better results than other methods in our cost and latency model.     

An adaptable distributed query processing architecture

Traditionally, distributed query optimization techniques generate static query plans at compile time. However, the optimality of these plans depends on many parameters (such as the selectivities of operations, the transmission speeds and workloads of servers) that are not only difficult to estimate but are also often unpredictable and fluctuant at runtime. As the query processor cannot dynamically adjust the plans at runtime, the system performance is often less than satisfactory. In this paper, we introduce a new highly adaptive distributed query processing architecture. Our architecture can quickly detect fluctuations in selectivities of operations, as well as transmission speeds and workloads of servers, and accordingly change the operation order of a distributed query plan during execution. We have implemented a prototype based on the Telegraph system [Telegragraph project. Available from >]. Our experimental study shows that our mechanism can adapt itself to the changes in the environment and hence approach to an optimal plan during execution.     

Multi-Objective region-Aware optimization of parallel programs

Auto-tuning has become increasingly popular for optimizing non-functional parameters of parallel programs. The typically large search space requires sophisticated techniques to find well-performing parameter values in a reasonable amount of time. Different parts of a program often perform best with different parameter values. We therefore subdivide programs into several regions, and try to optimize the parameter values for each of these regions separately as opposed to setting the parameter values globally for the entire program. In order to manage this enlarged search space, we have to extend existing auto-tuning techniques to ensure high quality solutions to this optimization problem. In this paper we introduce a novel enhancement to the RS-GDE3 algorithm that is used to explore the search space for auto-tuning programs with multiple regions regarding several objectives. We have implemented our auto-tuner using the Insieme compiler and runtime system, and provide a detailed analysis of the obtained results with the aim of gaining a better understanding of non-functional inter-region behavior in the context of auto-tuning. In comparison to a non-optimized parallel version of the tested programs, our novel approach achieves improvements of up to 7.6X, 10.5X, and 61.6X for three tuned objectives wall time, energy consumption, and resource usage, respectively.