Bitwise dimensional co-clustering for analytical workloads

Analytical workloads in data warehouses often include heavy joins where queries involve multiple fact tables in addition to the typical star-patterns, dimensional grouping and selections. In this paper we propose a new processing and storage framework called bitwise dimensional co-clustering (BDCC) that avoids replication and thus keeps updates fast, yet is able to accelerate all these foreign key joins, efficiently support grouping and pushes down most dimensional selections. The core idea of BDCC is to cluster each table on a mix of dimensions, each possibly derived from attributes imported over an incoming foreign key and this way creating foreign key connected tables with partially shared clusterings. These are later used to accelerate any join between two tables that have some dimension in common and additionally permit to push down and propagate selections (reduce I/O) and accelerate aggregation and ordering operations. Besides the general framework, we describe an algorithm to derive such a physical co-clustering database automatically and describe query processing and query optimization techniques that can easily be fitted into existing relational engines. We present an experimental evaluation on the TPC-H benchmark in the Vectorwise system, showing that co-clustering can significantly enhance its already high performance and at the same time significantly reduce the memory consumption of the system.     

Hybrid query optimization for hard-to-compress bit-vectors

Bit-vectors are widely used for indexing and summarizing data due to their efficient processing in modern computers. Sparse bit-vectors can be further compressed to reduce their space requirement. Special compression schemes based on run-length encoders have been designed to avoid explicit decompression and minimize the decoding overhead during query execution. Moreover, highly compressed bit-vectors can exhibit a faster query time than the non-compressed ones. However, for hard-to-compress bit-vectors, compression does not speed up queries and can add considerable overhead. In these cases, bit-vectors are often stored verbatim (non-compressed). On the other hand, queries are answered by executing a cascade of bit-wise operations involving indexed bit-vectors and intermediate results. Often, even when the original bit-vectors are hard to compress, the intermediate results become sparse. It could be feasible to improve query performance by compressing these bit-vectors as the query is executed. In this scenario, it would be necessary to operate verbatim and compressed bit-vectors together. In this paper, we propose a hybrid framework where compressed and verbatim bitmaps can coexist and design algorithms to execute queries under this hybrid model. Our query optimizer is able to decide at run time when to compress or decompress a bit-vector. Our heuristics show that the applications using higher-density bitmaps can benefit from using this hybrid model, improving both their query time and memory utilization.     

Accelerating SPARQL queries by exploiting hash-based locality and adaptive partitioning

State-of-the-art distributed RDF systems partition data across multiple computer nodes (workers). Some systems perform cheap hash partitioning, which may result in expensive query evaluation. Others try to minimize inter-node communication, which requires an expensive data preprocessing phase, leading to a high startup cost. Apriori knowledge of the query workload has also been used to create partitions, which, however, are static and do not adapt to workload changes. In this paper, we propose AdPart, a distributed RDF system, which addresses the shortcomings of previous work. First, AdPart applies lightweight partitioning on the initial data, which distributes triples by hashing on their subjects; this renders its startup overhead low. At the same time, the locality-aware query optimizer of AdPart takes full advantage of the partitioning to (1) support the fully parallel processing of join patterns on subjects and (2) minimize data communication for general queries by applying hash distribution of intermediate results instead of broadcasting, wherever possible. Second, AdPart monitors the data access patterns and dynamically redistributes and replicates the instances of the most frequent ones among workers. As a result, the communication cost for future queries is drastically reduced or even eliminated. To control replication, AdPart implements an eviction policy for the redistributed patterns. Our experiments with synthetic and real data verify that AdPart: (1) starts faster than all existing systems; (2) processes thousands of queries before other systems become online; and (3) gracefully adapts to the query load, being able to evaluate queries on billion-scale RDF data in subseconds.     

External sorting on flash storage: reducing cell wearing and increasing efficiency by avoiding intermediate writes

This paper studies the problem of how to conduct external sorting on flash drives while avoiding intermediate writes to the disk. The focus is on sort in portable electronic devices, where relations are only larger than the main memory by a small factor, and on sort as part of distributed processes where relations are frequently partially sorted. In such cases, sort algorithms that refrain from writing intermediate results to the disk have three advantages over algorithms that perform intermediate writes. First, on devices in which read operations are much faster than writes, such methods are efficient and frequently outperform Merge Sort. Secondly, they reduce flash cell degradation caused by writes. Thirdly, they can be used in cases where there is not enough disk space for the intermediate results. Novel sort algorithms that avoid intermediate writes to the disk are presented. An experimental evaluation, on different flash storage devices, shows that in many cases the new algorithms can extend the lifespan of the devices by avoiding unnecessary writes to the disk, while maintaining efficiency, in comparison with Merge Sort.     

A Closed-Loop Automatic Data-Mining Approach for Preventive Network Monitoring

As telecommunication networks grow in size and complexity, monitoring systems need to scale up accordingly. Alarm data generated in a large network are often highly correlated. These correlations can be explored to simplify the process of network fault management, by reducing the number of alarms presented to the network-monitoring operator. This makes it easier to react to network failures. But in some scenarios, it is highly desired to prevent the occurrence of these failures by predicting the occurrence of alarms before hand. This work investigates the usage of data mining methods to generate knowledge from historical alarm data, and using such knowledge to train a machine learning system, in order to predict the occurrence of the most relevant alarms in the network. The learning system was designed to be retrained periodically in order to keep an updated knowledge base.     

Rotation,scaling and translation invariant texture recognition by Bessel-Fourier moments

The ideal of Bessel-Fourier moments (BFMs) for image analysis and only rotation invariant image cognition has been proposed recently. In this paper, we extend the previous work and propose a new method for rotation, scaling and translation (RST) invariant texture recognition using Bessel-Fourier moments. Compared with the others moments based methods, the radial polynomials of Bessel-Fourier moments have more zeros and these zeros are more evenly distributed. It makes Bessel-Fourier moments more suitable for invariant texture recognition as a generalization of orthogonal complex moments. In the experiment part, we got three testing sets of 16, 24 and 54 texture images by way of translating, rotating and scaling them separately. The correct classification percentages (CCPs) are compared with that of orthogonal Fourier-Mellin moments and Zernike moments based methods in both noise-free and noisy condition. Experimental results validate the conclusion of theoretical derivation: BFM performs better in recognition capability and noise robustness in terms of RST texture recognition under both noise-free and noisy condition when compared with orthogonal Fourier-Mellin moments and Zernike moments based methods.     

Topology Optimization in Aircraft and Aerospace Structures Design

Topology optimization has become an effective tool for least-weight and performance design, especially in aeronautics and aerospace engineering. The purpose of this paper is to survey recent advances of topology optimization techniques applied in aircraft and aerospace structures design. This paper firstly reviews several existing applications: (1) standard material layout design for airframe structures, (2) layout design of stiffener ribs for aircraft panels, (3) multi-component layout design for aerospace structural systems, (4) multi-fasteners design for assembled aircraft structures. Secondly, potential applications of topology optimization in dynamic responses design, shape preserving design, smart structures design, structural features design and additive manufacturing are introduced to provide a forward-looking perspective.     

Non-intrusive Coupling: Recent Advances and Scalable Nonlinear Domain Decomposition

This paper provides a detailed review of the global/local non-intrusive coupling algorithm. Such method allows to alter a global finite element model, without actually modifying its corresponding numerical operator. We also look into improvements of the initial algorithm (Quasi-Newton and dynamic relaxation), and provide comparisons based on several relevant test cases. Innovative examples and advanced applications of the non-intrusive coupling algorithm are provided, granting a handy framework for both researchers and engineers willing to make use of such process. Finally, a novel nonlinear domain decomposition method is derived from the global/local non-intrusive coupling strategy, without the need to use a parallel code or software. Such method being intended to large scale analysis, we show its scalability. Jointly, an efficient high level Message Passing Interface coupling framework is also proposed, granting an universal and flexible way for easy software coupling. A sample code is also given.