Accelerating SPARQL queries by exploiting hash-based locality and adaptive partitioning

State-of-the-art distributed RDF systems partition data across multiple computer nodes (workers). Some systems perform cheap hash partitioning, which may result in expensive query evaluation. Others try to minimize inter-node communication, which requires an expensive data preprocessing phase, leading to a high startup cost. Apriori knowledge of the query workload has also been used to create partitions, which, however, are static and do not adapt to workload changes. In this paper, we propose AdPart, a distributed RDF system, which addresses the shortcomings of previous work. First, AdPart applies lightweight partitioning on the initial data, which distributes triples by hashing on their subjects; this renders its startup overhead low. At the same time, the locality-aware query optimizer of AdPart takes full advantage of the partitioning to (1) support the fully parallel processing of join patterns on subjects and (2) minimize data communication for general queries by applying hash distribution of intermediate results instead of broadcasting, wherever possible. Second, AdPart monitors the data access patterns and dynamically redistributes and replicates the instances of the most frequent ones among workers. As a result, the communication cost for future queries is drastically reduced or even eliminated. To control replication, AdPart implements an eviction policy for the redistributed patterns. Our experiments with synthetic and real data verify that AdPart: (1) starts faster than all existing systems; (2) processes thousands of queries before other systems become online; and (3) gracefully adapts to the query load, being able to evaluate queries on billion-scale RDF data in subseconds.     

External sorting on flash storage: reducing cell wearing and increasing efficiency by avoiding intermediate writes

This paper studies the problem of how to conduct external sorting on flash drives while avoiding intermediate writes to the disk. The focus is on sort in portable electronic devices, where relations are only larger than the main memory by a small factor, and on sort as part of distributed processes where relations are frequently partially sorted. In such cases, sort algorithms that refrain from writing intermediate results to the disk have three advantages over algorithms that perform intermediate writes. First, on devices in which read operations are much faster than writes, such methods are efficient and frequently outperform Merge Sort. Secondly, they reduce flash cell degradation caused by writes. Thirdly, they can be used in cases where there is not enough disk space for the intermediate results. Novel sort algorithms that avoid intermediate writes to the disk are presented. An experimental evaluation, on different flash storage devices, shows that in many cases the new algorithms can extend the lifespan of the devices by avoiding unnecessary writes to the disk, while maintaining efficiency, in comparison with Merge Sort.     

A Closed-Loop Automatic Data-Mining Approach for Preventive Network Monitoring

As telecommunication networks grow in size and complexity, monitoring systems need to scale up accordingly. Alarm data generated in a large network are often highly correlated. These correlations can be explored to simplify the process of network fault management, by reducing the number of alarms presented to the network-monitoring operator. This makes it easier to react to network failures. But in some scenarios, it is highly desired to prevent the occurrence of these failures by predicting the occurrence of alarms before hand. This work investigates the usage of data mining methods to generate knowledge from historical alarm data, and using such knowledge to train a machine learning system, in order to predict the occurrence of the most relevant alarms in the network. The learning system was designed to be retrained periodically in order to keep an updated knowledge base.     

Rotation,scaling and translation invariant texture recognition by Bessel-Fourier moments

The ideal of Bessel-Fourier moments (BFMs) for image analysis and only rotation invariant image cognition has been proposed recently. In this paper, we extend the previous work and propose a new method for rotation, scaling and translation (RST) invariant texture recognition using Bessel-Fourier moments. Compared with the others moments based methods, the radial polynomials of Bessel-Fourier moments have more zeros and these zeros are more evenly distributed. It makes Bessel-Fourier moments more suitable for invariant texture recognition as a generalization of orthogonal complex moments. In the experiment part, we got three testing sets of 16, 24 and 54 texture images by way of translating, rotating and scaling them separately. The correct classification percentages (CCPs) are compared with that of orthogonal Fourier-Mellin moments and Zernike moments based methods in both noise-free and noisy condition. Experimental results validate the conclusion of theoretical derivation: BFM performs better in recognition capability and noise robustness in terms of RST texture recognition under both noise-free and noisy condition when compared with orthogonal Fourier-Mellin moments and Zernike moments based methods.     

Topology Optimization in Aircraft and Aerospace Structures Design

Topology optimization has become an effective tool for least-weight and performance design, especially in aeronautics and aerospace engineering. The purpose of this paper is to survey recent advances of topology optimization techniques applied in aircraft and aerospace structures design. This paper firstly reviews several existing applications: (1) standard material layout design for airframe structures, (2) layout design of stiffener ribs for aircraft panels, (3) multi-component layout design for aerospace structural systems, (4) multi-fasteners design for assembled aircraft structures. Secondly, potential applications of topology optimization in dynamic responses design, shape preserving design, smart structures design, structural features design and additive manufacturing are introduced to provide a forward-looking perspective.     

Non-intrusive Coupling: Recent Advances and Scalable Nonlinear Domain Decomposition

This paper provides a detailed review of the global/local non-intrusive coupling algorithm. Such method allows to alter a global finite element model, without actually modifying its corresponding numerical operator. We also look into improvements of the initial algorithm (Quasi-Newton and dynamic relaxation), and provide comparisons based on several relevant test cases. Innovative examples and advanced applications of the non-intrusive coupling algorithm are provided, granting a handy framework for both researchers and engineers willing to make use of such process. Finally, a novel nonlinear domain decomposition method is derived from the global/local non-intrusive coupling strategy, without the need to use a parallel code or software. Such method being intended to large scale analysis, we show its scalability. Jointly, an efficient high level Message Passing Interface coupling framework is also proposed, granting an universal and flexible way for easy software coupling. A sample code is also given.     

3D model reconstruction with common hand-held cameras

A 3D model reconstruction workflow with hand-held cameras is developed. The exterior and interior orientation models combined with the state-of-the-art structure from motion and multi-view stereo techniques are applied to extract dense point cloud and reconstruct 3D model from digital images. An overview of the presented 3D model reconstruction methods is given. The whole procedure including tie point extraction, relative orientation, bundle block adjustment, dense point production and 3D model reconstruction is all reviewed in brief. Among them, we focus on bundle block adjustment procedure; the mathematical and technical details of bundle block adjustment are introduced and discussed. Finally, four scenes of images collected by hand-held cameras are tested in this paper. The preliminary results have shown that sub-pixel (<1 pixel) accuracy can be achieved with the proposed exterior–interior orientation models and satisfactory 3D models can be reconstructed using images collected by hand-held cameras. This work can be applied in indoor navigation, crime scene reconstruction, heritage reservation and other applications in geosciences.