Efficient similarity-based operations for data integration

Dealing with discrepancies in data is still a big challenge in data integration systems. The problem occurs both during eliminating duplicates from semantic overlapping sources as well as during combining complementary data from different sources. Though using SQL operations like grouping and join seems to be a viable way, they fail if the attribute values of the potential duplicates or related tuples are not equal but only similar by certain criteria. As a solution to this problem, we present in this paper similarity-based variants of grouping and join operators. The extended grouping operator produces groups of similar tuples, the extended join combines tuples satisfying a given similarity condition. We describe the semantics of this operator, discuss efficient implementations for the edit distance similarity and present evaluation results. Finally, we give examples of application from the context of a data reconciliation project for looted art.     

Bitwise dimensional co-clustering for analytical workloads

Analytical workloads in data warehouses often include heavy joins where queries involve multiple fact tables in addition to the typical star-patterns, dimensional grouping and selections. In this paper we propose a new processing and storage framework called bitwise dimensional co-clustering (BDCC) that avoids replication and thus keeps updates fast, yet is able to accelerate all these foreign key joins, efficiently support grouping and pushes down most dimensional selections. The core idea of BDCC is to cluster each table on a mix of dimensions, each possibly derived from attributes imported over an incoming foreign key and this way creating foreign key connected tables with partially shared clusterings. These are later used to accelerate any join between two tables that have some dimension in common and additionally permit to push down and propagate selections (reduce I/O) and accelerate aggregation and ordering operations. Besides the general framework, we describe an algorithm to derive such a physical co-clustering database automatically and describe query processing and query optimization techniques that can easily be fitted into existing relational engines. We present an experimental evaluation on the TPC-H benchmark in the Vectorwise system, showing that co-clustering can significantly enhance its already high performance and at the same time significantly reduce the memory consumption of the system.     

Stream engines meet wireless sensor networks: cost-based planning and processing of complex queries in AnduIN

Wireless sensor networks are powerful, distributed, self-organizing systems used for event and environmental monitoring. In-network query processors like TinyDB offer a user friendly SQL-like application development. Due to the sensor nodes?? resource limitations, monolithic approaches often support only a restricted number of operators. For this reason, complex processing is typically outsourced to the base station. Nevertheless, previous work has shown that complete or partial in-network processing can be more efficient than the base station approach. In this paper, we introduce AnduIN, a system for developing, deploying, and running complex in-network processing tasks. In particular, we present the query planning and execution strategies used in AnduIN, a system combining sensor-local in-network processing and a data stream engine. Query planning employs a multi-dimensional cost model taking energy consumption into account and decides autonomously which query parts will be processed within the sensor network and which parts will be processed at the central instance.     

Evaluation of a low-cost 3D sound system for immersive virtual reality training systems

Since head mounted displays (HMD), datagloves, tracking systems, and powerful computer graphics resources are nowadays in an affordable price range, the usage of PC-based "virtual training systems" becomes very attractive. However, due to the limited field of view of HMD devices, additional modalities have to be provided to benefit from 3D environments. A 3D sound simulation can improve the capabilities of VR systems dramatically. Unfortunately, realistic 3D sound simulations are expensive and demand a tremendous amount of computational power to calculate reverberation, occlusion, and obstruction effects. To use 3D sound in a PC-based training system as a way to direct and guide trainees to observe specific events in 3D space, a cheaper alternative has to be provided, so that a broader range of applications can take advantage of this modality. To address this issue, we focus in this paper on the evaluation of a low-cost 3D sound simulation that is capable of providing traceable 3D sound events. We describe our experimental system setup using conventional stereo headsets in combination with a tracked HMD device and present our results with regard to precision, speed, and used signal types for localizing simulated sound events in a virtual training environment     

Diagenetic transformation of dissolved organic nitrogen compounds under contrasting sedimentary redox conditions in the Black Sea

Remineralization of organic matter in reactive marine sediments releases nutrients and dissolved organic matter (DOM) into the ocean. Here we focused on the molecular-level characterization of DOM by high-resolution Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) in sediment pore waters and bottom waters from contrasting redox regimes in the northern Black Sea with particular emphasis on nitrogen-bearing compounds to derive an improved understanding of the molecular transformations involved in nitrogen release. The number of nitrogen-bearing molecules is generally higher in pore waters than in bottom waters. This suggests intensified degradation of nitrogen-bearing precursor molecules such as proteins in anoxic sediments: No significant difference was observed between sediments deposited under oxic vs anoxic conditions (average O/C ratios of 0.55) suggesting that the different organic matter quality induced by contrasting redox conditions does not impact protein diagenesis in the subseafloor. Compounds in the pore waters were on average larger, less oxygenated, and had a higher number of unsaturations. Applying a mathematical model, we could show that the assemblages of nitrogen-bearing molecular formulas are potential products of proteinaceous material that was transformed by the following reactions: (a) hydrolysis and deamination, both reducing the molecular size and nitrogen content of the products and intermediates; (b) oxidation and hydration of the intermediates; and (c) methylation and dehydration.     

Aluminum control of phosphorus sorption by lake sediments

Release of reactive (phosphate-like) phosphorus (P) from freshwater sediments represents a significant internal P source for many lakes. Hypolimnetic P release occurs under reducing conditions that cause reductive dissolution of ferric hydroxide [Fe(OH)3]. This hypolimnetic P release may be naturally low or artificially reduced by sediment with naturally high or artificially elevated concentrations of aluminum hydroxide [Al(OH)3]. We presentfield and laboratory data for a common extraction analysis of sediments from 43 lakes differing in trophic status, pH regime, climate, and P loading. The results indicate that a simple sequential extraction of sediment may be a useful predictor of sediment's ability to release P. Sequential extractions of sediment P, Al, and Fe by water (H2O), bicarbonate-dithionite (BD), and NaOH (at 25 degrees C) showed that negligible amounts of P would be released from lake sediments during hypolimnetic anoxia if either (1) the molar Al(NaOH-25):Fe(BD) ratio is > 3 or (2) the molar Al(NaOH-25):P(H2O+BD) ratio is > 25. These ratios can be used as operational targets for estimation of sediment P release potential and Al dosing of P-rich sediment to prevent hypolimnetic P release under anoxic conditions.     

Capacities of membrane lipids to accumulate neutral organic chemicals

Lipids have been considered as the predominant components for bioaccumulation of organic chemicals. However, differences in accumulation properties between different types of lipid (e.g., storage and membrane lipids) have rarely been considered. Moreover, in view of toxic effects on organisms, chemical accumulation specifically in biological membranes is of particular importance. In this review article, partition coefficients of 240 neutral organic compounds between liposomes (phospholipid membrane vesicles) and water (K(lipw)), reported in the literature or measured additionally for this work, were evaluated. Values of log K(lipw) and log K(ow) (octanol-water partition coefficients) differ by 0.4 on average. Polyparameter linear free energy relationships (PP-LFERs) can describe the log K(lipw) data even better (standard deviations = 0.28-0.31) than the log K(ow) model. Recent experimental data for highly hydrophobic compounds fit well to the PP-LFERs and do not indicate the existence of a previously postulated "hydrophobicity cutoff". Predictive approaches based only on the molecular structure (KOWWIN, SPARC, COSMOthermX, COSMOmic) were also evaluated for K(lipw) prediction. The PP-LFERs revealed that partition coefficients into membrane lipids can be two log units higher than those into storage lipids for H-bond donor compounds, suggesting that distinguishing between the two lipids is necessary to account for the bioaccumulation of these compounds, and that tissues rich in membrane lipids (e.g., kidneys, liver) instead of fat tissue can be the primary phase for accumulation.