Merging radiometric grids using histogram matching

Large-scale compilations of airborne geophysical data are an important part of regional-scale mineral exploration and in most cases, the compilation involves merging datasets of varying quality, acquired over a long period of time. Merging radiometric datasets involves the additional complication that many older surveys were flown with uncalibrated spectrometer systems. Conventional grid merging is based on calculation of base-level shift and scaling factors for each grid with the grid edges feathered. However, even after these corrections, soil moisture differences between surveys acquired over a long period of time and at different seasons when soil moisture content varied introduce scaling errors into the estimated radioelement concentrations. An alternative method based on histogram matching is suggested as an improvement on existing methods. Source code in Matlab format is available from the server at www.iamg.org.     

内存数据库的一种数据组织方式

近年来,电信和金融领域的主要应用已经变成数据密集型应用,数据库系统在其中扮演了关键的角色.但传统的基于磁盘的关系数据库系统却不能满足上述应用高性能、实时/近实时数据访问的要求,内存数据库系统则可以很好地满足各种应用系统的实时数据管理需求.介绍了内存数据库的一种内部数据组织方式,并对各种核心对象的存储及数据结构进行了分析.     

Merging local patterns using an evolutionary approach

This paper describes a Decentralized Agent-based model for Theory Synthesis (DATS) implemented by MASETS, a Multi-Agent System for Evolutionary Theory Synthesis. The main contributions are the following: first, a method for the synthesis of a global theory from distributed local theories. Second, a conflict resolution mechanism, based on genetic algorithms, that deals with collision/contradictions in the knowledge discovered by different agents at their corresponding locations. Third, a system-level classification procedure that improves the results obtained from both: the monolithic classifier and the best local classifier. And fourth, a method for mining very large datasets that allows for divide-and-conquer mining followed by merging of discoveries. The model is validated with an experimental application run on 15 datasets. Results show that the global theory outperforms all the local theories, and the monolithic theory (obtained from mining the concatenation of all the available distributed data), in a statistically significant way.     

Incremental recovery in main memory database systems

Recovery activities, like checkpointing and restart, in traditional database management systems are performed in a quiescent state where no transactions are active. This approach impairs the performance of online transaction processing systems, especially when a large volatile memory is used. An incremental scheme for performing recovery in main memory database systems (MMDBs), in parallel with transaction execution, is presented. A page-based incremental restart algorithm that enables the resumption of transaction processing as soon as the system is up is proposed. Pages are recovered individually and according to the demands of the post-crash transactions. A method for propagating updates from main memory to the backup database on disk is also provided. The emphasis is on decoupling the I/O activities related to the propagation to disk from the forward transaction execution in memory. The authors also construct a high-level recovery manager based on operation logging on top of the page-based algorithms. The proposed algorithms are motivated by the characteristics of large MMDBs, and exploit the technology of nonvolatile RAM     

Sorting unsorted and partially sorted lists using the natural merge sort

A version of the natural merge sort is shown to be an efficient method for both nearly sorted data and for random data. This method is compared with quickersort, 2-way merge sort, and Cook and Kim's method for various sortedness ratios.     

Time-shift replacement algorithm for main memory performance optimization

Page replacement algorithms of main memory in modern operating systems are crucial in system performance. When memory is full, a page replacement algorithm exploits temporal locality and frequency of page references to evict the page that is least likely to be accessed in the near future. Subsequently, loading the majority of data directly from memory improves performance by reducing I/O waits of accessing slow storage. Research of replacement algorithms that maximizes hit ratio while incurring as less overhead as possible has been constantly studied. In this paper, we propose a time-shift least recently used (TSLRU) algorithm that converts frequency information of page references into temporal locality. Frequent accesses of a page are thus recognized and accumulated in terms of time. Moreover, pages being loaded into memory for the first time are not necessarily the most recently used pages. As a result, one-pass pages are evicted sooner in our algorithm than in traditional LRU algorithm. Our performance evaluations show that the TSLRU outperforms conventional page replacement algorithms on both artificial and real application traces. For example, hit ratio of TSLRU advances ARC by \(4.17\%\) and LRU by \(5.91\%\) on normal distributed workloads. Moreover, TSLRU outperforms ARC by over \(2\%\) on half of the application traces tested.     

Suffix trees for inputs larger than main memory

A suffix tree is a fundamental data structure for string searching algorithms. Unfortunately, when it comes to the use of suffix trees in real-life applications, the current methods for constructing suffix trees do not scale for large inputs. As suffix trees are larger than the input sequences and quickly outgrow the main memory, the first attempts at building large suffix trees focused on algorithms which avoid massive random access to the trees being built. However, all the existing practical algorithms perform random access to the input string, thus requiring in essence that the input be small enough to be kept in main memory. The constantly growing pool of string data, especially biological sequences, requires us to build suffix trees for much larger strings.     

Road grade estimation for look-ahead vehicle control using multiple measurement runs

Look-ahead cruise controllers and other advanced driver assistance systems for heavy duty vehicles require high precision digital topographic road maps. This paper presents a road grade estimation algorithm for creation of such maps based on Kalman filter fusion of vehicle sensor data and GPS positioning information. The algorithm uses data from multiple passes over the same road to improve previously stored road grade estimates. Measurement data from three test vehicles and six experiments have been used to evaluate the quality of the obtained road grade estimate compared to a known reference. The obtained final grade estimate compares favorably to one acquired from a specialized road grade measurement vehicle with a DGPS receiver and inertial measurement unit, with an average root mean square error of 0.17% grade.     

空间实时内存数据库恢复机制研究与实现

为了满足空间应用对数据管理的实时性和可靠性的要求,针对Swallow空间实时内存数据库设计了一个内存数据库的恢复机制,结合日志驱动修改、影子页面、模糊检查点的思想实现了内存数据库数据和日志的快速备份以及在内存数据库出现事务故障和系统故障时的恢复策略.实验表明该策略保证了事务的并发度,并且在事务故障和系统故障时都有着较高的恢复效率.     

Direct RAMbus technology: the new main memory standard

Providing three times the memory bandwidth of the 66-MHz SDRAM subsystem, Direct RDRAM modules fit seamlessly into the existing mechanical space and airflow environment of the industry-standard PC chassis     

Dynamic recognition prefetch engine for DRAM-PCM hybrid main memory

The Journal of Supercomputing - This research is to design an effective prefetching method required for hybrid main memory systems consisting of dynamic random-access memory (DRAM) and phase-change...     

Distributed load balancing for parallel main memory hash join

Parallel joins have been widely studied during the past decade and a number of efficient algorithms were presented. While it is known that the performance of these algorithms may suffer greatly in the presence of skewed input data, the work on load balancing schemes for parallel join has been limited. The main contribution of this paper is the development and analysis of a new distributed data structure and an effective load balancing scheme for parallel main memory hash join on NUMA architecture. Multiprocessors based on this architecture are scalable in both size of main memory and number of processors, and provide very high memory bandwidth. The load balancing scheme is based on random probing to avoid the hot spot problems caused by probing sequentially. We have modeled this load balancing scheme both analytically and experimentally. The experiments were run on a BBN TC2000 multiprocessor system     

Effects of update techniques on main memory database systemperformance

Update technique is an important issue related to database recovery. In a main memory database environment, transaction execution can be processed without any I/O, and all I/O operations involved are for recovery purposes. The efficiency of update techniques therefore has an important impact on the performance of main memory database systems. We compared the techniques of immediate and deferred update based on a database machine, MARS. The simulation results showed that immediate update outperforms deferred update unless system failure is a frequent occurrence     

Distributed shared memory for roaming large volumes

We present a cluster-based volume rendering system for roaming very large volumes. This system allows to move a gigabyte-sized probe inside a total volume of several tens or hundreds of gigabytes in real-time. While the size of the probe is limited by the total amount of texture memory on the cluster, the size of the total data set has no theoretical limit. The cluster is used as a distributed graphics processing unit that both aggregates graphics power and graphics memory. A hardware-accelerated volume renderer runs in parallel on the cluster nodes and the final image compositing is implemented using a pipelined sort-last rendering algorithm. Meanwhile, volume bricking and volume paging allow efficient data caching. On each rendering node, a distributed hierarchical cache system implements a global software-based distributed shared memory on the cluster. In case of a cache miss, this system first checks page residency on the other cluster nodes instead of directly accessing local disks. Using two Gigabit Ethernet network interfaces per node, we accelerate data fetching by a factor of 4 compared to directly accessing local disks. The system also implements asynchronous disk access and texture loading, which makes it possible to overlap data loading, volume slicing and rendering for optimal volume roaming.     

Merging sub-networks in VANETs by using the IEEE 802.11xx protocols

Peer-to-Peer Networking and Applications - Nowadays most mobile ad-hoc networks are based on the IEEE 802.11 standards for wireless technology under the Wi-Fi brand. The structure of such protocols...     

A new memory allocation method for shared memory multiprocessors with large virtual address space

This paper proposes a new memory allocation method for shared memory multiprocessors with large virtual address spaces. An evaluation of its performance is also presented. For effective use of shared memory multiprocessors, it is important that no processor's execution is blocked. If several processors simultaneously access a shared variable, their processes are blocked and access to the variable is serialized. Thus, frequent access to shared variables reduces the parallelism. In particular, the parallelism is significantly reduced when a special shared variable – the ‘allocation pointer’ – is frequently accessed in the dynamic object allocation by an application program. In this paper, we propose a new method for allocating physical memory pages where the allocation pointer is monotonically increased in the virtual address space in contrast to the conventional method. This allows the critical sections for access to the allocation pointer to be executed effectively and atomically by using the fetch-and-add primitive. Our method improves the application program's parallelism by access to the allocation pointer with considerably short blocking time to the process. © 1997 John Wiley & Sons, Ltd.     

基于影子页面的MMDB的数据恢复方法

内存数据库数据主拷贝常驻内存,活动事务只与内存打交道,而由于内存的易失性,内存数据库的恢复成为内存数据库的核心技术.讨论了内存数据库的恢复技术,考虑所研究的系统环境限制,设计一种利用影子页面技术,并结合事务一致性检查点、模糊检查点思想,加上多版本技术的内存数据库恢复方法.该方法无需额外的硬件支持,解决了现有内存数据库恢复方法的一些问题.通过日志,检查点、恢复等方面来说明所设计的数据恢复方法,讲述它的备份过程和在事务故障和系统故障情况下恢复系统的过程.     

PRISMA/DB: a parallel, main memory relational DBMS

PRISMA/DB, a full-fledged parallel, main memory relational database management system (DBMS) is described. PRISMA/DB's high performance is obtained by the use of parallelism for query processing and main memory storage of the entire database. A flexible architecture for experimenting with functionality and performance is obtained using a modular implementation of the system in an object-oriented programming language. The design and implementation of PRISMA/DB are described in detail. A performance evaluation of the system shows that the system is comparable to other state-of-the-art database machines. The prototype implementation of the system runs on a 100-node parallel multiprocessor