基于嵌入二维数组的迁移聚集树的数据流突变检测算法

数据流突变检测技术由于在金融、医疗服务、电信等重要领域有广泛应用而受到国内外科研学者更多关注。为了能够检测正数据流、负数据流以及正负交错数据流的突变,提出了嵌入二维数组的迁移聚集树的数据流突变检测算法。该算法能够检测单调聚集函数和非单调聚集函数的突变,能够在较少时间内完成数据流突变检测的任务。实验证明本算法有良好的性能和效率,更适合检测突变的数据流。     

Image and video indexing using vector quantization

Visual (image and video) database systems require efficient indexing to enable fast access to the images in a database. In addition, the large memory capacity and channel bandwidth requirements for the storage and transmission of visual data necessitate the use of compression techniques. We note that image/video indexing and compression are typically pursued independently. This reduces the storage efficiency and may degrade the system performance. In this paper, we present novel algorithms based on vector quantization (VQ) for indexing of compressed images and video. To start with, the images are compressed using VQ. In the first technique, for each codeword in the codebook, a histogram is generated and stored along with the codeword. We note that the superposition of the histograms of the codewords, which are used to represent an image, is a close approximation of the histogram of the image. This histogram is used as an index to store and retrieve the image. In the second technique, the histogram of the labels of an image is used as an index to access the image. We also propose an algorithm for indexing compressed video sequences. Here, each frame is encoded in the intraframe mode using VQ. The labels are used for the segmentation of a video sequence into shots, and for indexing the representative frame of each shot. The proposed techniques not only provide fast access to stored visual data, but also combine compression and indexing. The average retrieval rates are 95% and 94% at compression ratios of 16:1 and 64:1, respectively. The corresponding cut detection rates are 97% and 90%, respectively.     

A fast and robust bulk-loading algorithm for indexing very large digital elevation datasets: I. Algorithm

Digital elevation models (DEMs) constitute a valuable source of data for a number of geoscience-related applications. The Shuttle Radar Topography Mission (SRTM) collected and made available to the public the world's largest DEM (composed of billions of points) until that date. The SRTM DEM is stored on the NASA repository as a well-organized collection of flat files. The retrieval of this stored topographic information about a region of interest involves one selection of a proper list of files, their downloading, data filtering in the desired region, and their processing according to user needs. With the aim to provide an easier and faster access to this data by improving its further analysis and processing, we have indexed the SRTM DEM by means of a spatial indexing based on the kd-tree data structure, called the Q-tree. This paper is the first in a two-part series that describes the method followed to build an index on such huge amounts of data, minimizing the number of insert operations. We demonstrate that our method can build a very efficient space-partitioning index, with good performance in both point and range queries on the spatial data. To the best of our knowledge, this is the only successful spatial indexing proposal in the literature that deals with such a huge volume of data.     

一种用于图象检索的聚类方法

设计和实现了一种对多维颜色特征进行聚类算法,对特征库按聚类模式建立索引。矣类方法大大缩短了检索时间。     

Performance analysis of timer-based burst assembly with slotted scheduling for optical burst switching networks

In this paper, we analyze the performance of a timer-based burst assembly for optical burst switching (OBS) networks. In our analytical model, an ingress edge node has multiple buffers where IP packets are stored depending on their egress edge nodes, and bursts are assembled at the buffers in round-robin manner. Moreover, bursts are transmitted in accordance with slotted scheduling where each burst transmission starts at the slot boundary. We construct a loss model with two independent arrival streams, and explicitly derive the burst loss probability, burst throughput, and data throughput. In numerical examples, we show the effectiveness of our analysis in comparison with the Erlang loss system. It is shown that our model is quite useful for an OBS network with a large number of input and output links.     

Discriminative and deterministic approaches towards entity resolution

To address the entity resolution problem, existing studies usually consist of two-steps. Given two lists of records, in the first step a small set of duplicate records (a candidate set) are selected based on index structures and algorithms for efficient entity resolution. Then, a given similarity function is applied to quantify the similarity of records in the candidate set. However, for real applications, it is a non-trivial task to select appropriate indexing techniques and similarity functions. In this paper, we tackle the problem of indexing and similarity function identification using both discriminative and deterministic approaches that select the best of indexing and similarity measures. According to our experimental results, our proposed solution considering both discriminative and deterministic approaches shows more than a 90 % average accuracy within hundreds of seconds.     

Sudden scene change detection in compressed video using interpolated macroblocks in B-frames

This paper addresses an important area in video processing, namely compressed domain processing. For video indexing, video scene transition detection is an essential step to segment the video. Current techniques for scene change detection tend to suffer from a major limitation as most of them cannot identify scene transitions in the compressed domain. Since most video is expected to be stored in the compressed domain, scene transition detection in this domain is highly desirable. In this paper an algorithm for video scene change detection is proposed to overcome this limitation. In this scheme, properties of the B-frames are used as it is capable of measuring the correlation between two adjacent reference frames. The results show that this scheme performs better than schemes based on P-frames. Proposed scheme can be directly applied with compressed data with minimum decompression and hence it is computationally efficient and makes real time implementations possible. Results show that video scene transitions can be identified satisfactorily with the proposed scheme.     

社交网络中隐式事件突发性检测

社交网络与人们的生活息息相关,其上的用户行为可用于检测社交网络中的事件突发性,进而准确定位事件的发生区间.但用户行为易受主观及外部因素的影响,有时会出现隐式事件突发性,给事件突发性检测带来困难.本文针对社交网络中的隐式事件突发性问题,在以社交行为特征进行事件突发性检测的基础上,引入关键词特征,动态调整各个时间窗口的候选关键词,将不同事件与不同的关键词特征绑定,避免事件之间及噪音带来的干扰,实现对隐式事件突发性的准确识别.相关实验表明,本文提出的算法可有效改善现有社交网络中事件突发性检测任务的效果.     

A Data Deduplication Framework of Disk Images with Adaptive Block Skipping

We describe an efficient and easily applicable data deduplication framework with heuristic prediction based adaptive block skipping for the real-world dataset such as disk images to save deduplication related overheads and improve deduplication throughput with good deduplication efficiency maintained. Under the framework, deduplication operations are skipped for data chunks determined as likely non-duplicates via heuristic prediction, in conjunction with a hit and matching extension process for duplication identification within skipped blocks and a hysteresis mechanism based hash indexing process to update the hash indices for the re-encountered skipped chunks. For performance evaluation, the proposed framework was integrated and implemented in the existing data domain and sparse indexing deduplication algorithms. The experimental results based on a real-world dataset of 1.0 TB disk images showed that the deduplication related overheads were significantly reduced with adaptive block skipping, leading to a 30%~80% improvement in deduplication throughput when deduplication metadata were stored on the disk for data domain, and 25%~40% RAM space saving with a 15%~20% improvement in deduplication throughput when an in-RAM sparse index was used in sparse indexing. In both cases, the corresponding deduplication ratios reduced were below 5%.     

Processing moving queries over moving objects using motion-adaptive indexes

This paper describes a motion-adaptive indexing scheme for efficient evaluation of moving continual queries (MCQs) over moving objects. It uses the concept of motion-sensitive bounding boxes (MSBs) to model moving objects and moving queries. These bounding boxes automatically adapt their sizes to the dynamic motion behaviors of individual objects. Instead of indexing frequently changing object positions, we index less frequently changing object and query MSBs, where updates to the bounding boxes are needed only when objects and queries move across the boundaries of their boxes. This helps decrease the number of updates to the indexes. More importantly, we use predictive query results to optimistically precalculate query results, decreasing the number of searches on the indexes. Motion-sensitive bounding boxes are used to incrementally update the predictive query results. Furthermore, we introduce the concepts of guaranteed safe radius and optimistic safe radius to extend our motion-adaptive indexing scheme to evaluating moving continual k-nearest neighbor (kNN) queries. Our experiments show that the proposed motion-adaptive indexing scheme is efficient for the evaluation of both moving continual range queries and moving continual kNN queries.     

An efficient compressed domain video indexing method

Video indexing is employed to represent the features of video sequences. Motion vectors derived from compressed video are preferred for video indexing because they can be accessed by partial decoding; thus, they are used extensively in various video analysis and indexing applications. In this study, we introduce an efficient compressed domain video indexing method and implement it on the H.264/AVC coded videos. The video retrieval experimental evaluations indicate that the video retrieval based on the proposed indexing method outperforms motion vector based video retrieval in 74 % of queries with little increase in computation time. Furthermore, we compared our method with a pixel level video indexing method which employs both temporal and spatial features. Experimental evaluation results indicate that our method outperforms the pixel level method both in performance and speed. Hence considering the speed and precision characteristics of indexing methods, the proposed method is an efficient indexing method which can be used in various video indexing and retrieval applications.     

Energy- and Latency-Efficient Processing of Full-Text Searches on a Wireless Broadcast Stream

In wireless mobile computing environments, broadcasting is an effective and scalable technique to disseminate information to a massive number of clients, wherein the energy usage and latency are considered major concerns. This paper presents an indexing scheme for the energy- and latency-efficient processing of full-text searches over the wireless broadcast data stream. Although a lot of access methods and index structures have been proposed in the past for full-text searches, all of them are targeted for data in disk storage, not wireless broadcast channels. For full-text searches on a wireless broadcast stream, we firstly introduce a naive, inverted list-style indexing method, where inverted lists are placed in front of the data on the wireless channel. In order to reduce the latency overhead, we propose a two-level indexing method which adds another level of index structure to the basic inverted list-style index. In addition, we propose a replication strategy of the index list and index tree to further improve the latency performance. We analyze the performance of the proposed indexing scheme with respect to the latency and energy usage measures, and show the optimality of index replication. The correctness of the analysis is demonstrated through simulation experiments, and the effectiveness of the proposed scheme is shown by implementing a real wireless information delivery system.     

R-树结点分配的新算法

近年来,针对空间数据库索引的研究引起了人们越来越多的兴趣和关注。为了快速、有效地处理存储于空间数据库中的海量空间数据,专家学者提出了大量的基于磁盘的空间索引方法。其中,1984年Guttman提出的R-树是目前非常有效的空间索引结构。针对R-树的结点分配算法存在的不足,提出了一种新的结点分配算法。研究结果表明： 新的分配算法比原始的算法产生的交叠会更小,从而有效地控制了多路查询的几率,较明显地提高了空间查询的效率。     

Density-preserving projections for large-scale local anomaly detection

Outlier or anomaly detection is a fundamental data mining task with the aim to identify data points, events, transactions which deviate from the norm. The identification of outliers in data can provide insights about the underlying data generating process. In general, outliers can be of two kinds: global and local. Global outliers are distinct with respect to the whole data set, while local outliers are distinct with respect to data points in their local neighbourhood. While several approaches have been proposed to scale up the process of global outlier discovery in large databases, this has not been the case for local outliers. We tackle this problem by optimising the use of local outlier factor (LOF) for large and high-dimensional data. We propose projection-indexed nearest-neighbours (PINN), a novel technique that exploits extended nearest-neighbour sets in a reduced-dimensional space to create an accurate approximation for k-nearest-neighbour distances, which is used as the core density measurement within LOF. The reduced dimensionality allows for efficient sub-quadratic indexing in the number of items in the data set, where previously only quadratic performance was possible. A detailed theoretical analysis of random projection (RP) and PINN shows that we are able to preserve the density of the intrinsic manifold of the data set after projection. Experimental results show that PINN outperforms the standard projection methods RP and PCA when measuring LOF for many high-dimensional real-world data sets of up to 300,000 elements and 102,600 dimensions. A further investigation into the use of high-dimensionality-specific indexing such as spatial approximate sample hierarchy (SASH) shows that our novel technique holds benefits over even these types of highly efficient indexing. We cement the practical applications of our novel technique with insights into what it means to find local outliers in real data including image and text data, and include potential applications for this knowledge.     

L-overlay: A layered data management scheme for peer-to-peer computing

Efficient storage and handling of data stored in a peer-to-peer (P2P) network, proves vital for various applications such as query processing and data mining. This paper presents a distributed, scalable and robust layered overlay (L-overlay) to index and manage multidimensional data in a dynamic P2P network. The proposed method distinguishes between the data and peer layers, with efficient mapping between the two. The data is organized such that semantically similar data objects are accessed hastily. Grid and tree structures are proposed for the peer layer. As application examples of L-overlay in query processing and data mining, k-nearest neighbors query processing and distributed Naïve Bayes classification algorithms, are proposed. We show the effectiveness of our scheme in static and dynamic environments using simulation. L-overlay is shown to be more efficient than SSW, an available semantic overly, in terms of maintenance and query processing costs.     

Using compressed index structures for processing moving objects in large spatio-temporal databases

This paper develops a novel, compressed B⁺-tree based indexing scheme that supports the processing of moving objects in one-, two-, and multi- dimensional spaces. The past, current, and anticipated future trajectories of movements are fully indexed and well organized. No parameterized functions and geometric representations are introduced in our data model so that update operations are not required and the maintenance of index structures can be accomplished by basic insertion and deletion operations. The proposed method has two contributions. First, the spatial and temporal attributes of trajectories are accurately preserved and well organized into compact index structures with very efficient memory space utilization and storage requirement. Second, index maintenance overheads are more economical and query performance is more responsive than those of conventional methods. Both analytical and empirical studies show that our proposed indexing scheme outperforms the TPR-tree.     

Image retrieval based on index compressed vector quantization

Increased amount of visual data in several applications necessitates content-based image retrieval. Since most of visual data is stored in compressed form, it is crucial to develop indexing techniques for searching images based on their content in compressed form. Therefore, it is desirable to explore image compression techniques with capability of describing image content in compressed form. Vector Quantization (VQ) is a compression scheme that exploits intra-block correlation and image correlation reflects image content, hence VQ is a suitable compression technique for compressed domain image retrieval.This paper introduces a novel indexing scheme for compressed domain image databases based on indices generated from IC-VQ. The proposed scheme extracts image features based on relationship between indices of IC-VQ compressed images. This relationship detects contiguous regions of compressed image based on inter- and intra-block correlation. Experimental results show effectiveness superiority of the new scheme compared to VQ and color-based schemes.     

A fractal-based clustering approach in large visual database systems

Large visual database systems require effective and efficient ways of indexing and accessing visual data on the basis of content. In this process, significant features must first be extracted from image data in their pixel format. These features must then be classified and indexed to assist efficient access to image content. With the large volume of visual data stored in a visual database, image classification is a critical step to achieve efficient indexing and retrieval. In this paper, we investigate an effective approach to the clustering of image data based on the technique of fractal image coding, a method first introduced in conjunction with fractal image compression technique. A joint fractal coding technique, applicable to pairs of images, is used to determine the degree of their similarity. Images in a visual database can be categorized in clusters on the basis of their similarity to a set of iconic images. Classification metrics are proposed for the measurement of the extent of similarity among images. By experimenting on a large set of texture and natural images, we demonstrate the applicability of these metrics and the proposed clustering technique to various visual database applications.     

Efficient binary code indexing with pivot based locality sensitive clustering

High-dimensional indexing is fundamental in multimedia research field. Compact binary code indexing has achieved significant success in recent years for its effective approximation of high-dimensional data. However, most of existing binary code methods adopt linear scan to find near neighbors, which involve unnecessary computations and thus degrade search efficiency especially in large scale applications. To avoid searching codes that are not near neighbors with high probability, we propose a framework that index binary codes in clusters and only codes in relevant clusters are scanned. Consequently, Pivot Based Locality Sensitive Clustering (PLSC) is proposed and Density Adaptive Binary coding (DAB) method in PLSC clusters is presented. PLSC uses pivots to estimate similarities between data points and generates clusters based on the Locality Sensitive Hashing scheme. DAB adopts different binary code generation methods according to cluster densities. Experiments on open datasets show that offline indexing based on PLSC is efficient and DAB codes in PLSC clusters achieve significant improvement on search efficiency compared to the state of the art binary codes.