Efficient clustering of uncertain data streams

Clustering uncertain data streams has recently become one of the most challenging tasks in data management because of the strict space and time requirements of processing tuples arriving at high speed and the difficulty that arises from handling uncertain data. The prior work on clustering data streams focuses on devising complicated synopsis data structures to summarize data streams into a small number of micro-clusters so that important statistics can be computed conveniently, such as Clustering Feature (CF) (Zhang et al. in Proceedings of ACM SIGMOD, pp 103–114, 1996) for deterministic data and Error-based Clustering Feature (ECF) (Aggarwal and Yu in Proceedings of ICDE, 2008) for uncertain data. However, ECF can only handle attribute-level uncertainty, while existential uncertainty, the other kind of uncertainty, has not been addressed yet. In this paper, we propose a novel data structure, Uncertain Feature (UF), to summarize data streams with both kinds of uncertainties: UF is space-efficient, has additive and subtractive properties, and can compute complicated statistics easily. Our first attempt aims at enhancing the previous streaming approaches to handle the sliding-window model by using UF instead of old synopses, inclusive of CluStream (Aggarwal et al. in Proceedings of VLDB, 2003) and UMicro (Aggarwal and Yu in Proceedings of ICDE, 2008). We show that such methods cannot achieve high efficiency. Our second attempt aims at devising a novel algorithm, cluUS , to handle the sliding-window model by using UF structure. Detailed analysis and thorough experimental reports on synthetic and real data sets confirm the advantages of our proposed method.     

Incremental join view maintenance on distributed log-structured storage

Modern database systems desperate for the ability to support highly scalable transactions and efficient queries simultaneously for real-time applications. One solution is to utilize query optimization techniques on the on-line transaction processing (OLTP) systems. The materialized view is considered as a panacea to decrease query latency. However, it also involves the significant cost of maintenance which trades away transaction performance. In this paper, we examine the design space and conclude several design features for the implementation of a view on a distributed log-structured merge-tree (LSMtree), which is a well-known structure for improving data write performance. As a result, we develop two incremental view maintenance (IVM) approaches on LSM-tree. One avoids join computation in view maintenance transactions. Another with two optimizations is proposed to decouple the view maintenance with the transaction process. Under the asynchronous update, we also provide consistency queries for views. Experiments on TPC-H benchmark show our methods achieve better performance than straightforward methods on different workloads.     

Detecting Overlapping Community Structures in Networks

Community structure has been recognized as an important statistical feature of networked systems over the past decade. A lot of work has been done to discover isolated communities from a network, and the focus was on developing of algorithms with high quality and good performance. However, there is less work done on the discovery of overlapping community structure, even though it could better capture the nature of network in some real-world applications. For example, people are always provided with varying characteristics and interests, and are able to join very different communities in their social network. In this context, we present a novel overlapping community structures detecting algorithm which first finds the seed sets by the spectral partition and then extends them with a special random walks technique. At every expansion step, the modularity function Q is chosen to measure the expansion structures. The function has become one of the popular standards in community detecting and is defined in Newman and Girvan (Phys. Rev. 69:026113, 2004). We also give a theoretic analysis to the whole expansion process and prove that our algorithm gets the best community structures greedily. Extensive experiments are conducted in real-world networks with various sizes. The results show that overlapping is important to find the complete community structures and our method outperforms the C-means in quality.     

On measuring network robustness for weighted networks

Knowledge and Information Systems - Network robustness measures how well network structure is strong and healthy when it is under attack, such as vertices joining and leaving. It has been widely...     

A distributed approach to enabling privacy-preserving model-based classifier training

This paper proposes a novel approach for privacy-preserving distributed model-based classifier training. Our approach is an important step towards supporting customizable privacy modeling and protection. It consists of three major steps. First, each data site independently learns a weak concept model (i.e., local classifier) for a given data pattern or concept by using its own training samples. An adaptive EM algorithm is proposed to select the model structure and estimate the model parameters simultaneously. The second step deals with combined classifier training by integrating the weak concept models that are shared from multiple data sites. To reduce the data transmission costs and the potential privacy breaches, only the weak concept models are sent to the central site and synthetic samples are directly generated from these shared weak concept models at the central site. Both the shared weak concept models and the synthetic samples are then incorporated to learn a reliable and complete global concept model. A computational approach is developed to automatically achieve a good trade off between the privacy disclosure risk, the sharing benefit and the data utility. The third step deals with validating the combined classifier by distributing the global concept model to all these data sites in the collaboration network while at the same time limiting the potential privacy breaches. Our approach has been validated through extensive experiments carried out on four UCI machine learning data sets and two image data sets.

基于OWL的软件工程数据建模

网络本体语言(Web ontology language,OWL)是语义网技术的一个重要组成部分,适合于对复杂的数据进行语义描述和建模.在软件系统的开发过程中通常会产生大量结构复杂、语义丰富的数据,而建立一个灵活的语义模型是对各类软件工程数据进行统一管理的基础.从设计和实现海量软件工程数据管理平台的需求出发,提出了一种基于OWL的软件工程数据描述模型.该模型不仅能够对源代码、需求、测试、版本和缺陷数据进行描述,同时还能对这些数据之问的语义关联进行描述.通过案例分析对模型的有效性进行了讨论.     

Learning user credibility for product ranking

As the explosion of user-generated data (UGC) in electronic commerce, this kind of data is scanned for trust or credibility calculation, which plays an important role in business selection. The commonly used UGC is user reviews and ratings. A new consumer without any experience with some product will read these UGCs to get an overview. However, the open and dynamic e-commerce platforms may rise the generation of unfair or deceitful reviews and ratings. Then, detecting trustful reviewers or generating authentic ratings for customers is urgent and useful. In this paper, we present a twin-bipartite graph model to catch the review and ranking relationship among users, products and shops. We design a feedback mechanism to get the consistent ranking among different level of objects, which are users and items. In the algorithm, we adjust customer credibility values by the feedback considering the rating consistency; we adjust ratings by combining customer credibility together with originally assigned ratings. We increase the credibility for a customer if the customer gives a high (low) score to a good (bad) product and decrease the value if the customer gives a low (high) score to a good (bad) product. We detect the inconsistency between semantic ratings (the review comments) and numerical ratings (scores). To deal with it, we train a classifier on the training data that are constructed automatically. The trained classifier is used to predict the semantic scores from review comments. Finally, we calculate the scores of products by considering both the customer credibility and the predicted scores. We conduct experiments using a large amount of real-world data. The experimental results show that our proposed approach provides better products ranking than the baseline systems.     

Top-<Emphasis Type="Italic">k</Emphasis> temporal keyword search over social media data

Social media services have already become main sources for monitoring emerging topics and sensing real-life events. A social media platform manages social stream consisting of a huge volume of timestamped user generated data, including original data and repost data. However, previous research on keyword search over social media data mainly emphasizes on the recency of information. In this paper, we first propose a problem of top-k most significant temporal keyword query to enable more complex query analysis. It returns top-k most popular social items that contain the keywords in the given query time window. Then, we design a temporal inverted index with two-tiers posting list to index social time series and a segment store to compute the exact social significance of social items. Next, we implement a basic query algorithm based on our proposed index structure and give a detailed performance analysis on the query algorithm. From the analysis result, we further refine our query algorithm with a piecewise maximum approximation (PMA) sketch. Finally, extensive empirical studies on a real-life microblog dataset demonstrate the combination of two-tiers posting list and PMA sketch achieves remarkable performance improvement under different query settings.     

Approaches for scaling DBSCAN algorithm to large spatial databases

The huge amount of information stored in datablases owned by coporations(e.g.retail,financial,telecom) has spurred a tremendous interest in the area of knowledge discovery and data mining.Clustering.in data mining,is a useful technique for discovering intersting data distributions and patterns in the underlying data,and has many application fields,such as statistical data analysis,pattern recognition,image processsing,and other business application,s Although researchers have been working on clustering algorithms for decades,and a lot of algorithms for clustering have been developed,there is still no efficient algorithm for clustering very large databases and high dimensional data,As an outstanding representative of clustering algorithms,DBSCAN algorithm shows good performance in spatial data clustering.However,for large spatial databases,DBSCAN requires large volume of memory supprot and could incur substatial I/O costs because it operates directly on the entrie database,In this paper,several approaches are proposed to scale DBSCAN algorithm to large spatial databases.To begin with,a fast DBSCAN algorithm is developed.which considerably speeeds up the original DBSCAN algorithm,Then a sampling based DBSCAN algorithm,a partitioning-based DBSCAN algorithm,and a parallel DBSCAN algorithm are introduced consecutively.Following that ,based on the above-proposed algorithms,a synthetic algorithm is also given,Finally,some experimental results are given to demonstrate the effectiveness and efficiency of these algorithms.