Online querying of d-dimensional hierarchies

In this paper we describe a distributed system designed to efficiently store, query and update multidimensional data organized into concept hierarchies and dispersed over a network. Our system employs an adaptive scheme that automatically adjusts the level of indexing according to the granularity of the incoming queries, without assuming any prior knowledge of the workload. Efficient roll-up and drill-down operations take place in order to maximize the performance by minimizing query flooding. Updates are performed on-line, with minimal communication overhead, depending on the level of consistency needed. Extensive experimental evaluation shows that, on top of the advantages that a distributed storage offers, our method answers the vast majority of incoming queries, both point and aggregate ones, without flooding the network and without causing significant storage or load imbalance. Our scheme proves to be especially efficient in cases of skewed workloads, even when these change dynamically with time. At the same time, it manages to preserve the hierarchical nature of data. To the best of our knowledge, this is the first attempt towards the support of concept hierarchies in DHTs.     

基于元数据的地下水数据共享

地下水数据共享平台是一个分布式面向网络的系统。采用元数据集中管理、数据体分散存储的策略可以共享分布异构的地下水数据资源。基于RDF/XML框架定制地下水元数据,使得地下水元数据可以与其他元数据标准进行交换,同时可以方便地对其进行扩展。以元数据为核心,以数据汇交—数据查询—数据浏览下载为主线,设计了地下水数据共享平台的功能。在J2EE环境下,实现了地下水数据共享平台,其中空间数据服务采用空间服务元数据和ArcIMS相结合的方式实现。     

DIASPORA: A highly distributed web-query processing system

Current proposals for web querying systems have assumed a centralized processing architecture wherein data is shipped from the remote sites to the user's site. We present here the design and implementation of DIASPORA, a highly distributed query processing system for the web. It is based on the premise that several web applications are more naturally processed in a distributed manner, opening up possibilities of significant reductions in network traffic and user response times. DIASPORA is built over an expressive graph-based data model that utilizes simple heuristics and lends itself to automatic generation. The model captures both the content of web documents and the hyperlink structural framework of a web site. Distributed queries on the model are expressed through a declarative language that permits users to explicitly specify navigation. DIASPORA implements a query-shipping model wherein queries are autonomously forwarded from one web-site to another, without requiring much coordination from the query originating site. Its design addresses a variety of interesting issues that arise in the distributed web context including determining query completion, handling query rewriting, supporting query termination and preventing multiple computations of a query at a site due to the same query arriving through different paths in the hyperlink framework. The DIASPORA system is currently operational and is undergoing testing on our campus network. In this paper we describe the design of the system and report initial performance results that indicate significant performance improvements over comparable centralized approaches. This revised version was published online in August 2006 with corrections to the Cover Date.     

基于分布式的远程复制系统研究

针对传统的集中式复制结构存在的单点瓶颈和扩展性差等不足,提出了基于分布式结构的数据复制系统。通过将数据同步分散的复制到从节点,使得从节点能在单位时间内获得主节点的最大备份。通过从节点之间点对点的数据复制,使得在保证主从端数据一致性的同时,充分利用了从节点的资源,分散了主节点的负荷。     

由西门子利多富和思爱普公司共同成立之德国社会市场经济基金会向浙江大学信息与管理软件中心提供捐赠

由西门子利多富和思爱普公司共同成立之德国社会市场经济基金会向浙江大学信息与管理软件中心提供捐赠德国社会市场经济基金会于浙江大学百年校庆之际，向浙江大学信息与管理软件中心捐赠软硬件设备。德国社会市场经济基金会是由德国西门子利多富（ＳＮＩ）和思爱普（ＳＡ...     

Improving availability and performance with application-specific data replication

The emerging edge services architecture promises to improve the availability and performance of Web services by replicating servers at geographically distributed sites. A key challenge in such systems is data replication and consistency, so that edge server code can manipulate shared data without suffering the availability and performance penalties that would be incurred by accessing a traditional centralized database. This work explores using a distributed object architecture to build an edge service data replication system for an e-commerce application, the TPC-W benchmark, which simulates an online bookstore. We take advantage of application-specific semantics to design distributed objects that each manages a specific subset of shared information using simple and effective consistency models. Our experimental results show that by slightly relaxing consistency within individual distributed objects, our application realizes both high availability and excellent performance. For example, in one experiment, we find that our object-based edge server system provides five times better response time over a traditional centralized cluster architecture and a factor of nine improvement over an edge service system that distributes code but retains a centralized database.     

基于均衡分簇的无线传感器网络压缩数据收集

应用压缩感知（CS）理论结合稀疏随机投影的无线传感器网络（WSN）压缩数据收集（CDG）可以大大减少网络传输的数据量。针对随机选择投影节点作为簇头来收集数据导致网络整体能耗不稳定和不平衡的问题,提出两种平衡投影节点的压缩数据收集方法。对于节点分布均匀WSN,提出基于空间位置的均衡分簇法：首先,均匀划分网格;然后,在每个网格选举投影节点,依距离最短原则成簇;最后,由投影节点收集簇内数据到汇聚节点完成数据收集,从而使得投影节点分布均匀、网络能耗均衡。对于节点分布不均匀的WSN,提出基于节点密度的均衡分簇法：同时考虑节点的位置和密度,对节点数量少的网格不再选择投影节点,将网格内的少量节点分配到邻近的网格,从而平衡网络能量,延长网络寿命。仿真结果表明,与随机投影节点法相比,所提的两种方法的网络寿命均延长了25%以上,剩余节点数在网络运行中期均能达到2倍左右,具有更好的网络连通性,显著提高了整个网络的生命周期。     

Online mining of fuzzy multidimensional weighted association rules

This paper addresses the integration of fuzziness with On-Line Analytical Processing (OLAP) based association rules mining. It contributes to the ongoing research on multidimensional online association rules mining by proposing a general architecture that utilizes a fuzzy data cube for knowledge discovery. A data cube is mainly constructed to provide users with the flexibility to view data from different perspectives as some dimensions of the cube contain multiple levels of abstraction. The first step of the process described in this paper involves introducing fuzzy data cube as a remedy to the problem of handling quantitative values of dimensional attributes in a cube. This facilitates the online mining of fuzzy association rules at different levels within the constructed fuzzy data cube. Then, we investigate combining the concepts of weight and multiple-level to mine fuzzy weighted multi-cross-level association rules from the constructed fuzzy data cube. For this purpose, three different methods are introduced for single dimension, multidimensional and hybrid (integrates the other two methods) fuzzy weighted association rules mining. Each of the three methods utilizes a fuzzy data cube constructed to suite the particular method. To the best of our knowledge, this is the first effort in this direction. We compared the proposed approach to an existing approach that does not utilize fuzziness. Experimental results obtained for each of the three methods on a synthetic dataset and on the adult data of the United States census in year 2000 demonstrate the effectiveness and applicability of the proposed fuzzy OLAP based mining approach. OLAP is one of the most popular tools for on-line, fast and effective multidimensional data analysis. In the OLAP framework, data is mainly stored in data hypercubes (simply called cubes).     

不确定数据上两种查询的分布式聚集算法

不确定数据查询技术在军事、金融、电信等领域中起到了越来越重要的作用.不确定性数据在传感器网络、分布式Web Server及P2P系统等分布式系统中广泛存在.从这些系统中收集所有数据进行集中式查询将带来巨大的通信开销、时间延迟和存储代价.同时,由于不确定数据的特点,大多数集中式不确定查询算法在分布式环境下并不适用.给出不确定数据的最大值和Top-k聚集查询定义,并分别提出了基于过滤策略的分布式聚集算法.算法根据给出的3个过滤策略,利用数据的分布区间和概率进行筛选概率上限的计算,尽可能将不影响查询结果的数据抛弃.同时,算法以相对较小的代价归并保存并传输了计算最终查询结果所需要的不可丢弃数据.实验结果表明,在各类系统和数据条件下,过滤算法都能够正确地得到查询结果并显著降低系统的数据通信开销.     

一种基于FCS的企业INTRANET信息网络的设计与实现

现代工业生产趋向于信息控制一体化,而信息控制一体化的前提是对生产现场的数据集成和信息共享。本文针对当今日益分散化、大规模的现代工业生产的数据集成和信息共享提出建立以SHCAN2 0 0 0为底层现场总线数据采集系统的企业IN TRANET信息网络。该系统不仅能实时采集生产数据并产生数据记录文件,同时还有在线监测智能节点和网络故障、系统定时清零、远程标准信号自动校验、产生网络数据信息供企业INTRANET网络共享等功能,满足现场实际的需要。     

一种P2P网络环境下的OLAP查询方案

传统网络环境和P2P环境中,客户端向OLAP服务器提交OLAP查询,并从服务器获取查询结果,OLAP服务器的负载将随着客户端的增加而急剧增加。设计了一种基于P2P（Peer-to-Peer,点对点技术）技术的DQDC（Distributed Query Data Cube,多维数据集的分布式查询）算法,实现P2P网络中语义级的多节点Data Cube数据共享,从而提高系统整体的决策分析性能。     

结构化覆盖网络模型Chord研究

在P2P应用系统中,如何有效地定位分布在网络中不同节点上的数据资源一直是研究的重点。Chord模型通过提供了一个分布式的资源查找协议成功地解决了这个问题,同时Chord协议能够有效支持节点动态地加入和退出网络。文章对Chord的系统基础及特性进行了论述,并重点分析了协议所提供的文件资源查询和节点加入退出算法,从理论上论证了Chord是一种具有可扩展性的低消耗系统。     

Distributed anomaly detection for industrial wireless sensor networks based on fuzzy data modelling

Modern infrastructure increasingly depends on large computerized systems for their reliable operation. Supervisory Control and Data Acquisition (SCADA) systems are being deployed to monitor and control large scale distributed infrastructures (e.g. power plants, water distribution systems). A recent trend is to incorporate Wireless Sensor Networks (WSNs) to sense and gather data. However, due to the broadcast nature of the network and inherent limitations in the sensor nodes themselves, they are vulnerable to different types of security attacks. Given the critical aspects of the underlying infrastructure it is an extremely important research challenge to provide effective methods to detect malicious activities on these networks. This paper proposes a robust and scalable mechanism that aims to detect malicious anomalies accurately and efficiently using distributed in-network processing in a hierarchical framework. Unsupervised data partitioning is performed distributively adapting fuzzy c-means clustering in an incremental model. Non-parametric and non-probabilistic anomaly detection is performed through fuzzy membership evaluations and thresholds on observed inter-cluster distances. Robust thresholds are determined adaptively using second order statistical knowledge at each evaluation stage. Extensive experiments were performed and the results demonstrate that the proposed framework achieves high detection accuracy compared to existing data clustering approaches with more than 96% less communication overheads opposed to a centralized approach.     

传感器网络中的数据查询处理

传统的传感器网络采用集中式数据管理,不能有效利用便宜的本地计算来代替昂贵的网络通信.采用分布式的方法,在sink节点的应用层与网络层之间增加查询代理层,把用户查询分发到相关的传感器节点上进行处理.这样,通过减少网络传输的数据量,来降低传感器节点的能量消耗,延长网络寿命.     

基于主动网络的分布式P2P网络模型

分布式P2P网络Gnutella模型的对等机共享信息查询的搜索、定位路由协议仅使用"洪泛"算法,因此存在可扩展性问题.同时,由于其协议机制仅在应用层实现,缺乏对Intemet底层通信子网路由资源的利用,存在性能与效率不高的问题.为了解决上述问题,考察了Gnutella网络和Internet网络节点的拓扑分布规律,二者不仅均呈现幂规律(power law)和小世界(small world)特征,而且其幂系数t非常近似.提出了一个基于主动网络技术的分布式P2P网络(active distributed pee     

数据收集传感器网络的负载平衡网络构建方法

传感器网络所具有的集中式数据收集、多跳数据传输、多对一流量模式这3种特征会造成漏斗效应的出现,这会导致严重的包碰撞、网络拥塞、包丢失,甚至拥塞崩溃,还会导致能量消耗的热点的出现,使某些节点甚至整个网络过早死亡.负载平衡技术能够有效缓解漏斗效应的产生.针对静态数据收集传感器网络,基于供求网络的思想,提出了一种分布式算法,将传感器节点组织成交易平衡网络,用于平衡传感器节点的负载.利用这种方法组织而成的网络结构不是一棵负载平衡树,而是一个负载平衡网络.实验结果验证了这种方法的有效性.     

XCube: Processing XPath queries in a hypercube overlay network

In this paper, we present the design and performance of XCube, a tag-based system for managing XML data in a hypercube overlay network. In XCube, each node in a d-dimensional hypercube is identified by a d-bit vector. A peer manages a smaller hypercube with dimension d′?d. An XML document is compactly represented as a structure summary and a content summary. The structure summary comprises a d-bit vector derived from the distinct tag names in the document and a synopsis capturing the structure of the document. The content summary consists of a bit map that summarizes the document content. The metadata of a document, i.e., owner IP, document identifier, structure summary and content summary, is indexed at its anchor peer (the peer that manages the node with matching bit vector). In addition, the structure summary is further indexed at all peers that manages nodes whose bit vectors are covered by the document’s bit vector. An XPath query is processed in four phases. In phase 1, the query is routed to its anchor peer according to the bit vector of the query. In phase 2, the query is evaluated against all the synopses stored in its anchor peer and forwarded to the anchor peers of the matching synopses. In phase 3, the anchor peer of each related synopsis examines the query on the related bit maps and forwards the query to the related owner peers. Finally in phase 4, the owner peers evaluate the query on the XML documents and return answers to the querying peer. We also present a scheme that dynamically partitions the hypercube to balance the load across peers. We further exploit the partition history to remove redundant messages. We conduct a comprehensive experimental study and the results show the efficiency of XCube.     

基于数据价值的无人机数据收集方法

数据收集是无线监测网络的关键环节.利用无人机进行数据收集,其本质是通过无人机的移动代替网络中的转发节点,减少数据从源节点到基站的转发次数,有效节约监测网络能量,从而成为未来发展的趋势.现有研究关注如何利用无人机有限的能量获得更多的数据,缺乏对获取数据的价值评估,从而导致无人机数据收集能效比不高.如何利用无人机最少的能量付出在监测区域获取最大的数据价值,其难点在于数据价值是针对不同应用的主观评价,而不同节点获取的数据价值如何比较,目前缺乏统一的标准.我们发现,数据相似节点的数据价值存在相似性.在此基础上,我们提出了一种数据收集方法OnValueGet,利用关键性代表节点的数据,最大程度的近似代表整个监测区域的数据,从而在能量约束下获得最大数据价值.其核心思想在于：从分析感知数据的时空相似性入手,确定数据价值较高的感知节点,本文称为数据关键节点,在应用的误差范围内,它们采集的数据可以近似表示全部网络感知节点采集的数据.无人机以数据关键节点为数据采集的核心目标,在能量有限的情况下,根据遇到的障碍物和节点感知到数据的异常与否,动态的规划数据收集路线,从而使收集到的数据具有最大价值,显著提升数据收集的能效比.     

空调客车实时集中监测系统数据IC卡转存方法研究

介绍了在空调客车微机实时集中监测系统中,利用接触式IC卡读写器及IC卡对系统上位机数据进行存储,实现客车运行数据在下位机中的查询、打印及存档等相应功能。该读写系统编程简单、使用方便、可靠性好,适合在复杂的工业现场使用。     

数据仓库中CUBE大小估计算法

文章叙述了数据仓库中Cube估计的三个相关算法,提出了一种优化的估计方法。这种新的优化方法在一定程度上弥补了常用估计方法的缺陷,大大提高了估计值的准确性。Cube的大小估计对于数据仓库进行多维分析是非常重要的。原因是为了提高多维分析的查询速度,要进行一定程度的预计算,这种计算是相当复杂的,而且会占用大量数据库的存储空间,因此预先估计出所需空间大小是非常必要的。