An efficient workload-based data layout scheme for multidimensional data

Physical data layout is a crucial factor in the performance of queries and updates in large data warehouses. Data layout enhances and complements other performance features such as materialized views and dynamic caching of aggregated results. Prior work has identified that the multidimensional nature of large data warehouses imposes natural restrictions on the query workload. A method based on a “uniform” query class approach has been proposed for data clustering and shown to be optimal. However, we believe that realistic query workloads will exhibit data access skew. For instance, if time is a dimension in the data model, then more queries are likely to focus on the most recent time interval. The query class approach does not adequately model the possibility of multidimensional data access skew. We propose the affinity graph model for capturing workload characteristics in the presence of access skew and describe an efficient algorithm for physical data layout. Our proposed algorithm considers declustering and load balancing issues which are inherent to the multidisk data layout problem. We demonstrate the validity of this approach experimentally.     

基于元数据的地下水数据共享*

地下水数据共享平台是一个分布式面向网络的系统。采用元数据集中管理、数据体分散存储的策略可以共享分布异构的地下水数据资源。基于RDF/XML框架定制地下水元数据,使得地下水元数据可以与其他元数据标准进行交换,同时可以方便地对其进行扩展。以元数据为核心,以数据汇交—数据查询—数据浏览下载为主线,设计了地下水数据共享平台的功能。在J2EE环境下,实现了地下水数据共享平台,其中空间数据服务采用空间服务元数据和ArcIMS相结合的方式实现。     

基于XML的数据立方体的实现

数据仓库系统正广泛用于联机分析处理系统，为了能将多个数据仓库集成到一起，需要解决技术上和语义上的一些问题。一种基本的解决方法是建立一种标准化的、独立于各供应商的多维数据描述格式。本文介绍一个基于XML的文档模板集——xCube，它可在任何网络上交换数据仓库数据。由于XCube被组织成模块化的形式，所以立方体的多维模式、维数据和事实数据能分步传输，因此立方体都能很容易从一个数据仓库传输到另一个数据仓库。     

Development of Secure XML Data Warehouses with QVT

ContextData warehouses are systems which integrate heterogeneous sources to support the decision making process. Data from the Web is becoming increasingly more important as sources for these systems, which has motivated the extensive use of XML to facilitate data and metadata interchange among heterogeneous data sources from the Web and the data warehouse. However, the business information that data warehouses manage is highly sensitive and must, therefore, be carefully protected. Security is thus a key issue in the design of data warehouses, regardless of the implementation technology. It is important to note that the idiosyncrasy of the unstructured and semi-structured data requires particular security rules that have been specifically tailored to these systems in order to permit their particularities to be captured correctly. Unfortunately, although security issues have been considered in the development of traditional data warehouses, current research lacks approaches with which to consider security when the target platform is based on XML technology.ObjectiveWe shall focus on defining transformations to obtain a secure XML Schema from the conceptual multidimensional model of a data warehouse.MethodWe have first defined the rationale behind the transformation rules and how they have been developed in natural language, and we have then established them clearly and formally by using the QVT language. Finally, in order to validate our proposal we have carried out a case study.ResultsWe have proposed an approach for the model driven development of Secure XML Data Warehouses, defining a set of QVT transformation rules.ConclusionThe main benefit of our proposal is that it is possible to model security requirements together with the conceptual model of the data warehouse during the early stages of a project, and automatically obtain the corresponding implementation for XML.     

基于IXF标准的分布式产品数据交换技术研究

产品数据交换技术是分布式产品开发过程的使能技术之一。中性的产品数据表达格式——IXF是一种新出现的基于XML的数据交换标准，属性(Attribute)、类(Class)、类行为(Class Behavior)和对象(Object)是IXF标准中的核心概念，基于XML语法的表达形式使之成为Internet时代产品数据交换标准家族中的重要一员，IXF的应用模式为从事于该领域的研究和应用人员提供了新的技术方案。     

Open internet gateways to archives of media art

The EU-projects OASIS Archive and GAMA aimed to develop systems for the common presentation of distributed Media Art works, independent of their location. The paper presents the technical solutions implemented during projects, and contextualise the work in the related media art scene. In order to ensure the preservation and availability (sustainability) of cultural heritage, the metadata systems of all participating institutions were interlinked and they can now be accessed by individual users (both researchers and general public) through an on-line interface. The developed multimedia archive servers enable the exchange of metadata within a distributed system and enable various play-out sources and media. The decentralised architecture, where data can remain at their physical location and are linked at the metadata level, is the key concept of the presented system.     

Efficient aggregation algorithms on very large compressed data warehouses

Multidimensional aggregation is a dominant operation on data warehouses for on-line analytical processing(OLAP).Many efficinet algorithms to compute multidimensional aggregation on relational database based data warehouses have been developed.However,to our knowledge,there is nothing to date in the literature about aggregation algorithms on multidimensional data warehouses that store datasets in mulitidimensional arrays rather than in tables.This paper presents a set of multidimensional aggregation algorithms on very large and compressed multidimensional data warehouses.These algorithms operate directly on compressed datasets in multidimensional data warehouses without the need to first decompress them.They are applicable to a variety of data compression methods.The algorithms have different performance behavior as a function of dataset parameters,sizes of out puts and ain memory availability.The algorithms are described and analyzed with respect to the I/O and CPU costs,A decision procedure to select the most efficient algorithm ,given an aggregation request,is also proposed.The analytical and experimental results show that the algorithms are more efficient than the traditional aggregation algorithms.     

数据仓库元数据集成技术研究与应用

元数据集成是数据仓库元数据管理的一项重要内容。文章在目前元数据集成研究成果的基础上,提出了一种基于SOA架构的数据仓库元数据集成技术。该技术以Web Service技术为应用框架,以CWM为元数据模型,采用XML设计元数据封装器,实现了分布环境下数据仓库元数据的集成与重用。文章重点介绍了系统的体系构架,及CWM元数据模型的结构设计和不同Web Service方法设计和调用。最后,将该技术应用到消防工程领域的一个实际数据仓库项目中,验证了该技术的可行性、有效性、实用性。     

基于XML的多维概念模型的研究

对于数据仓库概念模型的研究,目前缺乏统一的标准,不利于模型的交流与共享。基于XML的多维概念模型利用XML这一标准交换语言,不仅很好地解决了这一问题,而且也为元数据的集成与共享打下了基础。针对多维模型的特点,定义了一个特定的DTD,它能够完整地描述多维概念模型的各种语义特征,并针对基于UML的多维概念建模方法,定义了基于XML的多维概念模型与基于UML类图的多维概念模型的映射方法,为其应用奠定了实践基础。     

一种基于XML的数据仓库无数据表示模型

本文首先分析了元数据在数据仓库系统构建和使用过程中的重要作用及其表示模型应具有的特征,然后结合具体工程应用,提出了一种基于XML的数据仓库中无数据的表示模型。     

基于CWM的元数据管理系统中数据交换格式的研究

通用数据仓库元模型(CWM)是为数据仓库及商业智能环境间方便地交换元数据而制定的一个标准,可望成为未来数据仓库元数据管理的规范。它采用XMI作为元数据交换的基本格式。该文给出了一个基于CWM的元数据管理原型系统MMS的逻辑框架,并重点介绍了该系统的数据转换功能,其中给出了转换的形式化描述和几种元数据的XMI编码。     

一种数据仓库的多维数据模型

数据模型是数据仓库研究的核心问题之一.很多研究表明,传统数据模型(如实体联系模型和关系模型)不能有效地表示数据仓库的数据结构和语义,也难以有效地支持联机分析处理(on-line analysis processing,简称OLAP).最近,人们提出了几种多维数据模型.但是,这些多维数据模型在表示数据仓库的复杂数据结构和语义以及OLAP操作方面仍显不足.该文以偏序和映射为基础,提出了一种新的多维数据模型.该数据模型能够充分表达数据仓库的复杂数据结构和语义,并提供一个以OLAP操作为核心的操作代数,支持层次结构间的复杂聚集操作序列,能够有效地支持OLAP应用.该数据模型支持聚集函数约束的概念,提供了表示层次结构间聚集函数约束的机制.     

Intelligent video surveillance system: 3-tier context-aware surveillance system with metadata

This paper presents an intelligent video surveillance system with the metadata rule for the exchange of analyzed information. We define the metadata rule for the exchange of analyzed information between intelligent video surveillance systems that automatically analyzes video data acquired from cameras. The metadata rule is to effectively index very large video surveillance databases and to unify searches and management between distributed or heterogeneous surveillance systems more efficiently. The system consists of low-level context-aware, high-level context-aware and intelligent services to generate metadata for the surveillance systems. Various contexts are acquired from physical sensors in monitoring areas for the low-level context-aware system. The situation is recognized in the high-level context-aware system by analyzing the context data collected in the low-level system. The system provides intelligent services to track moving objects in Fields Of View (FOVs) and to recognize human activities. Furthermore, the system supports real-time moving objects tracking with Panning, Tilting and Zooming (PTZ) cameras in overlapping and non-overlapping FOVs.     

数据仓库中元数据互通的研究

介绍了数据仓库中元数据 ,着重分析了元数据的管理和当前的标准化进程 ,提出了一种建立企业级元数据库并采用元数据标准来解决元数据互通的方法。     

An RM-ODP enterprise view for spatial data infrastructures

Spatial Data Infrastructures (SDIs) are large, open, distributed and standards-based information systems which intend to facilitate and promote the use of spatial data and spatial services on the Internet. Spatial data describe information tied with locations on Earth, while spatial services allow to manipulate spatial data following a Service Oriented Architecture. This paper proposes to model SDIs as federations of autonomous communities following the enterprise language of the ITU-T and ISO/IEC ‘Reference Model of Open Distributed Processing’ (RM-ODP), and the recently approved ‘Use of UML for ODP Systems Specifications’ (UML4ODP). The enterprise language of the RM-ODP provides a conceptual foundation to address several aspects of SDIs not previously considered from a systems architecture point of view. The use of UML4ODP provides a modeling language to facilitate the exchange of knowledge about SDI, and it is an opportunity to try this recent standard for a class of large and complex systems.     

基于统一视图的数据仓库体系结构及其构建方法*

为了构建支持企业决策分析的数据仓库,分析了传统数据仓库模型的局限性,提出了一个基于统一视图模型的数据仓库体系结构。该体系结构是在传统数据仓库模型的数据源和数据仓库之间增加一个统一标准层,并利用统一视图—资源数据和数据仓库—统一视图的两级映射,保证了数据的透明访问和模型本身良好的可用性,进而支持灵活的多数据仓库的构建。基于该体系结构,给出了统一视图模型的建立和数据仓库三层之间两级映射的方法,提出了一种新的基于统一视图模型的数据映射—抽取—装载数据仓库ETL建模过程,并开发了相应的数据仓库构建系统。应用表明,     

物联网环境中数据存储与查询机制研究

物联网中存在射频识别设备、传感器、智能嵌入设备等数目众多的异质设备,运行着用于标识、感知、处理和传送信息的各类服务,因而存在大量、非连续且时间敏感的数据。这类数据对保证物联网产业链的正常运转非常重要,但现有的数据管理方法不能有效解决其存储与查询问题。基于此,针对物联网数据多维、多态的特征,在面向服务的物联网数据管理框架上,以都柏林元数据标准为基础,制定了支持学习的物联网数据多级元数据标准。设计了从XML元数据描述到关系数据库存储的数据处理流程,提出了优化的数据存储与查询方案。最后对提出的存储与查询方案进行了验证。     

数据网格中高效的元数据管理策略

在数据网格中,提出了采用完全分布方式、基于智能Cache和语义的元数据管理策略,在管理策略中设计一个元数据管理系统的体系结构SMICC.该体系结构引入了智能Cache和语义信息的方式来对元数据系统进行管理,减少远程访问时间和增加查询的准确度.SMICC可以利用智能Cache和语义信息处理不同的情况,可以根据本地可用的元数据进行查询的分解形成子查询,用户可以从本地Cache中获得元数据查询结果的一部分,剩下的部分可以从远程位置获得.通过语义信息和已有元数据,不仅可以使系统达到更好的扩展性和可用性,而且在保证用户访问质量的同时也提高了访问效率.     

基于XML与Web服务的多源气田信息整合

气田信息整合的主要目标是实现自治、分布、异构数据源的自动数据交换，并为用户提供统一的全局数据视图。Web服务作为一种面向服务的分布式计算技术，提供了一种建立基于Web的复杂松耦合分布式系统框架。讨论了Web服务应用于信息整合的基本模式，提出了基于Web服务的气田信息整合体系结构和基于元数据的气田信息整合方法。系统采用XML作为数据表示和交换的标准，具有较强的灵活性和可维护性。