一种列存储数据仓库中的数据复用策略

数据复用是数据仓库管理中节约存储空间、提高查询效率的重要手段.列存储技术将来自同一属性的数据连续存储,极大地提高了数据仓库等分析型应用软件的性能,同时增加了复用的可行性和灵活性.为此,提出了一种列存储数据仓库中的数据复用策略.首先,利用模式匹配技术发掘候选可复用列,排除大量无法复用的数据列,在此基础上对候选可复用数据进行筛选和过滤,大大降低复用数据检测的复杂度.针对确定的可复用数据,提出了基于列存储的复用实现策略,分别给出了原始数据列、压缩数据列及索引数据列的复用实现方法.最后提出了基于复用数据的查询实现技术.在大规模数据仓库基准数据集上的实验结果验证了数据复用策略在减少存储量、节省数据装载时间及提升查询性能方面的有效性.     

GPU数据库OLAP优化技术研究

GPU数据库近年来在学术界和工业界吸引了大量的关注. 尽管一些原型系统和商业系统(包括开源系统)开发了作为下一代的数据库系统, 但基于GPU的OLAP引擎性能是否真的超过CPU系统仍然存有疑问, 如果能够超越, 那什么样的负载/数据/查询处理模型更加适合, 则需要更深入的研究. 基于GPU的OLAP引擎有两个主要的技术路线: GPU内存处理模式和GPU加速模式. 前者将所有的数据集存储在GPU显存来充分利用GPU的计算性能和高带宽内存性能, 不足之处在于GPU容量有限的显存制约了数据集大小以及稀疏访问模式的数据存储降低GPU显存的存储效率. 后者只在GPU显存中存储部分数据集并通过GPU加速计算密集型负载来支持大数据集, 主要的挑战在于如何为GPU显存选择优化的数据分布和负载分布模型来最小化PCIe传输代价和最大化GPU计算效率. 致力于将两种技术路线集成到OLAP加速引擎中, 研究一个定制化的混合CPU-GPU平台上的OLAP框架OLAP Accelerator, 设计CPU内存计算、GPU内存计算和GPU加速3种OLAP计算模型, 实现GPU平台向量化查询处理技术, 优化显存利用率和查询性能, 探索GPU数据库的不同的技术路线和性能特征. 实验结果显示GPU内存向量化查询处理模型在性能和内存利用率两方面获得最佳性能, 与OmniSciDB和Hyper数据库相比性能达到3.1和4.2倍加速. 基于分区的GPU加速模式仅加速了连接负载来平衡CPU和GPU端的负载, 能够比GPU内存模式支持更大的数据集.     

中国出生缺陷数据挖掘系统HealthyBaby

在国家"十一五"科技支撑计划"出生缺陷干预效果评价系统"和国家自然科学基金"亚复杂系统中干预规则挖掘"支持下,中国出生缺陷数据挖掘系统HealthyBaby的测试版本已经在具有30年数据的全国出生缺陷数据仓库运行.介绍并演示HealthyBaby特色技术,包括中国出生缺陷数据仓库的列存储方式,可视化的OLAP查询,出生缺陷地理信息分析BD-GIS技术,面向出生缺陷数据的关联、分类、聚类知识发现,干预规则挖掘,亚复杂系统等.     

基于列存储机制下多维数据仓库模型的优化与研究

通过对分布式列存储机制下多维数据仓库模型的研究,考虑到多维数据仓库模型上的关联和聚集操作常常会引入大量的数据迁移,提出一种有效的列存储机制下多维数据仓库模型的优化方法即结合层次编码技术。采用维表层次全局域编码和维表层次局部域编码相结合的方式对传统星型模型维表中的层次信息进行二进制编码整合,将维表的层次信息压缩进事实表形成无连接星型模型,并针对新模型下的数据特征提出一种复合压缩策略,以期减少分布式列存储机制下的OLAP操作引入的数据迁移并降低数据存储空间,提升系统的查询性能。实验结果表明,该优化方法是可行且有效的。     

基于行存储模型的模拟列存储策略研究

列存储模型在只读的数据仓库应用中表现出非常好的性能,很多研究表明对于典型的OLAP查询,列存储数据库的性能大大优于行存储数据库.根据列存储模型的特性及数据处理特点,在传统的行存储模型关系数据库中模拟列存储的存储模式及数据处理过程,并通过优化的基于聚类的列存储模型、全索引模型与典型的行存储方式进行类比性能测试.实验结果显示,采用传统行存储模型模拟的列存储模型针对OLAP类查询具有很高的数据访问速度及查询性能,整体性能介于行存储数据库与列存储数据库之间.对于实际应用系统来说,该方案减少了整体系统部署代价,而且无需为提高OLAP查询的性能增加额外系统(列存储)投入,从而为OLAP应用提供了良好的性能支持.     

用字典法压缩数据仓库数据方案

数据仓库中的数据量是呈几何级数增长的。数据量的急剧增长严重影响数据仓库系统的性能,降低数据库系统的查询效率。文章在分析数据仓库数据特性的基础上,提出了使用字典法压缩数据仓库数据的方案。在对数据进行压缩的同时,也极大地改善了数据库系统的性能。     

列存储系统面向列的连接顺序优化研究

连接操作是影响列存储数据查询效率的重要操作之一,对于列存储系统中的连接操作优化,以往的研究工作大多专注于对数据组织结构的优化以及辅助物理结构的建立上,极少涉及逻辑层特别是早期的连接策略优化.为此,根据列存储数据的特点和分析型查询需求的特征,提出了一种新的列存储连接优化方法.该方法采用提早优化的策略,使用“事实表下推”的优化规则,并在多事实表查询条件下引入浓密树进行连接顺序决策,以较小的时空复杂度获得“最优”的连接执行顺序.使用代价估计模型对提出的连接策略优化方法进行了理论验证.同时,在大规模数据仓库基准数据集SSB上通过实验验证了提旱优化机制及下推规则的有效性.     

HUABASE:基于列存储的关系型数据库系统

HUABASE是基于列存储的关系型数据库系统.列存储技术的特点是数据查询效率高、读磁盘少、存储空间少,是构建数据仓库的理想架构.HUABASE实现了多种数据压缩机制、查询优化和稀疏索引技术,在支持高效率的商业智能方面具有良好的发展前景,可以帮助企业轻松做出明智的业务经营决策.     

列存储数据仓库中启发式查询优化机制

研究和实践表明列存储更加适合于大规模数据集上的即席查询的“读优化”应用需求.然而由于列存储的处理对象是列,此时传统的基于规则的查询优化方法并不完全适用.文中首先比较了列存储系统中查询优化与行存储系统的不同,在此基础上提出适合于列存储的启发式查询优化机制,其中包括启发式优化策略、重写规则、左深连接树结构和相关算法.实验表...     

Native XML数据库XSQS查询处理系统的设计与实现

XML已经成为互联网中数据表示和数据交换事实上的标准.实现专门对于XML数据的有效存储和查询变得十分重要.报告了当前NativeXML数据库在国内外的发展情况,讨论了一种NativeXML数据库管理系统原型XSQS的功能及概要设计.系统采用了基于压缩表的存储方式和索引结构,采用了基于结果集的查询处理方法.重点讨论了系统查询处理部分的设计及实现.实践结果表明,这种存储索引结构和查询方式相结合,使系统在查询效率上有了显著的提高.     

Only connect: teaching, technology and telesis

Abstract This paper describes an approach to the design of interactive multimedia materials being developed in a European Community project. The developmental process is seen as a dialogue between technologists and teachers. This dialogue is often problematic because of the differences in training, experience and culture between them. Conditions needed for fruitful dialogue are described and the generic model for learning design used in the project is explained.     

European Community policy and the market

Abstract This paper starts with some reflections on the policy considerations and priorities which are shaping European Commission (EC) research programmes. Then it attempts to position the current projects which seek to capitalise on information and communications technologies for learning in relation to these priorities and the apparent realities of the marketplace. It concludes that while there are grounds to be optimistic about the contribution EC programmes can make to the efficiency and standard of education and training, they are still too technology driven.     

一种自适应子融合集成多分类器方法

融合集成方法已经广泛应用在模式识别领域,然而一些基分类器实时性能稳定性较差,导致多分类器融合性能差,针对上述问题本文提出了一种新的基于多分类器的子融合集成分类器系统。该方法考虑在度量层融合层次之上通过对各类基多分类器进行动态选择,票数最多的类别作为融合系统中对特征向量识别的类别,构成一种新的自适应子融合集成分类器方法。实验表明,该方法比传统的分类器以及分类融合方法识别准确率明显更高,具有更好的鲁棒性。     

Avoiding semantic and temporal gaps in developing software intensive systems

Development of software intensive systems (systems) in practice involves a series of self-contained phases for the lifecycle of a system. Semantic and temporal gaps, which occur among phases and among developer disciplines within and across phases, hinder the ongoing development of a system because of the interdependencies among phases and among disciplines. Such gaps are magnified among systems that are developed at different times by different development teams, which may limit reuse of artifacts of systems development and interoperability among the systems. This article discusses such gaps and a systems development process for avoiding them.     

Designing economic np control charts: A programmed simulation approach

This paper presents control charts models and the necessary simulation software for the location of economic values of the control parameters. The simulation program is written in FORTRAN, requires only 10K of main storage, and can run on most mini and micro computers. Two models are presented - one describes the process when it is operating at full capacity and the other when the process is operating under capacity. The models allow the product quality to deteriorate to a further level before an existing out-of-control state is detected, and they can also be used in situations where no prior knowledge exists of the out-of-control causes and the resulting proportion defectives.     

The development of robot art

Going through a few examples of robot artists who are recognized worldwide, we try to analyze the deepest meaning of what is called “robot art” and the related art field definition. We also try to highlight its well-marked borders, such as kinetic sculptures, kinetic art, cyber art, and cyberpunk. A brief excursion into the importance of the context, the message, and its semiotics is also provided, case by case, together with a few hints on the history of this discipline in the light of an artistic perspective. Therefore, the aim of this article is to try to summarize the main characteristics that might classify robot art as a unique and innovative discipline, and to track down some of the principles by which a robotic artifact can or cannot be considered an art piece in terms of social, cultural, and strictly artistic interest. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

Explanation and prediction: an architecture for default and abductive reasoning

Although there are many arguments that logic is an appropriate tool for artificial intelligence, there has been a perceived problem with the monotonicity of classical logic. This paper elaborates on the idea that reasoning should be viewed as theory formation where logic tells us the consequences of our assumptions. The two activities of predicting what is expected to be true and explaining observations are considered in a simple theory formation framework. Properties of each activity are discussed, along with a number of proposals as to what should be predicted or accepted as reasonable explanations. An architecture is proposed to combine explanation and prediction into one coherent framework. Algorithms used to implement the system as well as examples from a running implementation are given.     

Three Process Perspectives: Organizations, Teams, and People

This paper provides the author's personal views and perspectives on software process improvement. Starting with his first work on technology assessment in IBM over 20 years ago, Watts Humphrey describes the process improvement work he has been directly involved in. This includes the development of the early process assessment methods, the original design of the CMM, and the introduction of the Personal Software Process (PSP)^SM and Team Software Process (TSP){^SM}. In addition to describing the original motivation for this work, the author also reviews many of the problems he and his associates encountered and why they solved them the way they did. He also comments on the outstanding issues and likely directions for future work. Finally, this work has built on the experiences and contributions of many people. Mr. Humphrey only describes work that he was personally involved in and he names many of the key contributors. However, so many people have been involved in this work that a full list of the important participants would be impractical.     

基于复小波噪声方差显著修正的SAR图像去噪

提出了一种基于复小波域统计建模与噪声方差估计显著性修正相结合的合成孔径雷达(Synthetic Aperture Radar,SAR)图像斑点噪声滤波方法。该方法首先通过对数变换将乘性噪声模型转化为加性噪声模型,然后对变换后的图像进行双树复小波变换(Dualtree Complex Wavelet Transform,DCWT),并对复数小波系数的统计分布进行建模。在此先验分布的基础上,通过运用贝叶斯估计方法从含噪系数中恢复原始系数,达到滤除噪声的目的。实验结果表明该方法在去除噪声的同时保留了图像的细节信息,取得了很好的降噪效果。     

How do children do mathematics with LOGO?

Abstract  This paper considers some results of a study designed to investigate the kinds of mathematical activity undertaken by children (aged between 8 and 11) as they learned to program in LOGO. A model of learning modes is proposed, which attempts to describe the ways in which children used and acquired understanding of the programming/mathematical concepts involved. The remainder of the paper is concerned with discussing the validity and limitations of the model, and its implications for further research and curriculum development.