基于EXN-Tree编码的XML结构连接算法研究

提出了EXN-Tree的概念,将XML文档树的节点映射到EXN-Tree,依据EXN-Tree的节点编码生成XML文档树节点数据结构。基于此新型的节点编码结构,就无序无索引节点集和有序有索引节点集两种情况下的XML结构连接算法展开研究,提出了一系列的结构连接算法,解决了无序无索引节点集和有序有索引节点集两种情况下的XML结构连接。分析表明该算法的I/O复杂性优于已有算法,具有良好的性能。     

XML压缩算法的分析与比较

XML为在Web上发布和交换异质数据提供了相当的灵活性。但由于这种语言自身就有冗余的特性所以XML文档在体积都大于有相同数据内容的其他类型的文档。随着XML在web上应用的扩展其数据大小自然也会随之而增加．而这实质上增加了数据的存储量、处理量和交换量，因此XML文档的体积问题也阻碍了XML的应用，特别是阻碍了XML在具有带宽和内存容量限制的应用上如移动通讯应用。在这篇文章中，我们将大致的介绍一下最近提出的几种针对XML的压缩算法并分析它们在解决XML文档体积问题上的技术和效能。     

XML压缩算法的分析与比较

XML为在Web上发布和交换异质数据提供了相当的灵活性。但由于这种语言自身就有冗余的特性所以XML文档在体积都大于有相同数据内容的其他类型的文档。随着XML在Web上应用的扩展其数据大小自然也会随之而增加,而这实质上增加了数据的存储量、处理量和交换量,因此XML文档的体积问题也阻碍了XML的应用,特别是阻碍了XML在具有带宽和内存容量限制的应用上如移动通讯应用。在这篇文章中,我们将大致的介绍一下最近提出的几种针对XML的压缩算法并分析它们在解决XML文档体积问题上的技术和效能。     

Comparative Analysis of XML Compression Technologies

XML provides flexibility in publishing and exchanging heterogeneous data on the Web. However, the language is by nature verbose and thus XML documents are usually larger in size than other specifications containing the same data content. It is natural to expect that the data size will continue to grow as XML data proliferates on the Web. The size problem of XML documents hinders the applications of XML, since it substantially increases the costs of storing, processing and exchanging the data. The hindrance is more apparent in bandwidth- and memory-limited settings such as those applications related to mobile communication. In this paper, we survey a range of recently proposed XML specific compression technologies and study their efforts and capabilities to overcome the size problem. First, by categorizing XML compression technologies into queriable and unqueriable compressors, we explain the efforts in the representative technologies that aim at utilizing the exposed structure information from the input XML documents. Second, we discuss the importance of queriable XML compressors and assess whether the compressed XML documents generated from these technologies are able to support direct querying on XML data. Finally, we present a comparative analysis of the state-of-the-art XML conscious compression technologies in terms of compression ratio, compression and decompression times, memory consumption, and query performance.     

TexNN: Fast Texture Encoding Using Neural Networks

We present a novel deep learning‐based method for fast encoding of textures into current texture compression formats. Our approach uses state‐of‐the‐art neural network methods to compute the appropriate encoding configurations for fast compression. A key bottleneck in the current encoding algorithms is the search step, and we reduce that computation to a classification problem. We use a trained neural network approximation to quickly compute the encoding configuration for a given texture. We have evaluated our approach for compressing the textures for the widely used adaptive scalable texture compression format and evaluate the performance for different block sizes corresponding to 4 × 4, 6 × 6 and 8 × 8. Overall, our method (TexNN) speeds up the encoding computation up to an order of magnitude compared to prior compression algorithms with very little or no loss in the visual quality.     

XML数据压缩技术的研究进展

XML的自描述特性使得XML数据中存在大量冗余信息。如何压缩XML数据以提高XML数据管理的效率成为一个新的研究领域。从2001年起,XML数据压缩技术的研究已经取得了很多研究成果。从面向存储的XML数据压缩技术、面向查询的XML数据压缩技术和面向特定应用的XML数据压缩技术三个方面综述了XML数据压缩技术的研究进展,讨论了目前存在的主要问题和需要进一步研究的方向,并提供了广泛的参考文献。     

New adaptive compressors for natural language text

Semistatic byte‐oriented word‐based compression codes have been shown to be an attractive alternative to compress natural language text databases, because of the combination of speed, effectiveness, and direct searchability they offer. In particular, our recently proposed family of dense compression codes has been shown to be superior to the more traditional byte‐oriented word‐based Huffman codes in most aspects. In this paper, we focus on the problem of transmitting texts among peers that do not share the vocabulary. This is the typical scenario for adaptive compression methods. We design adaptive variants of our semistatic dense codes, showing that they are much simpler and faster than dynamic Huffman codes and reach almost the same compression effectiveness. We show that our variants have a very compelling trade‐off between compression/decompression speed, compression ratio, and search speed compared with most of the state‐of‐the‐art general compressors. Copyright © 2008 John Wiley & Sons, Ltd.     

利用OpenXML实现XML转换成SQL

在XML文档和SQL数据库表的基础上,讲解了两者相互转换的转换模型、转换算法及其具体实现方法。     

基于XBW变换的XML数据压缩查询方法

XML数据格式虽然具有易于创建和解析的特点,但数据格式过于冗长,难以实现具体的查询。该文引入XBW变换,将XML数据压缩成3个线性序列,使XML数据的查询处理从树形结构转移到这3个序列上。基于XBW变换导航、子路径查询和内容查询,采用Rank&;Select方法来实现查询。实验结果表明,在压缩率和压缩时间方面,XBWZIP的性能接近或者超过一些支持查询的XML压缩方法和一些通用压缩方法。     

Millau: an encoding format for efficient representation and exchange of XML over the Web

XML is poised to take the World Wide Web to the next level of innovation. XML data, large or small, with or without associated schema, will be exchanged between increasing number of applications running on diverse devices. Efficient storage and transportation of such data is an important issue. We have designed a system called Millau for efficient encoding and streaming of XML structures. In this paper we describe the Millau algorithms for compression of XML structures and data. Millau compression algorithms, in addition to separating structure and text for compression, take advantage of the associated schema (if available) in compressing the structure. Millau also defines a programming model corresponding to XML DOM and SAX for XML APIs for Millau streams of XML documents. Our experiments have shown significant performance gains of our algorithms and APIs. We describe some of these results in this paper. We also describe some applications of XML-based remote procedure calls and client-server applications based on Millau that take advantage of the compression and streaming technology defined by the system.     

COX:高压缩率的中文XML文档压缩技术

针对当前常用的XML压缩算法没有考虑中文特点的情况,结合中文与XML的特点,提出一种高压缩率的适合中文XML文档的压缩算法COX。利用中文分词技术对XML文档进行分词处理,通过统计词频后获得排序的词典,利用Huffman编码思想对高频及长词汇进行压缩编码;解析XML文档后,把文档元素进行分类,同一类型的元素放入同一容器之中;算法还特别针对数字类型的数据进行了特殊处理。实验结果显示,相对于通用的压缩软件,COX具有更好的压缩效果,但压缩和解压缩时间要慢一些。     

Parallel Marching Blocks: A Practical Isosurfacing Algorithm for Large Data on Many‐Core Architectures

Interactive isosurface visualisation has been made possible by mapping algorithms to GPU architectures. However, current state‐of‐the‐art isosurfacing algorithms usually consume large amounts of GPU memory owing to the additional acceleration structures they require. As a result, the continued limitations on available GPU memory mean that they are unable to deal with the larger datasets that are now increasingly becoming prevalent. This paper proposes a new parallel isosurface‐extraction algorithm that exploits the blocked organisation of the parallel threads found in modern many‐core platforms to achieve fast isosurface extraction and reduce the associated memory requirements. This is achieved by optimising thread co‐operation within thread‐blocks and reducing redundant computation; ultimately, an indexed triangular mesh can be produced. Experiments have shown that the proposed algorithm is much faster (up to 10×) than state‐of‐the‐art GPU algorithms and has a much smaller memory footprint, enabling it to handle much larger datasets (up to 64×) on the same GPU.     

Revisiting dictionary‐based compression

An attractive way to increase text compression is to replace words with references to a text dictionary given in advance. Although there exist a few works in this area, they do not fully exploit the compression possibilities or consider alternative preprocessing variants for various compressors in the latter phase. In this paper, we discuss several aspects of dictionary‐based compression, including compact dictionary representation, and present a PPM/BWCA‐oriented scheme, word replacing transformation, achieving compression ratios higher by 2–6% than the state‐of‐the‐art StarNT (2003) text preprocessor, working at a greater speed. We also present an alternative scheme designed for LZ77 compressors, with the advantage over StarNT of reaching up to 14% in combination with gzip. Copyright © 2005 John Wiley & Sons, Ltd.     

面向XPath执行的XML数据流压缩方法

由于XML(extensible markup language)本身是自描述的,所以XML数据流中存在大量冗余的结构信息.如何压缩XML数据流,使得在减少网络传输代价的同时有效支持压缩数据流上的查询处理,成为一个新的研究领域.目前已有的XML数据压缩技术,都需要扫描数据多遍,或者不支持数据流之上的实时查询处理.提出了一种XML数据流的压缩技术XSC(XML stream compression),实时完成XML数据流的压缩和解压缩,XSC动态构建XML元素事件序列字典并输出相关索引,能够根据XML数据流所遵从的DTD,产生XML元素事件序列图,在压缩扫描之前,产生更加合理的结构序列编码.压缩的XML数据流能够直接解压缩用于XPath的执行.实验表明,在XML数据流环境中,XSC在数据压缩率和压缩时间上要优于传统算法.同时,在压缩数据之上查询的执行代价是可以接受的.     

LAF: a new XML encoding and indexing strategy for keyword‐based XML search

As a large number of corpuses are represented, stored and published in XML format, how to find useful information from XML databases has become an increasingly important issue. Keyword search enables web users to easily access XML data without the need to learn a structured query language or to study complex data schemas. Most existing indexing strategies for XML keyword search are based upon Dewey encoding. In this paper, we proposed a new encoding method called Level Order and Father (LAF) for XML documents. With LAF encoding, we devised a new index structure, called two‐layer LAF inverted index, which can greatly decrease the space complexity compared with Dewey encoding‐based inverted index. Furthermore, with two‐layer LAF inverted index, we proposed a new keyword query algorithm called Algorithm based on Binary Search (ABS) that can quickly find all Smallest Lowest Common Ancestor. We experimentally evaluate two‐layer LAF inverted index and ABS algorithm on four real XML data sets selected from Wikipedia. The experimental results prove the advantages of our index method and querying algorithm. The space consumed by two‐layer LAF index is less than half of that consumed by Dewey inverted index. Moreover, ABS is about one to two orders of magnitude faster than the classic Stack algorithm. Concurrency and Computation: Practice and Experience, 2012.© 2012 Wiley Periodicals, Inc.     

A relational model for XML structural joins and their size estimations

XML structural joins, which evaluate the containment (ancestor-descendant) relationships between XML elements, are important operations of XML query processing. Estimating structural join size accurately and quickly is crucial to the success of XML query plan selection and the query optimization. XML structural joins are essentially complex θ-joins, which render well-known estimation techniques for relational equijoins, such as discrete cosine transform, wavelet transform, and sketch, not applicable. In this paper, we model structural joins from a relational point of view and convert the complex θ-joins to equijoins so that those well-known estimation techniques become applicable to structural join size estimation. Theoretical analyses and extensive experiments have been performed on these estimation methods. It is shown that discrete cosine transform requires the least memory and yields the best estimates among the three techniques. Compared with state-of-the-art method IM-DA-Est, discrete cosine transform is much faster, requires less memory, and yields comparable estimates.     

Automated classification of content components in technical communication

Automated classification is usually not adjusted to specialized domains due to a lack of suitable data collections and insufficient characterization of the domain‐specific content and its effect on the classification process. This work describes an approach for the automated multiclass classification of content components used in technical communication based on a vector space model. We show that differences in the form and substance of content components require an adaption of document‐based classification methods and validate our assumptions with multiple real‐world data sets in 2 languages. As a result, we propose general adaptions on feature selection and token weighting, as well as new ideas for the measurement of classifier confidence and the semantic weighting of XML‐based training data. We introduce several potential applications of our method and provide prototypical implementation. Our contribution beyond the state of the art is a dedicated procedure model for the automated classification of content components in technical communication, which outperforms current document‐centered or domain‐agnostic approaches.     

基于结构标记树的XML可查询压缩方法

针对支持查询的XML数据压缩方法存在的路径和数据重复等问题,通过去除XML数据中的重复路径,简化XML数据结构,提出结构标记树的概念及其生成算法,设计一种基于结构标记树的可查询XML数据压缩方法SSTQC,对XML数据进行压缩和组织查询。SSTQC一次扫描XML文档,具有较好的的压缩性能和查询效率。     

一种复杂XML Twig查询处理算法

根据复杂Twig查询的特点,充分利用DTD资源,建立一种基于DTD的索引结构,采用Dewey编码方法对XML文档进行统一编码,并提出一种基于DTD的复杂Twig查询处理算法STwigScan;查询时,通过扫描DTD索引,将复杂Twig查询定位在条件节点以及目标节点上,有效的减少查询处理算法的处理规模;实验证明,STwigScan算法处理规模比较小,查询效率比较高.