Ternary Sparse Matrix Representation for Volumetric Mesh Subdivision and Processing on GPUs

In this paper, we present a novel volumetric mesh representation suited for parallel computing on modern GPU architectures. The data structure is based on a compact, ternary sparse matrix storage of boundary operators. Boundary operators correspond to the first‐order top‐down relations of k‐faces to their (k ? 1)‐face facets. The compact, ternary matrix storage format is based on compressed sparse row matrices with signed indices and allows for efficient parallel computation of indirect and bottom‐up relations. This representation is then used in the implementation of several parallel volumetric mesh algorithms including Laplacian smoothing and volumetric Catmull‐Clark subdivision. We compare these algorithms with their counterparts based on OpenVolumeMesh and achieve speedups from 3× to 531×, for sufficiently large meshes, while reducing memory consumption by up to 36%.     

Classification of run-length encoded binary data

In classification of binary featured data, distance computation is carried out by considering each feature. We represent the given binary data as run-length encoded data. This would lead to a compact or compressed representation of data. Further, we propose an algorithm to directly compute the Manhattan distance between two such binary encoded patterns. We show that classification of data in such compressed form would improve the computation time by a factor of 5 on large handwritten data. The scheme is useful in large data clustering and classification which depend on distance measures.     

State‐of‐the‐Art in Compressed GPU‐Based Direct Volume Rendering

Great advancements in commodity graphics hardware have favoured graphics processing unit (GPU)‐based volume rendering as the main adopted solution for interactive exploration of rectilinear scalar volumes on commodity platforms. Nevertheless, long data transfer times and GPU memory size limitations are often the main limiting factors, especially for massive, time‐varying or multi‐volume visualization, as well as for networked visualization on the emerging mobile devices. To address this issue, a variety of level‐of‐detail (LOD) data representations and compression techniques have been introduced. In order to improve capabilities and performance over the entire storage, distribution and rendering pipeline, the encoding/decoding process is typically highly asymmetric, and systems should ideally compress at data production time and decompress on demand at rendering time. Compression and LOD pre‐computation does not have to adhere to real‐time constraints and can be performed off‐line for high‐quality results. In contrast, adaptive real‐time rendering from compressed representations requires fast, transient and spatially independent decompression. In this report, we review the existing compressed GPU volume rendering approaches, covering sampling grid layouts, compact representation models, compression techniques, GPU rendering architectures and fast decoding techniques.     

HCCMeshes: Hierarchical‐Culling oriented Compact Meshes

Hierarchical culling is a key acceleration technique used to efficiently handle massive models for ray tracing, collision detection, etc. To support such hierarchical culling, bounding volume hierarchies (BVHs) combined with meshes are widely used. However, BVHs may require a very large amount of memory space, which can negate the benefits of using BVHs. To address this problem, we present a novel hierarchical‐culling oriented compact mesh representation, HCCMesh, which tightly integrates a mesh and a BVH together. As an in‐core representation of the HCCMesh, we propose an i‐HCCMesh representation that provides an efficient random hierarchical traversal and high culling efficiency with a small runtime decompression overhead. To further reduce the storage requirement, the in‐core representation is compressed to our out‐of‐core representation, o‐HCCMesh, by using a simple dictionary‐based compression method. At runtime, o‐HCCMeshes are fetched from an external drive and decompressed to the i‐HCCMeshes stored in main memory. The i‐HCCMesh and o‐HCCMesh show 3.6:1 and 10.4:1 compression ratios on average, compared to a naively compressed (e.g., quantized) mesh and BVH representation. We test the HCCMesh representations with ray tracing, collision detection, photon mapping, and non‐photorealistic rendering. Because of the reduced data access time, a smaller working set size, and a low runtime decompression overhead, we can handle models ten times larger in commodity hardware without the expensive disk I/O thrashing. When we avoid the disk I/O thrashing using our representation, we can improve the runtime performances by up to two orders of magnitude over using a naively compressed representation.     

Gradual model generator for single-pass clustering

We present an algorithm for generating a mixture model from a data set by converting the data into a model. The method is applicable when only part of the data fits in the main memory at the same time. The generated model is a Gaussian mixture model but the algorithm can be adapted to other types of models, too. The user cannot specify the size of the generated model. We also introduce a post-processing method, which can reduce the size of the model without using the original data. This will result in a more compact model with fewer components, but with approximately the same representation accuracy as the original model. Our comparisons show that the algorithm produces good results and is quite efficient. The whole process requires only 0.5-10% of the time spent by the expectation-maximization algorithm.     

压缩数据集上的快速Cube计算方法

现有压缩数据集上的Cube计算方法只适用于稀疏数据,针对该问题,设计一种用于压缩常量和基本单一元组的压缩方法并提出一种新的Cube算法。该算法在计算过程中无需解压缩、计算速度快、数据压缩率高,适用于冰山计算。实验结果表明,与自底向上立方体算法相比,新算法计算速度快、所需存储空间小。     

面向数据流的频繁模式挖掘研究*

数据流的无限性、高速性使得经典的频繁模式挖掘方法难以适用到数据流中。针对数据流的特点,对数据流中频繁模式挖掘问题进行了研究,提出了数据流频繁模式挖掘算法FP-SegCount。该算法将数据流分段并利用改进的FP-growth算法挖掘分段中的频繁项集,然后利用Count-Min Sketch进行项集计数。算法解决了压缩统计和计算快速高效的问题。通过实验分析,FP-SegCount算法是有效的。     

压缩传感在无线视频监控中的应用研究*

图像采集数据量大是制约视频监控系统向无线化方向发展的主要因素,提出利用压缩传感进行视频图像的采样,为无线视频监控带来一种新的应用研究。为了减少图像稀疏分解过程的计算量和存储量,在匹配追踪算法的基础上,引入量子遗传算法,实现快速的图像稀疏表示。以Fourier矩阵作为压缩传感的测量矩阵,能有效减少测量数据量,并提高重构图像的质量。仿真实验证明,采用压缩传感所得到的测量数据量远小于传统采样方法所获的数据量,突破了传统信号采样的瓶颈,提高了采样效率,最终获取的压缩测量值能够很好地恢复为监控场景。     

Matrix Trees

We propose a new data representation for octrees and kd‐trees that improves upon memory size and algorithm speed of existing techniques. While pointerless approaches exploit the regular structure of the tree to facilitate efficient data access, their memory footprint becomes prohibitively large as the height of the tree increases. Pointerbased trees require memory consumption proportional to the number of tree nodes, thus exploiting the typical sparsity of large trees. Yet, their traversal is slowed by the need to follow explicit pointers across the different levels. Our solution is a pointerless approach that represents each tree level with its own matrix, as opposed to traditional pointerless trees that use only a single vector. This novel data organization allows us to fully exploit the tree's regular structure and improve the performance of tree operations. By using a sparse matrix data structure we obtain a representation that is suited for sparse and dense trees alike. In particular, it uses less total memory than pointer‐based trees even when the data set is extremely sparse. We show how our approach is easily implemented on the GPU and illustrate its performance in typical visualization scenarios.     

Scalable and queryable compressed storage structure for raster data

Compact data structures are storage structures that combine a compressed representation of the data and the access mechanisms for retrieving individual data without the need of decompressing from the beginning. The target is to be able to keep the data always compressed, even in main memory, given that the data can be processed directly in that form. With this approach, we obtain several benefits: we can load larger datasets in main memory, we can make a better usage of the memory hierarchy, and we can obtain bandwidth savings in a distributed computational scenario, without wasting time in compressing and decompressing data during data exchanges.In this work, we follow a compact data structure approach to design a storage structure for raster data, which is commonly used to represent attributes of the space (temperatures, pressure, elevation measures, etc.) in geographical information systems. As it is common in compact data structures, our new technique is not only able to store and directly access compressed data, but also indexes its content, thereby accelerating the execution of queries.Previous compact data structures designed to store raster data work well when the raster dataset has few different values. Nevertheless, when the number of different values in the raster increases, their space consumption and search performance degrade. Our experiments show that our storage structure improves previous approaches in all aspects, especially when the number of different values is large, which is critical when applying over real datasets. Compared with classical methods for storing rasters, namely netCDF, our method competes in space and excels in access and query times.     

基于MatLab的海量数据处理方法

MatLab是MathWorks公司推出的一种科学计算软件,在使用MatLab进行数据处理过程中,常常会用到循环语句逐点处理数据,当数据量巨大时,经常会出现耗时长久的情况。将循环运算转换为矩阵运算,充分利用MatLab高效率的矩阵运算特点,可以缩短数据处理时间。通过一个算例分析了创建大容量矩阵的方法：如扩大内存空间、采用短字节数值数据类型等。算例运行结果表明,通过分配更多的内存空间进行大容量矩阵运算,可以显著缩短数据处理的时间,从而实现空间换时间的目的。     

Streaming compression of hexahedral meshes

We describe a method for streaming compression of hexahedral meshes. Given an interleaved stream of vertices and hexahedra our coder incrementally compresses the mesh in the presented order. Our coder is extremely memory efficient when the input stream also documents when vertices are referenced for the last time (i.e. when it contains topological finalization tags). Our coder then continuously releases and reuses data structures that no longer contribute to compressing the remainder of the stream. This means in practice that our coder has only a small fraction of the whole mesh in memory at any time. We can therefore compress very large meshes—even meshes that do not fit in memory. Compared to traditional, non-streaming approaches that load the entire mesh and globally reorder it during compression, our algorithm trades a less compact compressed representation for significant gains in speed, memory, and I/O efficiency. For example, on the 456k hexahedra “blade” mesh, our coder is twice as fast and uses 88 times less memory (only 3.1 MB) with the compressed file increasing about 3% in size. We also present the first scheme for predictive compression of properties associated with hexahedral cells.     

Streaming data reduction using low-memory factored representations

Many special purpose algorithms exist for extracting information from streaming data. Constraints are imposed on the total memory and on the average processing time per data item. These constraints are usually satisfied by deciding in advance the kind of information one wishes to extract, and then extracting only the data relevant for that goal. Here, we propose a general data representation that can be computed using modest memory requirements with limited processing power per data item, and yet permits the application of an arbitrary data mining algorithm chosen and/or adjusted after the data collection process has begun. The new representation allows for the at-once analysis of a significantly larger number of data items than would be possible using the original representation of the data. The method depends on a rapid computation of a factored form of the original data set. The method is illustrated with two real datasets, one with dense and one with sparse attribute values.     

Value locality based storage compression memory architecture for ECG sensor node

This paper proposes a value compression memory architecture for QRS detection in ultra-low-power ECG sensor nodes. Based on the exploration of value spatial locality in the most critical preprocessing stage of the ECG algorithm, a cost efficient compression strategy, which reorganizes several adjacent sample values into a base value with several displacements, is proposed. The displacements will be half or quarter scale quantifications; as a result, the storage size is reduced. The memory architecture saves memory space by storing compressed data with value spatial locality into a compressed memory section and by using a small, uncompressed memory section as backup to store the uncompressed data when a value spatial locality miss occurs. Furthermore,a low-power accession strategy is proposed to achieve low-power accession. An embodiment of the proposed memory architecture has been evaluated using the MIT/BIH database, the proposed memory architecture and a low-power accession strategy to achieve memory space savings of 32.5% and to achieve a 68.1% power reduction with a negligible performance reduction of 0.2%.     

Compact representation of Web graphs with extended functionality

The representation of large subsets of the World Wide Web in the form of a directed graph has been extensively used to analyze structure, behavior, and evolution of those so-called Web graphs. However, interesting Web graphs are very large and their classical representations do not fit into the main memory of typical computers, whereas the required graph algorithms perform inefficiently on secondary memory. Compressed graph representations drastically reduce their space requirements while allowing their efficient navigation in compressed form. While the most basic navigation operation is to retrieve the successors of a node, several important Web graph algorithms require support for extended queries, such as finding the predecessors of a node, checking the presence of a link, or retrieving links between ranges of nodes. Those are seldom supported by compressed graph representations.This paper presents the k²-tree, a novel Web graph representation based on a compact tree structure that takes advantage of large empty areas of the adjacency matrix of the graph. The representation not only retrieves successors and predecessors in symmetric fashion, but also it is particularly efficient to check for specific links between nodes, or between ranges of nodes, or to list the links between ranges. Compared to the best representations in the literature supporting successor and predecessor queries, our technique offers the least space usage (1–3 bits per link) while supporting fast navigation to predecessors and successors ($2–8\mathrm{\mu }\mathrm{s}$ per neighbor retrieved) and sharply outperforming the others on the extended queries. The representation is also of general interest and can be used to compress other kinds of graphs and data structures.     

ReduceM: Interactive and Memory Efficient Ray Tracing of Large Models

We present a novel representation and algorithm, ReduceM, for memory efficient ray tracing of large scenes. ReduceM exploits the connectivity between triangles in a mesh and decomposes the model into triangle strips. We also describe a new stripification algorithm, Strip‐RT, that can generate long strips with high spatial coherence. Our approach uses a two‐level traversal algorithm for ray‐primitive intersection. In practice, ReduceM can significantly reduce the storage overhead and ray trace massive models with hundreds of millions of triangles at interactive rates on desktop PCs with 4‐8GB of main memory.     

基于稀疏图的鲁棒谱聚类算法

为提高谱聚类算法的鲁棒性,基于稀疏编码在图的构造中提出一种改进L1稀疏表示图模型。每个样本表示为数据集中其他样本的稀疏线性组合,得到稀疏图的边权表示,所构造的稀疏图对数据噪声有很好的鲁棒性,同时能够反映数据局部线性结构。采用稀疏矩阵表示,该方法能够大大降低存储量和计算量,因而对于处理较大规模问题有着较好的可伸缩性。人工数据和实际数据上的谱聚类实验验证了该算法的性能。     

基于滑动窗口的数据流频繁闭项集挖掘

针对数据流的特点,根据Moment算法提出一种基于频繁闭项集挖掘的增量式维护算法。该算法通过滑动窗口增量更新数据流中的事务,采取一种高效的项的位序列表示方法降低窗口滑动的时问和空间复杂度,应用压缩的模式树进行频繁闭项集检查,以确保挖掘结果的准确性。实验证明了该方法的有效性。     

Compile and Run-Time Support for the Parallelization of Sparse Matrix Updating Algorithms

This work presents a survey of the capabilities that the sparse computation offers for improving performance when parallelized, either automatically or through a data-parallel compiler. The characterization of a sparse code gets more complicated as code length increases: Access patterns change from loop to loop, thus making necessary to redefine the parallelization strategy. While dense computation solely offers the possibility of redistributing data structures, several other factors influence the performance of a code excerpt in the sparse field, like source data representation on file, compressed data storage in memory, the creation of new nonzeroes at run-time (fill-in) or the number of processors available. We analize the alternatives that arise from each issue, providing a guideline for the underlying compilation work and illustrating our techniques with examples on the Cray T3E.     

A new algorithm for fast mining frequent itemsets using N-lists

Mining frequent itemsets has emerged as a fundamental problem in data mining and plays an essential role in many important data mining tasks.In this paper,we propose a novel vertical data representation called N-list,which originates from an FP-tree-like coding prefix tree called PPC-tree that stores crucial information about frequent itemsets.Based on the N-list data structure,we develop an efficient mining algorithm,PrePost,for mining all frequent itemsets.Efficiency of PrePost is achieved by the following three reasons.First,N-list is compact since transactions with common prefixes share the same nodes of the PPC-tree.Second,the counting of itemsets’ supports is transformed into the intersection of N-lists and the complexity of intersecting two N-lists can be reduced to O(m + n) by an efficient strategy,where m and n are the cardinalities of the two N-lists respectively.Third,PrePost can directly find frequent itemsets without generating candidate itemsets in some cases by making use of the single path property of N-list.We have experimentally evaluated PrePost against four state-of-the-art algorithms for mining frequent itemsets on a variety of real and synthetic datasets.The experimental results show that the PrePost algorithm is the fastest in most cases.Even though the algorithm consumes more memory when the datasets are sparse,it is still the fastest one.