空间数据库索引管理系统的设计与实现

By the special character and wide application of spatial data,it is needed to design a kind of spatial database system management system(SDBMS),index management is a core part of SDBMS,the performance of a database is decided by the management function of index largely ,this paper defines from the structure of the minimum data item to index tree,index page‘s storage structure and memory management,designs and implements ultimately a spatial database index management system.     

A cost effective fault-tolerant scheme for RAIDs

The rapid progress in mass storage technology has made it possible for designers to implement large data storage systems for a variety of applications.One of the efficient ways to build large storage systems is to use RAIDs only when one error occurs .But in large RAIDs systems ,the fault probability will increase when the number of disks increases ,and the use of disks with big storage capacity will cause the recovering time to prolong,thus the probability of the second disk‘‘‘‘‘‘‘‘s fault will incerease Therefore,it is necessary to develop methods to recover data when two or more errors have occurred In this paper,a fault tolerant scheme is proposed based on extended Reed-Solomon code,a recovery procedure is designed to correct up to two errors which is implemented by software and hardware together,and the scheme is verified by computer simulation,In this scheme,only two redundant disks are used to recover up to two disks‘‘‘‘‘‘‘‘ fault .The encoding and decoding methods,and the implementation based on software and hardware are described.The application of the scheme in software RAIDs that are builit in cluster computers are also described .Compared with the existing methods such as EVENODD and DH ,the proposed scheme has distinct improvement in implementation and redundancy.     

Approaches for scaling DBSCAN algorithm to large spatial databases

The huge amount of information stored in datablases owned by coporations(e.g.retail,financial,telecom) has spurred a tremendous interest in the area of knowledge discovery and data mining.Clustering.in data mining,is a useful technique for discovering intersting data distributions and patterns in the underlying data,and has many application fields,such as statistical data analysis,pattern recognition,image processsing,and other business application,s Although researchers have been working on clustering algorithms for decades,and a lot of algorithms for clustering have been developed,there is still no efficient algorithm for clustering very large databases and high dimensional data,As an outstanding representative of clustering algorithms,DBSCAN algorithm shows good performance in spatial data clustering.However,for large spatial databases,DBSCAN requires large volume of memory supprot and could incur substatial I/O costs because it operates directly on the entrie database,In this paper,several approaches are proposed to scale DBSCAN algorithm to large spatial databases.To begin with,a fast DBSCAN algorithm is developed.which considerably speeeds up the original DBSCAN algorithm,Then a sampling based DBSCAN algorithm,a partitioning-based DBSCAN algorithm,and a parallel DBSCAN algorithm are introduced consecutively.Following that ,based on the above-proposed algorithms,a synthetic algorithm is also given,Finally,some experimental results are given to demonstrate the effectiveness and efficiency of these algorithms.     

POTENTIAL: A highly adaptive core of parallel database system

POTENTIAL is a virtual database machine based on general computing platforms,especially parllel computing platforms.It provides a complete solution to high-performance database systems by a ‘virtual processor virtual data bus virtual memory‘ architecture.Virtual processors manage all CPU resources in the system,on which various operations are running.Virtual data bus is responsible for the management of data transmission between associated operations.which forms the higes of the entire system.Virtual memory provides efficient data storage and buffering mechanisms that conform to data reference behaviors in database systems.The architecture of POTENTIAL is very clear and has many good features,including high efficiency,high scalability,high extensibility,high portability,etc.     

Spatial database for Geographic Information System （GIS） applications

This paper explains the concepts of Oracle spatial as a way to store and analyze spatial data. Oracle spatial is designed to make spatial data management easier and more natural to users of location-enabled applications and Geographic Information System （GIS） applications. Once spatial data is stored in an Oracle database, it can be easily manipulated, retrieved, and related to all other data stored in the database. The data that indicates the Earth location （such as longitude and latitude） of these rendered objects is the spatial data. When the map is rendered, this spatial data is used to project the locations of the objects on a two-dimensional piece of paper. A GIS is often used to store, retrieve, and render this Earth-relative spatial data. Types of spatial data （other than GIS data） that can be stored using Oracle spatial include data from Computer Aided Design （CAD） and Computer Aided Manufacturing （CAM） systems.     

Design and implementation of an Interoperable Object Platform for Multi-Databases

In this paper,the authors present the design and implementation of an Interoperable Object Platform for Multi-Databases(IOPMD).The aim of the system is to provide a uniform object view and a set of tools for object manipulation and query based on heterogeneous multiple data sources under client/server environment.The common object model is compatible with ODMG2.0 and OMG‘s CORBA,which provides main OO features such as OID,attribute ,method,inheritance,reference,etc.Three types of interfaces,namely Vface,IOQL and C API,are given to provide the database programmer with tools and functionalities for application development.Nested transactions and compensating technology are adopted in transaction manager,In discussing some key implementation techniques.Translation and mapping approaches from various schemata to a common object schema are proposed.Buffer management provides the data caching policy and consistency maintenance of cached data.Version management presents some operations based on the definitions in semantic version model,and introduces the implementation of the semantic version graph.     

Range Query Processing in Multidisk Systems

In order to reduce the disk access time,a database can be stored on several simultaneously accessible disks.In this paper,we are concerned with the dynamic d-attribute database allocation problem for range queries,An allocation method,called coordinate moule allocation method,is proposed to allocate data in a d-attribute database among disks so that the maximum disk accessing concurrency can be achieved for range queries.Our analysis and experiments show that the method achieves the optimum or near-optimum parallelism for range queries.The paper offers the conditions under which the method is optimal .The worst case bounds of the performance of the method are also given.In addition,the parallel algorithm of processing range queries in described at the end of the paper.The method has been used in the statistic and scientific database management system whic is being designed by us.     

时空信息的层次存储和管理

To get high-performance data accessing irt GIS,a hierarchical model for spatiotemporal data presentation and storage, which is composed of four levelsgeimral level,global level,theme level and view level, is provided. The general level, global level and theme level are for logical presentation and physical storage, and the view level is for logical presentation only. The general level provides uniform presentation for spatial objects by time measurement. The global level.as the mediate level in data presentatmn, simplifies the decomposition and recomposition for spatial data, which can lead to fast data access and easy level-to4evel data transformation. The theme level classifies the spatial object according the objects’original type. Which is a semantk level. Lastly, the view level provides the data access interrace for end-user.     

时空数据库索引机制研究

Spatio-temporal database manages the large amount of spatial objects that change over time.It is necessary to query the spatio-temporal objects of the past and the current and to anticipate the future of spatio-temporal objects.It is important to design an efficient index mechanism for accessing the spatio-temporal data efficiently.The paper analyzes the features of the spatio-temporal objects,studies the methods of spatio-temporal index mechanisms,classifies the index mechanisms,and discusses the key technologies of spatio-temporal indexes.And it also presents the index methods of STADBS that we are studying.     

Spatial Data Mining to Support Environmental Management and Decision Making--A Case Study in Brazil

The growth of geo-technologies and the development of methods for spatial data collection have resulted in large spatial data repositories that require techniques for spatial information extraction, in order to transform raw data into useful previously unknown information. However, due to the high complexity of spatial data mining, the need for spatial relationship comprehension and its characteristics, efforts have been directed towards improving algorithms in order to provide an increase of performance and quality of results. Likewise, several issues have been addressed to spatial data mining, including environmental management, which is the focus of this paper. The main original contribution of this work is the demonstration of spatial data mining using a novel algorithm with a multi-relational approach that was applied to a database related to water resource from a certain region of S~o Paulo State, Brazil, and the discussion about obtained results. Some characteristics involving the location of water resources and the profile of who is administering the water exploration were discovered and discussed.     

倒排文件页式存储方法

页是磁盘与内存进行数据交换的基本单位,它在操作系统、数据库管理系统以及倒排文件的数据组织中占据十分重要的地位。为减少倒排索引的磁盘 I/O 读写开销,提出了一种倒排文件按页存储的构建方法,实现了按页读写文件。该方法主要包括磁盘I/O层设计、页管理器设计以及堆文件管理器设计三个部分,实现了页大小可变的分块式数据文件管理,支持页内定长记录、变长记录的组装以及超长数据记录的跨页存储。经实验测试,结果表明该方法是行之有效的,可以将其应用到实际的垂直搜索引擎中。     

Efficient Management of Persistent Knowledge

Although computer speed has steadily increased and memory is getting cheaper, the need for storage managers to deal efficiently with applications that cannot be held into main memory is vital. Dealing with large quantities of clauses implies the use of persistent knowledge and thus, indexing methods are essential to access efficiently the subset of clauses relevant to answering a query. We introduce PerKMan, a storage manager that uses G-trees, and aims at efficient manipulation of large amounts of persistent knowledge. PerKMan may be connected to Prolog systems that offer an external C language interface. As well as the fact that the storage manager allows different arguments of a predicate to share a common index dimension in a novel manner, it indexes rules and facts in the same manner. PerKMan handles compound terms efficiently and its data structures adapt their shape to large dynamic volumes of clauses, no matter what the distribution. The storage manager achieves fast clause retrieval and reasonable use of disk space.     

Storage Support for Multidimensional Discrete Data in Multimedia Databases

Multidimensional discrete data (MDD) is enormously important for multimedia applications since diverse types of such data are used simultaneously by any multimedia application. However, multidimensional discrete data is not adequately supported by most existing database systems. In this paper we discuss the major management requirements imposed by such data on multimedia database management systems both at the conceptual level and at the internal level. At the internal level, the spatial structure of the data must also be preserved and special emphasis must be given to the efficient spatial access to subrectangles of very large data of arbitrary dimensionality. Our major contribution is the presentation of a Storage Manager for this kind of data which preserves the spatial nature of the data at the internal level of the database management system and provides efficient spatial access. This Storage Manager is based on an innovative combination of the subdivision of data in multidimensional tiles and of a spatial index to access those tiles. Furthermore, it supports advanced registration data. The lower level modules of the Storage Manager are responsible for the access to the data stored in secondary storage. The implementation of these modules can rely on any secondary storage access system to access data on disk. The Storage Manager was implemented using the functionality of a relational database system. The implementation of this relational Storage Manager is presented.     

The partitioned exponential file for database storage management

The rate of increase in hard disk storage capacity continues to outpace the rate of decrease in hard disk seek time. This trend implies that the value of a seek is increasing exponentially relative to the value of storage. With this trend in mind, we introduce the partitioned exponential file (PE file) which is a generic storage manager that can be customized for many different types of data (e.g., numerical, spatial, or temporal). The PE file is intended for use in environments with intense update loads and concurrent, analytic queries. Such an environment may be found, for example, in long-running scientific applications which can produce petabytes of data. For example, the proposed Large Synoptic Survey Telescope [36] will produce 50–100 petabytes of observational, scientific data over its multi-year lifetime. This database will never be taken off-line, so bursty update loads of tens of terabytes per day must be handled concurrently with data analysis. In the PE file, data are organized as a series of on-disk sorts with a careful, global organization. Because the PE file relies heavily on sequential I/O, only a fraction of a disk seek is required for a typical record insertion or retrieval. In addition to describing the PE file, we also detail a set of benchmarking experiments for T1SM, which is a PE file customized for use with multi-attribute data records ordered on a single numerical attribute. In our benchmarking, we implement and test many competing data organizations that can be used to index and store such data, such as the B+-Tree, the LSM-Tree, the Buffer Tree, the Stepped Merge Method, and the Y-Tree. As expected, no organization is the best over all benchmarks, but our experiments show that T1SM is the best choice in many situations, suggesting that it is the best overall. Specifically, T1SM performs exceptionally well in the case of a heavy query workload that must be handled concurrently with an intense insertion stream. Our experiments show that T1SM (and its close cousin, the T2SM storage manager for spatial data) can handle very heavy mixed workloads of this type, and still maintain acceptably small query latencies.     

采用AM插件管理OLAP数据结构的研究

采用插件接口对象编写的基于微软管理控制台（MMC）的分析管理器（AM）插件可以调用决策支持对象（DSO）来管理OLAP Server中的各种对象，从而大大地提高OLAP数据结构管理的安全性，准确性，灵活性以及快速性，分别介绍了分析管理器插件中使用DSO管理OLAP数据的具体方法。     

基于磁光虚拟存储系统的文件调度算法

基于光盘库的Hadoop分布式文件系统（HDFS光盘库）在单位存储成本、数据安全性、使用寿命等方面非常符合当前大数据存储要求，但是HDFS不适合存储大量小文件和实时数据读取。为了使HDFS光盘库能更好地运用到更多大数据存储场景，本文提出一种更加适合大数据存储的磁光虚拟存储系统（MOVS, Magneto-optical Virtual Storage System）。系统在HDFS光盘库与用户之间加入磁盘缓存，并在磁盘缓存内通过文件标签分类、虚拟存储、小文件合并等技术将磁盘缓存内小文件合并为适合HDFS光盘库存储的大文件，提高系统的数据传输速度。系统还使用了文件预取、缓存替换等文件调度算法对磁盘缓存内文件进行动态更新，减少用户访问HDFS光盘库次数。实验结果表明，MOVS相对HDFS光盘库在响应时间和数据传输速度方面得到很大改善。     

关系型数据库内存化存储模型研究

大数据环境下,磁盘数据库存在高并发I/O瓶颈,磁盘数据内存化是解决传统关系型磁盘数据库I/O瓶颈的有效方案。已有的内存化技术存在数据丢失、配置复杂等问题。基于内存数据库Redis,以典型开源关系型数据库MySQL为例,提出一种解决关系型数据库磁盘I/O瓶颈的轻量级内存化解决方案,实现MySQL的内存化存储。同时构建行式键值（RB-KVM）和分段列式键值交叉（PCB-KVCM）存储转换模型,实现异构数据库存储模型的转换和自动化数据迁移。通过分析对比,RB-KVM数据存取效率更高,而PCB-KVCM具有更高的内存利用率且在时间开销上优于RB-KVM。通过实际运行分析,该技术不仅提高了数据库在高并发场景下的吞吐能力,实现了海量热点数据的高效存取,在实际应用中也更加快捷,实施成本更低。     

一种闪存敏感的多级缓存管理方法

基于闪存的固态硬盘(solid state driver,简称SSD)已经广泛应用于各种移动设备、PC机和服务器.与磁盘相比,尽管SSD具有数据存取速度高、抗震、低功耗等优良特性,但SSD自身也存在读写不对称、价格昂贵等不利因素,这使得SSD 短期内不会完全取代磁盘.将SSD和磁盘组合构建混合系统,可以发挥不同的硬件特性,提升系统性能.基于 MLC 型 SSD 和 SLC 型 SSD 之间的特性差异,提出了一种闪存敏感的多级缓存管理策略——FAMC.FAMC将SSD用在内存和磁盘之间作扩展缓存,针对数据库系统、文件管理中数据访问的特点,有选择地将内存牺牲页缓存到不同类型的SSD.FAMC同时考虑写请求模式和负载类型对系统性能的影响,设计实现对SSD友好的数据管理策略.此外,FAMC基于不同的数据置换代价提出了适用于SSD的缓冲区管理算法.基于多级缓存存储系统对FAMC的性能进行了评测,实验结果表明,FAMC可以大幅度降低系统响应时间,减少磁盘I/O.     

智能公交实时测控系统中集成监管软件的研制

以嵌入式MCU、无线蜂窝通信网络和蓝牙技术为硬件支撑,研制了一种公交车辆综合自动化智能实时测控系统。在系统的监控核心PC机上,运用VB．net高级程序设计和数据库软件ACCESS2003开发了集成监管软件,成功地解决了城市公交车辆的综合自动化监管。详细介绍了集成监管软件的设计与实现。     

机群系统的可视化管理研究与实现:VisualNPC

本文提出一种机群管理系统的实现方案 :Visual NPC.首先 ,采用分布式关系数据库系统来存储管理数据、资源情况、检查点等信息 ,使得数据检索与访问的速度优于采用文件系统存储时的速度 ;其次 ,采用 WEB用户界面来操控机群系统 ,使得在与机群系统联通的网络中的任意一台计算机上 ,都可使用 WEB浏览器控制与访问机群系统 ;最后 ,采用独立的管理服务器 ,使得这些管理操作对机群系统本身的运算影响最小 ,并且对这个独立的管理服务器作镜像容错处理 ,在成本和效率上要优于为每个计算节点作镜像容错处理