人工智能赋能的数据管理技术研究

大数据时代,数据规模庞大、数据管理应用场景复杂,传统数据库和数据管理技术面临很大的挑战.人工智能技术因其强大的学习、推理、规划能力,为数据库系统提供了新的发展机遇.人工智能赋能的数据库系统通过对数据分布、查询负载、性能表现等特征进行建模和学习,自动地进行查询负载预测、数据库配置参数调优、数据分区、索引维护、查询优化、查询调度等,以不断提高数据库针对特定硬件、数据和负载的性能.同时,一些机器学习模型可以替代数据库系统中的部分组件,有效减少开销,如学习型索引结构等.分析了人工智能赋能的数据管理新技术的研究进展,总结了现有方法的问题和解决思路,并对未来研究方向进行了展望.     

学习索引:现状与研究展望

索引是数据库系统中用于提升数据存取性能的主要技术之一.在大数据时代，随着数据量的不断增长，传统索引（如B+树）的问题日益突出：（1）空间代价过高.例如，B+树索引需要借助O（n）规模的额外空间来索引原始的数据，这对于大数据环境而言是难以容忍的.（2）每次查询需要多次的间接搜索.例如，B+树中的每次查询都需要访问从树根到叶节点路径上的所有节点，这使得B+树的查找性能受限于数据规模.自2018年来，人工智能与数据库领域的结合催生了“学习索引”这一新的研究方向.学习索引利用机器学习技术学习数据分布和查询负载特征，并用基于数据分布拟合函数的直接式查找代替传统的间接式索引查找，从而降低索引的空间代价并提升查询性能.首先对学习索引技术的现有工作进行了系统梳理和分类;然后，介绍了各种学习索引技术的研究动机与关键技术，对比分析了各种索引结构的优劣;最后，对学习索引的未来研究方向进行了展望.     

面向数据库应用的隐私保护问题概述

伴随着计算机网络技术与无线通信技术的蓬勃发展,包括数据挖掘以及数据发布在内的多种数据库关联性应用功能的构建及发展成为了相关工作人员作为关注的问题之一。在当前技术条件支持之下,如何以数据库应用为中心,针对数据库系统所涉及到的各类隐私数据进行系统保护并防止数据库系统中的敏感信息发生外泄,是相关工作人员应当研究的重点。我们清楚的认识到:现阶段能够充分发挥数据库隐私保护职能的技术方针可以分为以下几大部分—1.数据失真技术;2.数据加密技术;3.限制保护技术。以上三种面向数据库应用隐私保护技术方针最大的优势在于其在确保隐私保护职能发挥的同时兼顾了整个数据库系统的正常、稳定运行,这一点需要我们加以认同与肯定。本文依据这一实际情况,以新时期面向数据库应用的隐私保护问题为研究对象,从隐私保护研究方向及研究现状分析、隐私保护技术分类分析以及面向数据库应用的隐私保护技术分析这三个方面入手,对其进行了较为详细的分析与阐述,并据此论证了做好数据库隐私保护工作在进一步提升数据库应用质量与应用效率的过程中所起到的至关重要的作用与意义。     

DBMS负载自适应体系中系统监测部件的研究

数据库系统负载自适应体系是一个性能管理过程,在这一过程中自主计算数据库系统能够自动地控制和过滤负载、并有效地使用数据库系统资源来满足网络应用的性能要求。在数据库系统负载自适应体系框架中,系统监测部件负责监控系统资源的使用情况,并将监控到的信息反馈给相关部件。提出了系统监测部件应该具备的功能,给出了实现的途径,并研究了监测部件的性能要求,分析了如何降低监测部件本身对数据库的影响。最后基于Oracle数据库进行了验证。     

数据模型智能化扩充研究——知识数据库模型

数据模型是数据库系统的核心。数据模型智能化扩充旨在提高现有数据库系统的可扩充性及智能化程度,使传统的数据库存贮知识,能进行演绎推理,从而为高级咨询与决策提供支持。本文引入人工智能、知识工程及专家系统领域的概念,原理,方法和技术,对关系数据模型进行了智能化扩充,并提出了实现这一扩充的DBMS的体系结构及实现方法。我们的工作将为了一步关于智能数据库、知识库的研究与开发打下基础,并希望在MIS,DSS,EDS,CAD,CAM等方向上得到应用。     

基于替换策略的自适应数据库负载控制*

自适应数据库负载控制是一个性能优化过程, 用来提高数据库系统的整体性能。分析了自适应数据库负载控制的研究现状,提出了基于替换策略的负载控制方法,该方法通过竞争决策算法和替换策略对负载进行优化过滤来提高数据库系统的性能。仿真实验表明,该方法可以有效提高客户的满意率和系统实现价值,并且可以增强系统的稳定性。     

面向数据库查询加速的异构体系结构设计与实现

数据库是数据分析、人工智能、云计算和大数据等领域的关键工作负载,是提高系统整体性能的关键。传统数据库系统查询执行效率偏低,并且CPU通常需要优先处理事务性负载,使数据查询逐渐成为制约整个数据库系统性能和效率提升的瓶颈。为了提高数据库在大规模并发访问时的数据处理能力,提出了一种CPU+FPGA加速数据库查询的异构体系结构,通过使用异构缓存一致性加速接口将加速器集成到CPU中,在FPGA内定制可配置的多引擎查询方式,对数据库查询进行加速。聚焦常用的SQL查询语句SELECT,详细分析了系统在延迟和简化的软件堆栈方面的优势,最后使用浪潮F37X加速卡和浪潮服务器对加速模型功能和性能进行了验证。实验结果表明,与使用POWER 9 CPU进行同样的查询操作相比,数据库系统整体处理速度提高了3～9倍,这种先进的异构计算加速结构可应用在未来数据库硬件专业化的设计中。     

主动数据库系统及其事件监测器的实现

传统数据库系统已经不能完全适应现代应用.近些年,主动数据库的研究发展比较迅速,主动数据库已经成为数据库研究的一个前沿方向.文章介绍了主动数据库的系统模型、需解决的主要问题,以及事件监测器的实现机制,并且讨论了一种基于ECA模型的通用主动数据库中数据库事件监测器的设计与实现.     

数据库系统应用基准测试规范设计方法

为准确度量数据库系统对领域应用系统的支撑能力,提出了一种应用基准测试规范设计方法,指导数据库系统的基准测试。提出以动作、环境、负载等信息项为框架,建立数据库系统使用需求集,规范初始信息格式;将使用需求初分为操作场景,建立关键变量矩阵,并应用K-means算法、层次式聚类等方法进行操作场景分组,确认典型应用系统集;设计了基准测试用例结构,提出了环境要求、事务要求、执行要求等重要信息项,并构建测试规范框架,描述了典型应用系统与应用基准测试规范的关系;通过实例应用,证明了该方法的可行性与适用性。所提方法可辅助解决应用系统在多个数据库系统中择优选型问题,也可用于数据库系统厂商自测,促进产品能力提升。     

数据库备份恢复技术的研究及应用设计

数据库系统的备份和恢复是当前数据库技术发展的一个重要课题,在实际应用中有着很重要的地位。本论文介绍了作者参与的某银行综合业务系统的备份和恢复部分,这是一个利用ORACLE数据库系统和EMC的TimeFinder实现的数据库备份和恢复的系统。本文从作者实际所从事的工作出发,通过ORACLE数据库系统对数据库系统的备份方法进行研究,探讨和分析了有关数据库系统的备份和恢复的理论、技术和方法。     

Data adapter for querying and transformation between SQL and NoSQL database

As the growing of applications with big data in cloud computing become popular, many existing systems expect to expand their service to support the explosive increase of data. We propose a data adapter system to support hybrid database architecture including a relational database (RDB) and NoSQL database. It can support query from application and deal with database transformation at the same time. We provide three modes of query approach in data adapter system: blocking transformation mode (BT mode), blocking dump mode (BD mode), and direct access mode (DA mode). We provide a data synchronization mechanism and describe the design and implementation in detail. This paper focuses on velocity with proposed three modes and partly variety with data stored in RDB, NoSQL database and temporary files. With the proposed data adapter system, we can provide a seamless mechanism to use RDB and NoSQL database at the same time.     

分布式数据库中一致性与可用性的关系

随着各类应用在数据量和业务量上的扩展,单机数据库系统越发难以应对现实需求。分布式数据库能够根据业务的需求动态地扩容,因此逐步开始受到应用的青睐。近年来,分布式数据库产品层出不穷,并在互联网应用中被大量投入使用。然而,分布式数据库的系统复杂度前所未有。为了让系统可用,设计者需要在多种属性中作合理选择和折中。这造成现有的数据库产品形态各异、优缺点对比分明。至今为止,尚未有人对分布式数据库的设计空间和折中方案进行过深入分析和整理。本文作者在对多个分布式数据库产品进行深入理解之后认识到：分布式数据库系统的设计方案可以通过三个属性进行基本刻画–操作一致性、事务一致性和系统可用性。虽然这三个属性并不新颖,但它们在数据库语境下的含义在文献中尚未得到充分澄清。本文对这三个属性进行澄清,并通过它们对典型数据库产品的格局进行概括、对现有的分布式数据库技术进行综述。此外,本文还对这三个属性之间的相互关系进行深入分析,以期帮助未来的开发者在分布式数据库的设计过程中作出合理选择。     

Distance semantics for database repair

In many scenarios, a database instance violates a given set of integrity constraints. In such cases, it is often required to repair the database, that is, to restore its consistency. A primary motif behind the repairing approaches is the principle of minimal change, which is the aspiration to keep the recovered data as faithful as possible to the original (inconsistent) database. In this paper, we represent this qualitative principle quantitatively, in terms of distance functions and some underlying metrics, and so introduce a general framework for repairing inconsistent databases by distance-based considerations. The uniform way of representing repairs and their semantics clarifies the essence behind several approaches to consistency restoration in database systems, helps to compare the underlying formalisms, and relates them to existing methods of defining belief revision operators, merging data sets, and integrating information systems.      

Algorithms for asynchronous parallel processing of object-orienteddatabases

Management of large quantities of complex data is essential in many advanced application areas. Object-oriented (OO) database management system have been developed to effectively model and process the complex domain knowledge. They have been shown to outperform some existing relational systems. The existing implementations of OO database management systems attempt to improve the efficiency of OO queries by explicitly capturing the relationships among objects. However, the execution of complex queries involving the retrieval of objects from many classes and relationships among them causes the existing system to operate inefficiently. In this paper, we present parallel algorithms for the processing of queries against a large OO database. The algorithms are based on a closed model of query processing pattern-based access instead of the conventional value-based access. During processing, the algorithms avoid the execution of time-consuming join operations by making use of the explicitly stored object associations. Generation of large quantities of temporary data is avoided by marking objects using their identifiers and by employing a two-phase query processing strategy. A query is processed by concurrent multiple waves, thereby improving parallelism avoiding the complexities introduced in their sequential implementation. The correctness and the performance of the parallel algorithms have been tested and analyzed by running parallel programs on a 32-node transputer based parallel machine designed and developed at the IBM Research Center at Yorktown Heights, New York. Benchmark queries of different semantic complexities are generated, and their performance is analyzed for various data and query parameters     

WWW与数据库集成系统的用户权限管理

在WWW上的应用,越来越多的体现为多人合作应用形式,这些应用系统大多都需要通过WWW与数据库的集成来实现。但是由于WWW与数据库系统都有自己的用户权限管理,因此集成后在用户管理、权限设定及系统完整性方面就产生了一些问题。文章针对WWW与数据库集成系统,提出用户权限管理结构和机制,并设计了实际系统模型,它将服务器对用户的认证,转变成基于用户角色的对数据库的访问控制,不但能保护WWW与数据库的数据,而且能确保系统结合的完整性,同时更可以大量简化用户管理与权限管制的工作,提高权限管理的效率。     

Advanced data processing in KRISYS: modeling concepts, implementation techniques, and client/server issues

The increasing power of modern computers is steadily opening up new application domains for advanced data processing such as engineering and knowledge-based applications. To meet their requirements, concepts for advanced data management have been investigated during the last decade, especially in the field of object orientation. Over the last couple of years, the database group at the University of Kaiserslautern has been developing such an advanced database system, the KRISYS prototype. In this article, we report on the results and experiences obtained in the course of this project. The primary objective for the first version of KRISYS was to provide semantic features, such as an expressive data model, a set-oriented query language, deductive as well as active capabilities. The first KRISYS prototype became completely operational in 1989. To evaluate its features and to stabilize its functionality, we started to develop several applications with the system. These experiences marked the starting point for an overall redesign of KRISYS. Major goals were to tune KRISYS and its query-processing facilities to a suitable client/server environment, as well as to provide elaborate mechanisms for consistency control comprising semantic integrity constraints, multi-user synchronization, and failure recovery. The essential aspects of the resulting client/server architecture are embodied by the client-side data management needed to effectively support advanced applications and to gain the required system performance for interactive work. The project stages of KRISYS properly reflect the essential developments that have taken place in the research on advanced database systems over the last years. Hence, the subsequent discussions will bring up a number of important aspects with regard to advanced data processing that are of significant general importance, as well as of general applicability to database systems. Received June 18, 1996 / Accepted November 11, 1997     

Federated databases and systems: Part II — A tutorial on their resource consolidation

The issues and solutions for the interoperability of a class of heterogeneous databases and their database, systems are expounded in two parts. Part I presented the data-sharing issues in federated databases and systems (Hsiao, 1992). The present article explores resource-consolidation issues.Interoperability in this context refers to data sharing among heterogeneous databases, and to resource consolidation of computer hardware, system software, and support personnel.Resource consolidation requires the presence of a database system architecture which supports the heterogeneous system software, thereby eliminating the need for various computer hardware and support personnel. The class of heterogeneous databases and database systems expounded herein is termedfederated, meaning that they are joined in order to meet certain organizational requirements and because they require their respective application specificities, integrity constraints, and security requirements to be upheld. Federated databases and systems are new. While there are no technological solutions, there has been considerable research towards their development. This tutorial is aimed at exposing the need for such solutions. A taxonomy is introduced in our, review of existing research undertakings and exploratory developments. With this taxonomy, we contrast and compare various approaches to federating databases and systems.Editorial note: Portions of the introductory sections of Part II of this artlcle replicate material in Part I (Hsiao, 1992) in order to provide the reader with a self-standing article.     

一个开放可扩展的SDAI实现系统

STEP标准数据存取界面－SDAI为应用程序提供了独立于数据存储的STEP数据访问界面，SDAI允许访问不同的数据存储系统，有不同的实现语言联编方式，如果对每一种实用语言和数据存储系统单独实现SDAI，其工作量将是巨大的，该文选择系数据库作为SDAI数据存储系统，C语言作为实现语言，实现系统独立于不同的关系数据库系统，并且为不同语言的联编提供了一个统一的开发平台，提高了系统的开放性和可扩展性，为STEP在企业信息集成中的应用提供了核心操作，STEP建模语言EXPRESS，关系数据库，SDAI实现语言在模式表示上的不匹配是系统实现要解决的主要问题，该文从数据字典，STEP数据的字储与访问等方面阐明了系统实现时没模式之间的匹配过程。     

On the errors that learning machines will make

Associated with each learning system there is a class of learnable behaviors. If the target behavior to be acquired is in the learnable class, it will be learned perfectly. If it is outside that class, the machine will only be able to acquire a behavior that approximates the target and it will always make errors. It is desirable for a learning machine to have a large learnable class to maximize the chances of acquiring the unknown behavior and to minimize the expected error when only an approximation is possible. However, it is also desirable to have a small learnable class so that learning can be achieved rapidly. Thus the design of learning machines involves selecting a position on the spectrum: minimum error and slow learning time versus larger error and faster learning time. A computational method is given for finding where a given learning machine is on this spectrum. Machines that have fast learning times, relatively small learnable classes, and thus relatively large expected errors are called realization sparse in this article. These machines do little better than a random coin flipping algorithm in many situations. It is shown that many common learning systems are of this type including signature tables, linear system models, and conjunctive normal form expression based systems. These studies lead to the concept of an “optimum” machine which spreads its learnable behaviors across the behavior space in a manner to minimize the expected error. an approximation to such optimum machines is presented and its behavior is compared to the more traditional learning machines. © 1994 John Wiley & Sons, Inc.     

Integration of engineering change objects in product data management databases to support engineering change analysis

This study proposes a product data management (PDM) database that can support engineering change analysis (ECA). It can integrate ECA with the existing main product development process managed by PDM systems. Since engineering change (EC) history is a key EC data element that enables ECA, this study extends PDM databases to represent the EC history with existing entities for ECs and associated products and their product structures. To show the feasibility of the proposed PDM database, this study integrates a prototype PDM system with on-line analytical processing (OLAP) tools and a data mining module for ECA. It also applies the implemented tools to two typical ECA applications, EC evaluation and EC propagation problems. The illustrative application examples show that the proposed PDM database can support ECA through multidimensional data analysis with OLAP and data mining with association rules.