人工智能赋能的数据管理技术研究

大数据时代,数据规模庞大、数据管理应用场景复杂,传统数据库和数据管理技术面临很大的挑战.人工智能技术因其强大的学习、推理、规划能力,为数据库系统提供了新的发展机遇.人工智能赋能的数据库系统通过对数据分布、查询负载、性能表现等特征进行建模和学习,自动地进行查询负载预测、数据库配置参数调优、数据分区、索引维护、查询优化、查询调度等,以不断提高数据库针对特定硬件、数据和负载的性能.同时,一些机器学习模型可以替代数据库系统中的部分组件,有效减少开销,如学习型索引结构等.分析了人工智能赋能的数据管理新技术的研究进展,总结了现有方法的问题和解决思路,并对未来研究方向进行了展望.     

数据库索引与优化

大型的企业或部门每天都需要处理大量的数据业务,数据库系统的设计变得异常复杂,而数据库的性能直接影响到各项业务的顺利进行。然而数据库优化涉及计算机硬件调优、应用程序调优、数据库索引优化、SQL语句优化和事务处理调优法等多个方面．对数据库系统进行优化。通过对数据库的索引技术进行研究,论述如何设置有效的数据库索引达到数据库优化．以期给数据库设计者和系统开发者提供参考。     

面向关系数据库的智能索引调优方法

数据库索引是关系数据库系统实现快速查询的有效方式之一.智能索引调优技术可以有效地对数据库实例进行索引调节,从而保持数据库高效的查询性能.现有的方法大多利用了数据库实例的查询日志,它们先从查询日志中得到候选索引,再利用人工设计的模型选择索引,从而调节索引.然而,从查询日志中产生出的候选索引可能并未实际存在于数据库实例中,因此导致这些方法不能有效地估计这类索引对于查询的优化效果.首先,设计并实现了一种面向关系数据库的智能索引调优系统;其次,提出了一种利用机器学习方法来构造索引的量化模型,根据该模型,可以准确地对索引的查询优化效果进行估计;接着设计了一种高效的最优索引选择算法,实现快速地从候选索引空间中选择满足给定大小约束的最优的索引组合;最后,通过实验测试不同场景下智能索引调优系统的调优性能.实验结果表明,所提出的技术可以在不同的场景下有效地对索引进行优化,从而实现数据库系统查询性能的提升.     

面向AI的数据管理技术综述

人工智能技术因其强大的学习和泛化能力已经被广泛应用到各种真实场景中.然而,现有人工智能技术还面临着三大挑战.第一,现有AI技术使用门槛高,依赖于AI从业者选择合适模型、设计合理参数、编写程序,因此很难被广泛应用到非计算机领域;第二,现有AI算法训练效率低,造成了大量计算资源浪费,甚至延误决策时机;第三、现有AI技术强依赖高质量数据,如果数据质量较低,可能造成计算结果的错误.数据库技术可以有效解决这三个难题,因此目前面向AI的数据管理得到了广泛关注.本文首先给出AI中数据管理的整体框架,然后详细综述基于声明式语言模型的AI系统、面向AI优化的计算引擎、执行引擎和面向AI的数据治理引擎四个方面.最后展望未来的研究方向和挑战.     

面向云数据中心的人工智能模型自动优化框架设计

人工智能模型的自动调优技术能够以较低资源成本提供云数据中心的高性能智能服务。然而,人工智能模型和硬件设备具有异构性,云数据中心执行自动调优操作会产生大量计算时间,占用算力资源,产生能耗成本。针对此问题,本文设计面向云计算数据中心的人工智能模型自动优化框架。提出人工智能模型候选配置项过滤方法,利用模型构建、特征提取、候选项探索、配置查询等技术对候选项搜索空间重新采样,将高效候选项替换低效候选项。在算子优化层面,框架分批并行执行计算组件实现的硬件测量,避免连续探测搜索空间。在模型优化层面,根据多人工智能模型的相对性能加速优先跨集群的计算组件优化。该框架旨在面向不同人工智能模型,降低人工智能模型推理延迟,减少云计算数据中心能耗,从而提升人工智能模型自动调优的成本效益。     

云原生数据库综述

云基础设施的虚拟化、高可用、可弹性调度等特点,为云数据库提供了开箱即用、可靠可用、按需计费等优势.云数据库按照架构可以划分为云托管数据库(cloud-hosted database)以及云原生数据库(cloud-native database).云托管数据库将数据库系统直接部署到云上虚拟机环境中,具备低成本、易运维、高可靠的优势.在此基础上,云原生数据库充分利用云基础设施弹性伸缩的特点,采用计算存储分离的架构,实现了计算资源和存储资源的独立伸缩,进一步提升数据库性价比.然而计算存储分离的架构为数据库系统设计带来了新的挑战.深入分析云原生数据库系统的架构和技术.首先将云原生OLTP和云原生OLAP的数据库架构按照资源分离模式的差异分别进行归类分析,并对比各类架构的优势与局限.其次,基于计算存储分离的架构,按照各个功能模块深入探讨云原生数据库的关键技术:主要包括云原生OLTP关键技术(数据组织、副本一致性、主备同步、故障恢复以及混合负载处理)和云原生OLAP关键技术(存储管理、查询处理、无服务器感知计算、数据保护以及机器学习优化).最后,总结现有云原生数据库的技术挑战并展望未来研究方向.     

AutoConfig: 面向深度学习编译优化的自动配置机制

随着深度学习模型和硬件架构的快速发展,深度学习编译器已经被广泛应用.目前,深度学习模型的编译优化和调优的方法主要依赖基于高性能算子库的手动调优和基于搜索的自动调优策略.然而,面对多变的目标算子和多种硬件平台的适配需求,高性能算子库往往需要为各种架构进行多次重复实现.此外,现有的自动调优方案也面临着搜索开销大和缺乏可解释性的挑战.为了解决上述问题,本文提出了AutoConfig,一种面向深度学习编译优化的自动配置机制.针对不同的深度学习计算负载和特定的硬件平台,AutoConfig可以构建具备可解释性的优化算法分析模型,采用静态信息提取和动态开销测量的方法进行综合分析,并基于分析结果利用可配置的代码生成技术自动完成算法选择和调优.本文创新性地将优化分析模型与可配置的代码生成策略相结合,不仅保证了性能加速效果,还减少了重复开发的开销,同时简化了调优过程.在此基础上,本文进一步将AutoConfig集成到深度学习编译器Buddy Compiler中,对矩阵乘法和卷积的多种优化算法建立分析模型,并将自动配置的代码生成策略应用在多种SIMD硬件平台上进行评估.实验结果验证了AutoConfig在代码生成策略中有效地完成了参数配置和算法选择.与经过手动或自动优化的代码相比,由AutoConfig生成的代码可达到相似的执行性能,并且无需承担手动调优的重复实现开销和自动调优的搜索开销.     

基于Kolla的人工智能与大数据教学实验平台设计

随着人工智能和大数据的蓬勃发展,目前各高职院校都陆续开设了人工智能技术服务和大数据技术等专业,但AI和大数据技术对实验环境有着较高的要求,如何更好的利用学校的硬件设备,提高硬件的使用效率成为一个新的课题。本文提出了一种基于Kolla OpenStack的人工智能与大数据教学实验平台,通过开源云计算技术、GPU透传技术对硬件资源的共享利用,让用户可以自由分配资源,自主搭建AI或BigData实验环境。     

人工智能赋能的数据管理、分析与系统专刊前言

大数据时代,数据规模庞大,数据管理应用场景复杂,传统数据库和数据管理技术面临很大的挑战.人工智能技术因其强大的学习、推理、规划能力,为数据库系统提供了新的发展机遇.专刊强调数据管理与人工智能的深度融合,研究人工智能赋能的数据库新技术和新型系统,包括两方面:(1)传统数据管理、数据分析技术及系统与人工智能相结合,将会焕发新的生机;(2)大数据管理与分析是新一代人工智能技术发展的基石.因此,围绕传统数据管理的不同技术层面,需要新的理论和系统经验.     

面向多核处理器平台的并行Hash JOIN算法设计与实现

利用多核处理器提供的强大计算能力提升数据库系统性能是当前国内外数据库研究的重要问题.利用基于多核处理器上的并行编程模型MSI和Intel处理器上的SIMD(单指令流多数据流)指令有效地加速了数据库查询的Join操作,与串行实现相比其最大加速可以达13倍.同时,还对比不同数据分块大小情况下对算法的影响,找到了优化的数据分块方法.     

Preface

Database and Artificial Intelligence (AI) can benefit from each other. On the one hand, AI can make database more intelligent (AI4DB) by exploiting learning-based techniques. On the other hand, database techniques can optimize AI models (DB4AI), such as reducing the complexity of using AI models and accelerating the deployment of AI algorithms. In this special section, we discuss 1) how to exploit AI or machine learning techniques for index design, performance tuning, query processing in database systems, and 2) how to utilize database and data management techniques to make AI models more reusable and more tolerant to dirty data.     

DiSC: Benchmarking Secure Chip DBMS

Secure chips, e.g. present in smart cards, USB dongles, i-buttons, are now ubiquitous in applications with strong security requirements. And they require embedded data management techniques. However, secure chips have severe hardware constraints which make traditional database techniques irrelevant. The main problem faced by secure chip DBMS designers is to be able to assess various design choices and trade-offs for different applications. Our solution is to use a benchmark for secure chip DBMS in order to (1) compare different database techniques, (2) predict the limits of on-chip applications, and (3) provide co-design hints. In this paper, we propose DiSC (Data management in Secure Chip), a benchmark which matches these three objectives. This work benefits from our long experience in developing and tuning data management techniques for the smart card. To validate DiSC, we compare the behavior of candidate data management techniques thanks to a cycle-accurate smart card simulator. Finally, we show the applicability of DiSC to future designs involving new hardware platforms and new database techniques.     

A systematic literature review on hardware implementation of artificial intelligence algorithms

Artificial intelligence (AI) and machine learning (ML) tools play a significant role in the recent evolution of smart systems. AI solutions are pushing towards a significant shift in many fields such as healthcare, autonomous airplanes and vehicles, security, marketing customer profiling and other diverse areas. One of the main challenges hindering the AI potential is the demand for high-performance computation resources. Recently, hardware accelerators are developed in order to provide the needed computational power for the AI and ML tools. In the literature, hardware accelerators are built using FPGAs, GPUs and ASICs to accelerate computationally intensive tasks. These accelerators provide high-performance hardware while preserving the required accuracy. In this work, we present a systematic literature review that focuses on exploring the available hardware accelerators for the AI and ML tools. More than 169 different research papers published between the years 2009 and 2019 are studied and analysed.

     

High-Performance Telemedicine Information Management

The rapid advances in high performance global communication have accelerated cooperative image-based medical services to a new frontier. Traditional image-based medical services such as radiology and diagnostic consultation can now fully utilize multimedia technologies to provide novel services, including remote cooperative medical triage, distributed virtual simulation of operations, as well as cross-country collaborative medical research and training. Fast (efficient) and easy (flexible) retrieval of relevant images remains a critical requirement for the provision of remote medical services. This paper describes the database system requirements and presents a system architecture of a distributed multimedia database system, MISSION-DBS, which has been designed to fulfill the goals of Project MISSION (medical imaging support via satellite integrated optical network)—an experimental high performance gigabit satellite communication network linking remote supercomputing power, medical image databases, and 3D visualization capabilities, in addition to medical expertise anywhere and anytime around the globe. The MISSION-DBS design employs a synergistic fusion of techniques in distributed databases (DDB) and artificial intelligence (AI) for storing, migrating, accessing, and exploring images. The efficient storage and retrieval of voluminous image information is achieved by integrating DDB modeling and AI techniques for image processing while the flexible retrieval mechanisms are accomplished by combining attribute-based and content-based retrievals.     

RUL prediction for automatic machines: a mixed edge-cloud solution based on model-of-signals and particle filtering techniques

This work aims to provide useful insights into the course of action and the challenges faced by machine manufacturers when dealing with the actual application of Prognostics and Health Management procedures in industrial environments. Taking into account the computing capabilities and connectivity of the hardware available for smart manufacturing, we propose a particular solution that allows meeting one of the essential requirements of intelligent production processes, i.e., autonomous health management. Indeed, efficient and fast algorithms, that does not require a high computational cost and can be appropriately performed on machine controllers, i.e., on edge, are combined with others, which can handle large amounts of data and calculations, executed on remote powerful supervisory platforms, i.e., on the cloud. In detail, new condition monitoring algorithms based on Model-of-Signals techniques are developed and implemented on local controllers to process the raw sensor readings and extract meaningful and compact features, according to System Identification rules and guidelines. These results are then transmitted to remote supervisors, where Particle Filters are exploited to model components degradation and predict their Remaining Useful Life. Practitioners can use this information to optimise production planning and maintenance policies. The proposed architecture allows keeping the communication traffic between edge and cloud in the nowadays affordable “Big data” range, preventing the unmanageable “Huge data” scenario that would follow from the transmission of raw sensor data. Furthermore, the robustness and effectiveness of the proposed method are tested considering a meaningful benchmark, the PRONOSTIA dataset, allowing reproducibility and comparison with other approaches.

     

Formal approach for reengineering fuzzy XML in fuzzy object-oriented databases

Since semi-structured documents (e.g., XML) could benefit greatly from database support and more specifically from object-oriented (OO) database management systems, we study the methodology of reengineering XML to object-oriented databases when database migration occurs in this paper. In particular, considering the need of processing the imprecise and uncertain information existing in practical applications, we investigate the problem of migrating fuzzy XML to fuzzy object-oriented databases. To find the object-oriented schema that best describes the existing fuzzy XML schema (DTD), we devise a comprehensive approach centering on a set of mapping rules. Such reengineering practices could not only provide a significant consolidation of the interoperability between fuzzy OO and fuzzy XML modeling techniques, but also develop the practical design methodology for fuzzy OO databases.     

Utility and mechanism design in multi-agent systems: An overview

Future cities promise to be more autonomous than ever, largely owing to our ability of coordinating complex systems in real time: fleets of self-driving cars will offer on-demand transportation services, delivery drones will fly parcels in our skies, power plants will provide renewable energy reliably. In many of these systems, there is no single decision-maker with full information and authority. Instead, the system performance greatly depends on the decisions made by interacting entities with local information and limited communication capabilities. Game theory, intended as the study of multi-agent decision-making, is a fitting paradigm to tackle many of the associated challenges. Moving from this observation, in this paper we review how tools and ideas from game theory can be brought to bear on the coordination of multi-agent systems. At the heart of the proposed approach is the design and influence of agents’ preferences so that their local optimization induces a desirable system behavior. Its applicability spans a variety of settings irrespective of whether the decision makers are strategic (e.g., drivers in a road network), or not (e.g., delivery drones). Along the way, we also discuss future research directions and connections with related research areas including algorithmic game theory, incentive and mechanism design, economics, computational complexity, and approximation algorithms.     

A survey of deep learning techniques for autonomous driving

The last decade witnessed increasingly rapid progress in self‐driving vehicle technology, mainly backed up by advances in the area of deep learning and artificial intelligence (AI). The objective of this paper is to survey the current state‐of‐the‐art on deep learning technologies used in autonomous driving. We start by presenting AI‐based self‐driving architectures, convolutional and recurrent neural networks, as well as the deep reinforcement learning paradigm. These methodologies form a base for the surveyed driving scene perception, path planning, behavior arbitration, and motion control algorithms. We investigate both the modular perception‐planning‐action pipeline, where each module is built using deep learning methods, as well as End2End systems, which directly map sensory information to steering commands. Additionally, we tackle current challenges encountered in designing AI architectures for autonomous driving, such as their safety, training data sources, and computational hardware. The comparison presented in this survey helps gain insight into the strengths and limitations of deep learning and AI approaches for autonomous driving and assist with design choices.     

Creating probabilistic databases from duplicated data

A major source of uncertainty in databases is the presence of duplicate items, i.e., records that refer to the same real-world entity. However, accurate deduplication is a difficult task and imperfect data cleaning may result in loss of valuable information. A reasonable alternative approach is to keep duplicates when the correct cleaning strategy is not certain, and utilize an efficient probabilistic query-answering technique to return query results along with probabilities of each answer being correct. In this paper, we present a flexible modular framework for scalably creating a probabilistic database out of a dirty relation of duplicated data and overview the challenges raised in utilizing this framework for large relations of string data. We study the problem of associating probabilities with duplicates that are detected using state-of-the-art scalable approximate join methods. We argue that standard thresholding techniques are not sufficiently robust for this task, and propose new clustering algorithms suitable for inferring duplicates and their associated probabilities. We show that the inferred probabilities accurately reflect the error in duplicate records.