基于特征和图结构信息增强的多教师学习图神经网络

近年来,图神经网络对图数据强大的表征能力和建模能力使其在诸多领域广泛应用并取得了重大突破。然而,现有模型往往倾向于对图卷积聚合策略和网络结构进行优化,缺乏了对图数据自身先验知识的探索。针对上述问题,通过知识蒸馏的方法,设计了一种基于特征信息和结构信息增强的多教师学习图神经网络,打破了现有模型对于数据先验知识提取的局限性。针对图数据背后所蕴涵的丰富特征与结构信息,分别设计了节点特征和边的数据增强方式。在此基础上,将原始数据和增强后的数据通过多教师学习模块进行知识嵌入,使得学生模型学习到更多关于数据的先验知识。在Cora、Citeseer和PubMed数据集上,节点分类准确率分别提升了1%、1.3%、1.1%。实验结果表明,提出的信息增强的多教师学习模型能够有效地捕获先验知识。     

基于卷积神经网络的图像数据增强算法

提升卷积神经网络的泛化能力和降低过拟合的风险是深度卷积神经网络的研究重点。遮挡是影响卷积神经网络泛化能力的关键因素之一,通常希望经过复杂训练得到的模型能够对遮挡图像有良好的泛化性。为了降低模型过拟合的风险和提升模型对随机遮挡图像识别的鲁棒性,提出了激活区域处理算法,在训练过程中对某一卷积层的最大激活特征图进行处理后对输入图像进行遮挡,然后将被遮挡的新图像作为网络的新输入并继续训练模型。实验结果表明,提出的算法能够提高多种卷积神经网络模型在不同数据集上的分类性能,并且训练好的模型对随机遮挡图像的识别具有非常好的鲁棒性。     

基于知识图谱驱动的图神经网络推荐模型

知识图谱(KG)蕴含丰富的结构与关联信息,不仅可以缓解推荐系统中数据稀疏、冷启动等问题,还可以更准确地进行个性化推荐,因此提出一种基于知识图谱驱动的端到端图神经网络推荐模型KGLN.首先使用单层神经网络框架对图中单个节点进行特征融合,并加入影响因子来改变不同邻居实体的聚合权重;然后通过迭代的方式将单层扩展到多层,使实体...     

基于同义词数据增强的汉越神经机器翻译方法

汉越平行语料库的资源稀缺,很大程度上影响了汉越机器翻译效果.数据增强是提升汉越机器翻译的有效途径,基于双语词典的词汇替换数据增强是当前较为流行的方法.由于汉语-越南语属于低资源语言对,双语词典难以获得,而通过单语词向量获取低频词的同义词较为容易.因此,提出一种基于低频词的同义词替换的数据增强方法.该方法利用小规模的平行...     

基于图神经网络多模态融合的语音情感识别模型

目前,基于多模态融合的语音情感识别模型普遍存在无法充分利用多模态特征之间的共性和互补性、无法借助样本特征间的拓扑结构特性对样本特征进行有效地优化和聚合,以及模型复杂度过高的问题。为此,引入图神经网络,一方面在特征优化阶段,将经过图神经网络优化后的文本特征作为共享表示重构基于声学特征的邻接矩阵,使得在声学特征的拓扑结构特性中包含文本信息,达到多模态特征的融合效果;另一方面在标签预测阶段,借助图神经网络充分聚合当前节点的邻接节点所包含的相似性信息对当前节点特征进行全局优化,以提升情感识别准确率。同时为防止图神经网络训练过程中可能出现的过平滑问题,在图神经网络训练前先进行图增强处理。在公开数据集IEMOCAP 和RAVDESS上的实验结果表明,所提出的模型取得了比基线模型更高的识别准确率和更低的模型复杂度,并且模型各个组成部分均对模型性能提升有所贡献。     

基于图神经网络的代码漏洞检测方法

使用神经网络进行漏洞检测的方案大多基于传统自然语言处理的思路,将源代码当作序列样本处理,忽视了代码中所具有的结构性特征,从而遗漏了可能存在的漏洞.提出了一种基于图神经网络的代码漏洞检测方法,通过中间语言的控制流图特征,实现了函数级别的智能化代码漏洞检测.首先,将源代码编译为中间表示,进而提取其包含结构信息的控制流图,同...     

Intelligent design of shear wall layout based on graph neural networks

Structural scheme design of shear wall structures is important because it is the first stage that guides the project along its entire structural design process and significantly impacts the subsequent design stages. Design methods for shear wall layouts based on deep generative algorithms have been proposed and achieved some success. However, current generative algorithms rely on pixel images to design shear wall layouts, which have many model parameters and require intensive calculations. Moreover, it is challenging to use pixel image-based methods to reflect the topological characteristics of structures and connect them with the subsequent design stages. The above defects can be effectively solved by representing a shear wall structure in graph data form and adopting graph neural networks (GNNs), which have a robust topological-characteristic-extraction capability. However, there is no existing research using GNN methods in the design of shear wall structures owing to the lack of graph representation methods and high-quality structural graph data for shear walls. Therefore, this study develops an intelligent design method for shear wall layouts based on GNNs. Two graph representation methods for a shear wall structure—graph edge representation and graph node representation—are examined. A data augmentation method for shear wall structures in graph data form is established to enhance the universality of the GNN performance. An evaluation method for both graph representation methods is developed. Case studies show that the shear wall layout designed using the established GNN method is highly similar to the design by experienced engineers.     

一种改进的降噪自编码神经网络不平衡数据分类算法

针对少数类样本合成过采样技术(Synthetic Minority Over-Sampling Technique, SMOTE)在合成少数类新样本时会带来噪音问题,提出了一种改进降噪自编码神经网络不平衡数据分类算法(SMOTE-SDAE)。该算法首先通过SMOTE方法合成少数类新样本以均衡原始数据集,考虑到合成样本过程中会产生噪音的影响,利用降噪自编码神经网络算法的逐层无监督降噪学习和有监督微调过程,有效实现对过采样数据集的降噪处理与数据分类。在UCI不平衡数据集上实验结果表明,相比传统SVM算法,该算法显著提高了不平衡数据集中少数类的分类精度。     

Towards Self-X cognitive manufacturing network: An industrial knowledge graph-based multi-agent reinforcement learning approach

Empowered by the advanced cognitive computing, industrial Internet-of-Things, and data analytics techniques, today’s smart manufacturing systems are ever-increasingly equipped with cognitive capabilities, towards an emerging Self-X cognitive manufacturing network with higher level of automation. Nevertheless, to our best knowledge, the readiness of ‘Self-X’ levels (e.g., self-configuration, self-optimization, and self-adjust/adaptive/healing) is still in the infant stage. To pave its way, this work stepwise introduces an industrial knowledge graph (IKG)-based multi-agent reinforcement learning (MARL) method for achieving the Self-X cognitive manufacturing network. Firstly, an IKG should be formulated based on the extracted empirical knowledge and recognized patterns in the manufacturing process, by exploiting the massive human-generated and machine-sensed multimodal data. Then, a proposed graph neural network-based embedding algorithm can be performed based on a comprehensive understanding of the established IKG, to achieve semantic-based self-configurable solution searching and task decomposition. Moreover, a MARL-enabled decentralized system is presented to self-optimize the manufacturing process, and to further complement the IKG towards Self-X cognitive manufacturing network. An illustrative example of multi-robot reaching task is conducted lastly to validate the feasibility of the proposed approach. As an explorative study, limitations and future perspectives are also highlighted to attract more open discussions and in-depth research for ever smarter manufacturing.     

BP神经网络在缺失数据估计中的应用

由于飞行参数记录系统所记录的数据很容易被污染,所以对飞参数据进行预处理已显得十分重要,而预处理的一项重要内容就是对缺失参数数据进行合理且有效地估计,真实地反映飞行器当时的状态.通过分析神经网络理论和飞参数据特征,提出了一种基于BP神经网络的缺失数据估计的方法,有效地解决了目前飞行参数记录系统记录数据时缺失数据的问题.利用某型飞机真实的数据进行仿真,结果表明了这种方法是可行且有效的.     

Feature recognition using ART2: a self-organizing neural network

A self-organizing neural network, ART2, based on adaptive resonance theory (ART), is applied to the problem of feature recognition from a boundary representation (B-rep) solid model. A modified face score vector calculation scheme is adopted to represent the features by continuous-valued vectors, suitable to be input to the network. The face score is a measure of the face complexity based upon the convexity or concavity of the surrounding region. The face score vector depicts the topological relations between a face and its neighbouring faces. The ART2 network clusters similar features together. The similarity of the features within a cluster is controlled by a vigilance parameter. A new feature presented to the net is associated with one of the existing clusters, if the feature is similar to the members of the cluster. Otherwise, the net creates a new cluster. An algorithm of the ART2 network is implemented and tested with nine different features. The results obtained indicate that the network has significant potential for application to the problem of feature recognition.     

A hybrid data-fusion system using modal data and probabilistic neural network for damage detection

This paper addresses a novel hybrid data-fusion system for damage detection by integrating the data fusion technique, probabilistic neural network (PNN) models and measured modal data. The hybrid system proposed consists of three models, i.e. a feature-level fusion model, a decision-level fusion model and a single PNN classifier model without data fusion. Underlying this system is the idea that we can choose any of these models for damage detection under different circumstances, i.e. the feature-level model is preferable to other models when enormous data are made available through multi-sensors, whereas the confidence level for each of multi-sensors must be determined (as a prerequisite) before the adoption of the decision-level model, and lastly, the single model is applicable only when data collected is somehow limited as in the cases when few sensors have been installed or are known to be functioning properly. The hybrid system is suitable for damage detection and identification of a complex structure, especially when a huge volume of measured data, often with uncertainties, are involved, such as the data available from a large-scale structural health monitoring system. The numerical simulations conducted by applying the proposed system to detect both single- and multi-damage patterns of a 7-storey steel frame show that the hybrid data-fusion system cannot only reliably identify damage with different noise levels, but also have excellent anti-noise capability and robustness.     

基于层次意图解耦的图卷积神经网络推荐模型

当前意图推荐研究提取出的用户意图趋向扁平化,忽略了意图间的层次关系。针对以上问题,提出了一种基于层次意图解耦的图卷积神经网络推荐模型(HIDR),将用户—项目交互图划分为多个动态交互子图,以刻画从细粒度到粗粒度的用户意图层次图。首先,在每个意图交互子图中根据节点高阶连接性自适应地聚合来自高阶邻域的信息,解耦提取用户细粒度意图表示;然后,依据低层次细粒度意图之间的相似关系在高层网络上构建粗粒度意图超节点,显式建模从细粒度到粗粒度的意图层次结构;最后,将解耦得到的层次意图向量聚合为高质量的用户和项目表示,并进行内积预测和迭代优化。在Gowalla和Amazon-book两个数据集上的实验结果表明,相较于最优基线模型CLSR,HIDR的召回率(recall)分别提升了10.82%、6.63%,归一化折损累计增益(NDCG)分别提升了14.65%、9.63%,精度(precision)分别提升了10.46%和7.73%。     

粗糙集和神经网络方法在数据挖掘中的应用

提出了一种基于神经网络和粗集的数据挖掘新方法。首先利用粗集理论对原始数据进行一致性属性约简,然后使用神经网络对数据进行学习,并同时完成属性的不一致约简,最后再由粗集对神经网络中的知识进行规则抽取。该方法充分融合了粗集理论强大的属性约简、规则生成能力和神经网络优良的分类、容错能力。实验表明,该方法快速有效,生成规则简单准确,具有良好的鲁棒性。     

基于图编码网络的社交网络节点分类方法

针对如何融合节点自身属性以及网络结构信息实现社交网络节点分类的问题,提出了一种基于图编码网络的社交网络节点分类算法。首先,每个节点向邻域节点传播其携带的信息;其次,每个节点通过神经网络挖掘其与邻域节点之间可能隐含的关系,并且将这些关系进行融合;最后,每个节点根据自身信息以及与邻域节点关系的信息提取更高层次的特征,作为节点的表示,并且根据该表示对节点进行分类。在微博数据集上,与经典的深度随机游走模型、逻辑回归算法有以及最近提出的图卷积网络算法相比,所提算法分类准确率均有大于8%的提升;在DBLP数据集上,与多层感知器相比分类准确率提升4.83%,与图卷积网络相比分类准确率提升0.91%。     

Multivariate time series prediction of complex systems based on graph neural networks with location embedding graph structure learning

Graph convolutional neural networks (GNNs) have an excellent expression ability for complex systems. However, the smoothing hypothesis based GNNs have certain limitations for complex process industrial systems with high dynamics and noisy environment. In addition, it is difficult to obtain an accurate information about the interconnections of sensor networks in manufacturing systems, which brings challenges to the application of GNNs. This paper introduces a graph convolution filter with a serial alternating structure of low-pass filter and high-pass filter to alleviate the problem of node feature loss. Furthermore, we propose a simple and effective method to learn graph structure information during training. This method combines the advantages of graph structure learning based on metric method and direct optimization method. Finally, a spatiotemporal parallel feature extraction framework for multivariate time series prediction is constructed. Experiments are carried out on real industrial datasets, and the results demonstrate the effectiveness of the model.     

图神经网络综述

图神经网络和超图神经网络（hypergraph neural network, HGNN）已经成为协同过滤推荐领域的研究热点. 然而实际场景中用户和项目的交互非常复杂,导致用户之间存在高阶的复杂关系,而普通图结构只能表达简单的成对关系,对网络结构的堆叠容易导致中间层表征的过度平滑,在稀疏场景下的用户建模、用户相似性发现与挖掘方面能力较弱;同时,异质超图神经网络的复杂结构使得模型的训练效率较低. 在以微信“搜一搜”等内容平台为代表的高度稀疏数据场景中,对于基于用户所属群体画像的圈层内容推荐任务,现有模型推荐效果差、用户表示的可解释性弱. 因此, 针对该类任务,提出了一个新的轻量同质超图神经网络模型,该模型包含用户交互数据至超图的转化、卷积生成用户表征序列、用户表征计算过滤. 模型首先将用户-项目交互数据转化为只含用户节点的同质超图并计算得到用户表征解耦序列初始值,随后根据超图拉普拉斯过滤矩阵进行信息传播与序列值的迭代生成,通过不使用激活层的卷积方法简化模型结构,并根据提出的均值差JK注意力机制为每个序列值生成权重矩阵. 最终,通过对解耦序列加权求和、过滤实现对用户表示的编码,并在真实数据集上进行实验验证了所提模型的相对更优效果.

     

轻量级多信息图卷积神经网络动作识别方法

针对如何在保持低参数量和低计算量前提下构建高性能模型的问题,提出一种轻量级多信息图卷积神经网络（LMI-GCN）。LMI-GCN通过将关节坐标、关节速度、骨骼边、骨骼边速度四种不同信息编码至高维空间的方式进行信息融合,并引入可以聚合重要特征的多通道自适应图和分流时间卷积块以减少模型参数量。同时,提出一种随机池数据预处理方法。在NTU-RGB+D120数据集上与基线方法SGN（语义引导神经网络）相比,在两种评估设置cross-subject和cross-setup上提高5.4%和4.7%。实验结果表明,LMI-GCN性能高于SGN。     

基于BP神经网络的多传感器数据融合技术优化

传统的数据融合算法要求获得比较精确的对象数学模型,对于复杂的难于建立模型的场合无法适用。为解决上述问题,提出了一种基于BP神经网络算法的多传感器数据融合方法,对对象的先验要求不高,具有较强的自适应能力。仿真结果表明,采用BP神经网络对传感器数据进行融合处理大大提高了传感器的稳定性及其精度,效果良好。     

面向COVID-19疫情预测的图卷积神经网络时空数据学习

目的 当前的疾病传播研究主要集中于时序数据和传染病模型,缺乏运用空间信息提升预测精度的探索和解释。在处理时空数据时需要分别提取时间特征和空间特征,再进行特征融合得到较为可靠的预测结果。本文提出一种基于图卷积神经网络（graph convolutional neural network,GCN）的时空数据学习方法,能够运用空间模型端对端地学习时空数据,代替此前由多模块单元相集成的模式。方法 依据数据可视化阶段呈现出的地理空间、高铁线路、飞机航线与感染人数之间的正相关关系,将中国各城市之间的空间分布关系和交通连接关系映射成网络图并编码成地理邻接矩阵、高铁线路直达矩阵、飞机航线直达矩阵以及飞机航线或高铁线路直达矩阵。按滑动时间窗口对疫情数据进行切片后形成张量,依次分批输入到图深度学习模型中参与卷积运算,通过信息传递、反向传播和梯度下降更新可训练参数。结果 在新型冠状病毒肺炎疫情数据集上的实验结果显示,采用GCN学习这一时空数据的分布特征相较于循环神经网络模型,在训练过程中表现出了更强的拟合能力,在训练时间层面节约75%以上的运算成本,在两类损失函数下的平均测试集损失能够下降80%左右。结论 本文所采用的时空数据学习方法具有较低的运算成本和较高的预测精度,尤其在空间特征强于时间特征的时空数据中有着更好的性能,并且为流行病传播范围和感染人数的预测提供了新的方法和思路,有助于相关部门在公共卫生事件中制定应对措施和疾病防控决策。