基于多头注意力机制的半监督卷积网络嵌入

网络嵌入旨在综合利用网络特性来学习节点的低维向量.然而,传统的网络嵌入方法不能全面考虑外部信息,通常只关注一种属性而忽略其他属性,或者分别学习不同属性的表示.对此提出一种基于多头注意力机制的半监督卷积网络嵌入模型(SMAC).利用近年来在自然语言处理中广泛应用的多头注意机制与多层图卷积神经网络,将外部信息与结构信息以半...     

基于样本关联感知的无监督深度异常检测模型

异常检测的目标是识别正常模式中的异常模式.如何充分利用数据的各种特征信息来识别异常是当前异常检测的研究热点之一.许多数据挖掘及机器学习等方面的智能算法都被用于异常检测规则训练以提高其检测性能.目前已有模型存在着对复杂数据处理困难、没有充分利用数据样本间关联特征等问题,从而造成异常检测效果不甚理想.基于此,本文提出一种基于样本关联感知的深度学习模型并用于异常检测.模型通过对样本的原始特征和样本间的关联关系进行深入分析,利用无向图结构来提取样本间的关联特征,然后基于由特征编码器和图编码器构成的双路自编码器实现对样本的原始特征和关联特征的融合,产生样本在低维特征空间中高质量数据嵌入,然后进行解码重构并计算重构误差和重构特征,最后设计基于高斯混合模型的估计网络,基于重构特征和高质量的数据嵌入估计样本的概率密度,通过给定阈值来进行异常检测.实验结果表明,本模型的异常检测各项性能指标均比其他基于机器学习和深度学习的异常检测方法提升了2％左右,参数、消融和噪声实验结果也较其他算法更稳定,可视化实验也能够突出本模型在数据特征提取和充分利用等方面的优势.     

指针网络改进遗传算法求解旅行商问题

虽然遗传算法相较于其他算法能够更好地求解旅行商问题,但这种算法在使用的过程中容易陷入局部最优的问题,进而导致问题求解遭遇困境。文章在简要介绍旅行商问题的基础上,介绍了遗传算法求解旅行商问题的思路和方法,并明确算法应用中存在的不足。在此基础上提出基于指针网络改进遗传算法求解旅行商问题的新思路,为弥补遗传算法的缺陷提供相应的原理支持。     

基于用户关联的立场检测

立场检测的主要目的是挖掘用户对话题或事件等的立场态度.与其他文本分类任务不同,立场的表达更隐晦,立场的态度对用户更加敏感.目前已有的立场检测方法主要是对话题内容自身信息进行建模,该类方法忽略了话题内容的用户背景信息,但用户及其喜好信息极大地影响着对文本信息的精准挖掘,通过关联用户信息能够获得潜在的信息特征.因此,提出了...     

融合超图注意力机制与图卷积网络的信息扩散预测

针对传统的信息预测缺乏对用户全局性依赖挖掘进行研究,提出了一种融合超图注意力机制与图卷积网络的信息扩散预测模型(HGACN)。首先构建用户社交关系子图,采样获得子级联序列,输入图卷积神经网络学习用户社交关系结构特征;其次,综合考虑用户间和级联间的全局依赖,采用超图注意机制(HGAT)学习用户不同时间间隔的交互特征;最后,将学习到的用户表示捕获到嵌入模块,利用门控机制将其融合获得更具表现力的用户表示,利用带掩码的多头注意力机制进行信息预测。在Twitter等五个数据集上的实验结果表明,提出的HGACN模型在hits@N提高了4.4%,map@N提高了2.2%,都显著优于已有的MS-HGAT等扩散预测模型,证明HGACN模型是合理、有效的。这对谣言监测以及恶意账户的检测有非常重大的意义。     

基于注意力机制融合多阶邻域信息的网络表示学习模型

近年来,图神经网络(Graph Neural Networks,GNNs)在网络表示学习领域中发挥着越来越重要的作用.然而,大多数现有的GNNs在每一层中只考虑节点的直接相连的(1阶)邻居,忽略了高阶邻域信息.在节点表示学习过程中引入高阶拓扑知识是一个关键问题.本文中,我们提出了多邻域注意力图卷积网络(Multi-ne...     

嵌入指针网络的深度循环神经网络模型求解作业车间调度问题

提出了一种数据驱动的作业车间调度算法,训练样本来源于基准实例和部分实际生产数据,通过特征函数来构建样本的特征数据并进行归一化处理,标签数据由调度任务和相应的调度规则的映射关系构成,以LSTM模型为主框架,在模型中嵌入指针网络,将当前序列中概率最大的工件优先进入缓冲区,提高了神经网络的训练速度和质量,采用训练后的模型对新问题进行求解。结果证明了所构建模型的有效性,同时为求解作业车间调度问题提供了新思路。     

指针网络改进遗传算法求解旅行商问题

针对遗传算法在求解旅行商问题时,受限于初始种群质量而存在收敛速度慢、易陷入局部最优等问题,提出一种基于指针网络改进遗传算法种群模型。通过经改进指针网络生成初始种群取代原种群,并结合基于汉明距离轮盘赌策略对种群个体进行择优,形成个体质量和种群多样性高的新种群。实验在TSPLIB标准库上多组实例进行测试,并和研究进展种群改进算法和多种主流启发式算法进行多项系数对比。结果表明,经过优化后算法的收敛速度和寻优能力有显著提高,能够有效用于改善遗传算法在旅行商问题上的应用。     

基于知识图谱的双端知识感知图卷积推荐模型

针对现有基于知识图谱的推荐模型仅从用户或项目一端进行特征提取, 从而缺乏对另一端的特征提取的问题, 提出一种基于知识图谱的双端知识感知图卷积推荐模型. 首先, 对于用户、项目及知识图谱中的实体进行随机初始化表征得到初始特征表示; 接着, 采用基于用户和项目的知识感知注意力机制同时从用户、项目两端在知识图谱中进行特征提取; 其次, 使用图卷积网络采用不同的聚合方式聚合知识图谱传播过程中的特征信息并预测点击率; 最后, 为了验证模型的有效性, 在Last.FM和Book-Crossing两个公开数据集上与4个基线模型进行对比实验. 在Last.FM数据集上, AUC和F1分别比最优的基线模型提升了4.4%、3.8%, ACC提升了1.1%. 在Book-Crossing数据集上, AUC和F1分别提升了1.5%、2.2%, ACC提升了1.4%. 实验结果表明, 本文的模型在AUC、F1和ACC指标上比其他的基线模型具有更好的鲁棒性.     

面向交通流量预测的多头注意力时空卷积图网络模型

为了充分获取交通流量数据中隐藏的复杂动态时空相关性,提高交通流量预测精度,提出一种多头注意力时空卷积图网络模型MASCGN。首先,采用多头注意力机制为路网中的交通传感器节点自动分配注意力权重,实现对不同邻居节点的权值自适应匹配,充分获取空间相关性;其次,采用带有门控和注意力机制的时空卷积网络充分提取时间序列相关性,并使用残差块结构实现时空卷积层之间的连接,使得模型更具有泛化能力;最后,分别提取周相关、日相关、邻近时间的序列数据,输入三个并行的时空组件以挖掘周、日、邻近三个时间窗口间的时间周期相关性,并通过全连接层获取最终的交通流量预测结果。利用高速公路交通数据集PEMSO4、PEMSO8进行了15 min、30 min、45 min和60 min的交通流量预测实验。实验结果表明MASCGN模型与现有基线模型相比,在未来短期和长期的交通流量预测任务上都具有更优的建模能力。     

SESF-Fuse: an unsupervised deep model for multi-focus image fusion

Neural Computing and Applications - Muti-focus image fusion is the extraction of focused regions from different images to create one all-in-focus fused image. The key point is that only objects...     

EMFusion: An unsupervised enhanced medical image fusion network

Existing image fusion methods always use the same representations for different modal medical images. Otherwise, they solve the fusion problem by subjectively defining characteristics to be preserved. However, it leads to the distortion of unique information and restricts the fusion performance. To address the limitations, this paper proposes an unsupervised enhanced medical image fusion network. We perform both surface-level and deep-level constraints for enhanced information preservation. The surface-level constraint is based on the saliency and abundance measurement to preserve the subjectively defined and intuitive characteristics. In the deep-level constraint, the unique information is objectively defined based on the unique channels of a pre-trained encoder. Moreover, in our method, the chrominance information of fusion results is also enhanced. It is because we use the high-quality details in structural images (e.g., MRI) to alleviate the mosaic in functional images (e.g., PET, SPECT). Both qualitative and quantitative experiments demonstrate the superiority of our method over the state-of-the-art fusion methods.     

I2DKPCN: an unsupervised deep learning network

Applied Intelligence - In this paper, we proposed an incremental two-dimensional kernel PCA-based convolutional network (I2DKPCN) which is a novel unsupervised deep learning network. In our...     

基于自注意力与指针网络的自动摘要模型

针对生成式摘要方法中的序列到序列模型存在准确率不高、 词语重复、 训练时间长等问题,提出一个改进的模型.引入自注意力机制替代原有循环神经网络和卷积神经网络,实现并行训练和损失函数值的快速下降与稳定,减少训练时间;引入指针网络解决未登录词问题,将未登录词直接扩展到字典中,实现将未登录词从输入序列复制到生成序列中;引入输入...     

基于无监督深度图像生成的盲降噪模型

鉴于有监督神经网络降噪模型的数据依赖缺陷,提出了一种基于无监督深度生成（UDIG）的盲降噪模型。首先,利用噪声水平评估（NLE）算法测定给定噪声图像中的噪声水平值并输入到主流FFDNet降噪模型中,所得到降噪后的图像（称为初步降噪图像）作为UDIG降噪模型的输入。其次,选用编码器—解码器架构作为UDIG模型的骨干网络并用UDIG模型的输出图像（即生成图像）分别与初步降噪图像、噪声图像之间的均方误差之和构建混合loss函数;再次,以loss最小化为优化目标,通过随机梯度下降（SGD）网络训练算法调整网络模型的参数值从而获得一系列生成图像;最后,当残差图像（噪声图像与生成图像之间）的标准差逼近之前NLE算法所测定的噪声水平估计值时及时终止网络迭代训练过程,从而确保生成图像（作为降噪后图像）的图像质量最佳。实验结果表明：与现有的主流降噪模型（算法）相比,UDIG降噪模型在降噪效果上具有显著优势。     

A regularization-based deep unsupervised model for affine multimodal co-registration

Extensive network receptive field is key for unsupervised affine registration because instead of deformable registration that takes care of local subtleties, the affine registration is global so that the last layers need to see big patches of the organ-in-interest. To extend the network's receptive field, we need to go for deeper networks, which causes producing complex models. On the other hand, affine transformation is restricted by its low degree-of-freedom (DoF) where larger models increasingly develop the hazard of overfitting. To worsen the situation, the regularizer module cannot be applied to the affine transformation with such a restricted DoF. In this paper, we propose a differentiable computational layer to convert the affine transformation outputted by the network to its corresponding dense displacement field. Such an affine-to-field layer enables us to apply different regularization terms on the outputted transformation in order to avoid the overfitting phenomenon while deepening the network. The proposed approach was evaluated on an annotated hard multimodal dataset containing 1109 pairs of CT/MR images of the brain with different heterogeneity for example, variety in scanners, setups and resolutions. Based on the results, the proposed customized layer is fully successful to handle the overfitting for deeper networks that are able to produce richer transformations than the shallower networks from different evaluation metrics for example, in target registration error the proposed network with seven layers has a 13.3% (or 9.1 mm) improvement in performance. The implementation of the proposed customized affine-to-field layer in the Python, Keras package with the Tensorflow backend can be publically accessed via https://github.com/boveiri/Deep-coReg .     

A neural network model which combines unsupervised and supervisedlearning

A neural network that combines unsupervised and supervised learning for pattern recognition is proposed. The network is a hierarchical self-organization map, which is trained by unsupervised learning at first. When the network fails to recognize similar patterns, supervised learning is applied to teach the network to give different scaling factors for different features so as to discriminate similar patterns. Simulation results show that the model obtains good generalization capability as well as sharp discrimination between similar patterns.     

UIFGAN: An unsupervised continual-learning generative adversarial network for unified image fusion

In this paper, we propose a novel unsupervised continual-learning generative adversarial network for unified image fusion, termed as UIFGAN. In our model, for multiple image fusion tasks, a generative adversarial network for training a single model with memory in a continual-learning manner is proposed, rather than training an individual model for each fusion task or jointly training multiple tasks. We use elastic weight consolidation to avoid forgetting what has been learned from previous tasks when training multiple tasks sequentially. In each task, the generation of the fused image comes from the adversarial learning between a generator and a discriminator. Meanwhile, a max-gradient loss function is adopted for forcing the fused image to obtain richer texture details of the corresponding regions in two source images, which applies to most typical image fusion tasks. Extensive experiments on multi-exposure, multi-modal and multi-focus image fusion tasks demonstrate the advantages of our method over the state-of-the-art approaches.     

Human action recognition using three orthogonal planes with unsupervised deep convolutional neural network

Deep learning models have attained great success for an extensive range of computer vision applications including image and video classification. However, the complex architecture of the most recently developed networks imposes certain memory and computational resource limitations, especially for human action recognition applications. Unsupervised deep convolutional neural networks such as PCANet can alleviate these limitations and hence significantly reduce the computational complexity of the whole recognition system. In this work, instead of using 3D convolutional neural network architecture to learn temporal features of video actions, the unsupervised convolutional PCANet model is extended into (PCANet-TOP) which effectively learn spatiotemporal features from Three Orthogonal Planes (TOP). For each video sequence, spatial frames (XY) and temporal planes (XT and YT) are utilized to train three different PCANet models. Then, the learned features are fused after reducing their dimensionality using whitening PCA to obtain spatiotemporal feature representation of the action video. Finally, Support Vector Machine (SVM) classifier is applied for action classification process. The proposed method is evaluated on four benchmarks and well-known datasets, namely, Weizmann, KTH, UCF Sports, and YouTube action datasets. The recognition results show that the proposed PCANet-TOP provides discriminative and complementary features using three orthogonal planes and able to achieve promising and comparable results with state-of-the-art methods.