基于反卷积特征提取的深度卷积神经网络学习

在深度卷积神经网络的学习过程中,卷积核的初始值通常是随机赋值的.另外,基于梯度下降法的网络参数学习法通常会导致梯度弥散现象.鉴于此,提出一种基于反卷积特征提取的深度卷积神经网络学习方法.首先,采用无监督两层堆叠反卷积神经网络从原始图像中学习得到特征映射矩阵;然后,将该特征映射矩阵作为深度卷积神经网络的卷积核,对原始图像进行逐层卷积和池化操作;最后,采用附加动量系数的小批次随机梯度下降法对深度卷积网络微调以避免梯度弥散问题.在MNIST、CIFAR-10和CIFAR-100数据集上的实验结果表明,所提出方法可有效提高图像分类精度.     

区分交互意图的图卷积协同过滤算法

近几年提出了一些基于图卷积网络的协同过滤推荐模型,然而大部分模型将邻域权重视为常量且不区分用户和物品间的交互关系,无法获取令用户满意的推荐列表。因此,为了得到用户和物品更准确的嵌入表示,提出一种区分交互意图的图卷积协同过滤推荐算法MiGCCF(multi-intention graph convolutional collaborative filtering)。该算法将交互关系进行分解,细粒度分析用户与物品间的交互意图,并引入注意力机制,在消息传播过程中赋予邻域可学习的注意力权重,挖掘用户对于不同交互物品的喜爱度。在Gowalla与Amazon-book上的实验表明,该算法相比于基准算法,在两个数据集上的HR@50和NDCG@50指标分别提高了12.5%和8.5%,具有更好的性能表现。     

基于图卷积的双通道协同过滤推荐算法

在推荐系统中,利用图卷积网络等方法提取图的高阶信息缓解了冷启动问题。为了在此基础上融合神经网络协同过滤的深层特征提取能力,提出一种基于图卷积的双通道协同过滤推荐算法（GCNCF-2C）。首先,将推荐问题分为上游任务和下游任务;其次,在上游任务中,预训练编码器利用包含残差的一维卷积层和多个图卷积层在两个独立通道中对节点特征和图高阶特征进行分离提取,形成节点的特征表示;最后,解码器通过节点特征进行评级预测,进行端到端的训练。在数据集MovieLens-100K和MovieLens-1M上的实验表明,该算法相比于基线模型在两个数据集上的RMSE指标平均提高1.72%和1.76%,MAE指标平均提高2.7%和1.98%,同时在基于用户和项目的冷启动实验中RMSE指标平均提高5.9%,具有更好的综合性能。     

Flexible data representation with feature convolution for semi-supervised learning

Applied Intelligence - Data representation plays a crucial role in semi-supervised learning. This paper proposes a framework for semi-supervised data representation. It introduces a flexible...     

融合超图注意力机制与图卷积网络的信息扩散预测

针对传统的信息预测缺乏对用户全局性依赖挖掘进行研究,提出了一种融合超图注意力机制与图卷积网络的信息扩散预测模型(HGACN)。首先构建用户社交关系子图,采样获得子级联序列,输入图卷积神经网络学习用户社交关系结构特征;其次,综合考虑用户间和级联间的全局依赖,采用超图注意机制(HGAT)学习用户不同时间间隔的交互特征;最后,将学习到的用户表示捕获到嵌入模块,利用门控机制将其融合获得更具表现力的用户表示,利用带掩码的多头注意力机制进行信息预测。在Twitter等五个数据集上的实验结果表明,提出的HGACN模型在hits@N提高了4.4%,map@N提高了2.2%,都显著优于已有的MS-HGAT等扩散预测模型,证明HGACN模型是合理、有效的。这对谣言监测以及恶意账户的检测有非常重大的意义。     

基于超图卷积网络和目标多意图感知的会话推荐算法

当前先进的会话推荐算法主要通过图神经网络从全局和目标会话中挖掘项目的成对转换关系,并将目标会话压缩成固定的向量表示,忽略了项目间复杂的高阶信息和目标项目对用户偏好多样性的影响。为此提出了基于超图卷积网络和目标多意图感知的会话推荐算法HCN-TMP。通过学习会话表示来表达用户偏好,首先依据目标会话构建会话图,依据全局会话构建超图,通过意图解纠缠技术将原有反映用户耦合意图的项目嵌入表示转换为项目多因素嵌入表示,再经图注意力网络和超图卷积网络分别学习目标会话节点的会话级和全局级项目表示,并使用距离相关性损失函数增强多因素嵌入块间的独立性;然后嵌入目标会话中节点位置信息,加权每个节点的注意力权重,得到全局级和会话级会话表示;利用对比学习最大化两者互信息,经目标多意图感知,针对不同的目标项目自适应地学习目标会话中多意图的用户偏好,得到目标感知级会话表示,最后线性融合三个级别的会话表示得到最终的会话表示。在Tmall和Nowplaying两个公开数据集上进行大量实验,实验结果验证了HCN-TMP算法的有效性。     

自监督对比特征学习的多模态乳腺超声诊断

超声图像的乳腺癌自动诊断具有重要的临床价值。然而,由于缺乏大量人工标注数据,构建高精度的自动诊断方法极具挑战。近年来,自监督对比学习在利用无标签自然图像产生具有辨别性和高度泛化性的特征方面展现出巨大潜力。然而,采用自然图像构建正负样本的方法在乳腺超声领域并不适用。为此,本文引入超声弹性图像(elastography ultrasound, EUS),利用超声图像的多模态特性,提出一种融合多模态信息的自监督对比学习方法。该方法采用同一病人的多模态超声图像构造正样本;采用不同病人的多模态超声图像构建负样本;基于模态一致性、旋转不变性和样本分离性来构建对比学习的目标学习准则。通过在嵌入空间中学习两种模态的统一特征表示,从而将EUS信息融入模型,提高了模型在下游B型超声分类任务中的表现。实验结果表明本文提出的方法能够在无标签的情况下充分挖掘多模态乳腺超声图像中的高阶语义特征,有效提高乳腺癌的诊断正确率。     

Improving backpropagation learning with feature selection

There exist redundant, irrelevant and noisy data. Using proper data to train a network can speed up training, simplify the learned structure, and improve its performance. A two-phase training algorithm is proposed. In the first phase, the number of input units of the network is determined by using an information base method. Only those attributes that meet certain criteria for inclusion will be considered as the input to the network. In the second phase, the number of hidden units of the network is selected automatically based on the performance of the network on the training data. One hidden unit is added at a time only if it is necessary. The experimental results show that this new algorithm can achieve a faster learning time, a simpler network and an improved performance.     

结合句法特征和卷积神经网络的多意图识别模型

短文本的多意图识别是口语理解（SLU）中的难题,因短文本的特征稀疏、字数少但包含信息量大,在分类问题中难以提取其有效特征。为解决该问题,将句法特征和卷积神经网络（CNN）进行结合,提出一种多意图识别模型。首先,将句子进行依存句法分析以确定是否包含多意图;然后,利用词频-逆文档频率（TF-IDF）和训练好的词向量计算距离矩阵,以确定意图的个数;其次,把该距离矩阵作为CNN模型的输入,进行意图分类;最后,判断每个意图的情感极性,计算用户的真实意图。采用现有的智能客服系统的真实数据进行实验,实验结果表明,结合句法特征的CNN模型在10个意图上的单分类精准率达到93.5%,比未结合句法特征的CNN模型高1.4个百分点;而在多意图识别上,精准率比其他模型提高约30个百分点。     

基于多尺度特征融合和对比池化的点云补全网络

点云补全在点云处理任务中具有重要作用,它可以提高数据质量、辅助生成精确三维模型,为多种应用提供可靠数据支撑。然而,现有基于深度网络的点云补全算法采用的单层次全局特征提取方法较为简单,没有充分挖掘潜在语义信息,并在编码过程中丢失部分细节信息。为解决这些问题,提出了一种多尺度特征逐级融合的点云补全网络,并结合注意力机制提出了一种全新的池化方法。实验结果表明,在PCN、ShapeNet34和ShapeNet55三个数据集上取得了SOTA水平,证明该网络具有更好的特征表示能力和补全效果。     

Optimized collaborative filtering recommendation based on users' interest degree and feature

协同过滤技术目前被广泛应用于个性化推荐系统中.为了使用户的最近邻居集合更加精确有效,提出了基于用户兴趣度和用户特征的优化协同过滤推荐算法.首先通过计算用户对项目的兴趣度来对用户进行分组;然后采用贝叶斯算法分析出用户具有不同特征时对项目的喜好程度;最后采用一种新的相似度度量方法计算出目标用户的最近邻居集合.实验表明该算法提高了最近邻居集合的有效性和准确度,推荐质量较以往算法有明显提高.     

融合协同过滤信息的知识图注意力网络

知识图谱（KG）能够缓解协同过滤算法存在的数据稀疏和冷启动问题,在推荐领域被广泛地研究和应用。现有的很多基于KG的推荐模型混淆了用户物品二部图中的协同过滤信息和KG中实体间的关联信息,导致学习到的用户向量和物品向量无法准确表达其特征,甚至引入与用户、物品无关的信息从而干扰推荐。针对上述问题提出一种融合协同信息的知识图注意力网络（KGANCF）。首先,为了避免KG实体信息的干扰,网络的协同过滤层从用户物品二部图中挖掘出用户和物品的协同过滤信息;然后,在知识图注意力嵌入层中应用图注意力机制,从KG中继续提取与用户和物品密切相关的属性信息;最后,在预测层将用户物品的协同过滤信息和KG中的属性信息融合,得到用户和物品最终向量表示,进而预测用户对物品的评分。在MovieLens-20M和Last.FM数据集上进行了实验,与协同知识感知注意力网络（CKAN）相比,KGANCF在MovieLens-20M数据集上的F1分数提升了1.1个百分点,曲线下面积（AUC）提升了0.6个百分点;而在KG相对稀疏的Last.FM数据集上,模型的F1分数提升了3.3个百分点,AUC提升了8.5个百分点。实验结果表明,KGANCF能够有效提高推荐结果的准确度,在KG稀疏的数据集上显著优于协同知识嵌入（CKE）、知识图谱卷积网络（KGCN）、知识图注意网络（KGAT）和CKAN模型。     

基于自适应特征卷积网络的行人检测方法

针对行人检测方法未能充分利用卷积网络浅层特征的问题,改进Faster R-CNN框架,提出了一种基于自适应特征卷积网络的行人检测方法。该方法有两处改进：a）设计了SFCM模块,用于提取卷积神经网络浅层细节特征;b）引用挤压与激励操作设计了AFCM模块,用于筛选检测所需的强辨识力行人特征。此外,利用公开的Caltech和INRIA行人数据集,通过在基准框架中逐一添加SFCM和AFCM模块训练行人检测器,验证了所提模块的有效性,并对比了主流行人检测算法。实验结果显示,所提方法的误检率分别降到了9.13%和9.46%,具有更优的检测性能。     

基于对比学习的多特征融合戴口罩人脸识别

随着计算机视觉技术应用的发展和智能终端的普及,口罩遮挡人脸识别已成为人物身份信息识别的重要部分。口罩的大面积遮挡对人脸特征的学习带来极大挑战。针对戴口罩人脸特征学习困难这一问题,提出了一种基于对比学习的多特征融合口罩遮挡人脸识别算法,该算法改进了传统的基于三元组关系的人脸特征向量学习损失函数,提出了基于多实例关系的损失函数,充分挖掘戴口罩人脸和完整人脸多个正负样本之间的同模态内和跨模态间的关联关系,学习人脸中具有高区分度的能力的特征,同时结合人脸的眉眼等局部特征和轮廓等全局特征,学习口罩遮挡人脸的有效特征向量表示。在真实的戴口罩人脸数据集和生成的戴口罩人脸数据上与基准算法进行了比较,实验结果表明所提算法相比传统的基于三元组损失函数和特征融合算法具有更高的识别准确率。     

Pessimists and optimists: Improving collaborative filtering through sentiment analysis

This work presents a novel application of Sentiment Analysis in Recommender Systems by categorizing users according to the average polarity of their comments. These categories are used as attributes in Collaborative Filtering algorithms. To test this solution a new corpus of opinions on movies obtained from the Internet Movie Database (IMDb) has been generated, so both ratings and comments are available. The experiments stress the informative value of comments. By applying Sentiment Analysis approaches some Collaborative Filtering algorithms can be improved in rating prediction tasks. The results indicate that we obtain a more reliable prediction considering only the opinion text (RMSE of 1.868), than when apply similarities over the entire user community (RMSE of 2.134) and sentiment analysis can be advantageous to recommender systems.     

Improving memory-based collaborative filtering via similarity updating and prediction modulation

Memory-based collaborative filtering (CF) makes recommendations based on a collection of user preferences for items. The idea underlying this approach is that the interests of an active user will more likely coincide with those of users who share similar preferences to the active user. Hence, the choice and computation of a similarity measure between users is critical to rating items. This work proposes a similarity update method that uses an iterative message passing procedure. Additionally, this work deals with a drawback of using the popular mean absolute error (MAE) for performance evaluation, namely that ignores ratings distribution. A novel modulation method and an accuracy metric are presented in order to minimize the predictive accuracy error and to evenly distribute predicted ratings over true rating scales. Preliminary results show that the proposed similarity update and prediction modulation techniques significantly improve the predicted rankings.     

Equivalence of backpropagation and contrastive Hebbian learning in a layered network

Backpropagation and contrastive Hebbian learning are two methods of training networks with hidden neurons. Backpropagation computes an error signal for the output neurons and spreads it over the hidden neurons. Contrastive Hebbian learning involves clamping the output neurons at desired values and letting the effect spread through feedback connections over the entire network. To investigate the relationship between these two forms of learning, we consider a special case in which they are identical: a multilayer perceptron with linear output units, to which weak feedback connections have been added. In this case, the change in network state caused by clamping the output neurons turns out to be the same as the error signal spread by backpropagation, except for a scalar prefactor. This suggests that the functionality of backpropagation can be realized alternatively by a Hebbian-type learning algorithm, which is suitable for implementation in biological networks.     

基于多尺度特征递归卷积的稠密点云重建网络

针对在三维重建任务中,由于弱纹理区域的光度一致性测量误差较大,使得传统的多视图立体算法难以处理的问题,提出了一种多尺度特征聚合的递归卷积网络(MARDC-MVSNet),用于弱纹理区域的稠密点云重建。为了使输入图像分辨率更高,该方法使用一个轻量级的多尺度聚合模块自适应地提取图像特征,以解决弱纹理甚至无纹理区域的问题。在代价体正则化方面,采用具有递归结构的分层处理网络代替传统的三维卷积神经网络(CNN),极大程度地降低了显存占用,同时实现高分辨率重建。在网络的末端添加一个深度残差网络模块,以原始图像为指导对正则化网络生成的初始深度图进行优化,使深度图表述更准确。实验结果表明,在DTU数据集上取得了优异的结果,该网络在拥有较高深度图估计精度的同时还节约了硬件资源,且能扩展到航拍影像的实际工程之中。     

Improving collaborative filtering recommender system results and performance using genetic algorithms

This paper presents a metric to measure similarity between users, which is applicable in collaborative filtering processes carried out in recommender systems. The proposed metric is formulated via a simple linear combination of values and weights. Values are calculated for each pair of users between which the similarity is obtained, whilst weights are only calculated once, making use of a prior stage in which a genetic algorithm extracts weightings from the recommender system which depend on the specific nature of the data from each recommender system. The results obtained present significant improvements in prediction quality, recommendation quality and performance.     

Supervised contrastive learning over prototype-label embeddings for network intrusion detection

Contrastive learning makes it possible to establish similarities between samples by comparing their distances in an intermediate representation space (embedding space) and using loss functions designed to attract/repel similar/dissimilar samples. The distance comparison is based exclusively on the sample features. We propose a novel contrastive learning scheme by including the labels in the same embedding space as the features and performing the distance comparison between features and labels in this shared embedding space. Following this idea, the sample features should be close to its ground-truth (positive) label and away from the other labels (negative labels). This scheme allows to implement a supervised classification based on contrastive learning. Each embedded label will assume the role of a class prototype in embedding space, with sample features that share the label gathering around it. The aim is to separate the label prototypes while minimizing the distance between each prototype and its same-class samples. A novel set of loss functions is proposed with this objective. Loss minimization will drive the allocation of sample features and labels in embedding space. Loss functions and their associated training and prediction architectures are analyzed in detail, along with different strategies for label separation. The proposed scheme drastically reduces the number of pair-wise comparisons, thus improving model performance. In order to further reduce the number of pair-wise comparisons, this initial scheme is extended by replacing the set of negative labels by its best single representative: either the negative label nearest to the sample features or the centroid of the cluster of negative labels. This idea creates a new subset of models which are analyzed in detail.The outputs of the proposed models are the distances (in embedding space) between each sample and the label prototypes. These distances can be used to perform classification (minimum distance label), features dimensionality reduction (using the distances and the embeddings instead of the original features) and data visualization (with 2 or 3D embeddings).Although the proposed models are generic, their application and performance evaluation is done here for network intrusion detection, characterized by noisy and unbalanced labels and a challenging classification of the various types of attacks. Empirical results of the model applied to intrusion detection are presented in detail for two well-known intrusion detection datasets, and a thorough set of classification and clustering performance evaluation metrics are included.