基于语义理解注意力神经网络的多元特征融合中文文本分类

在中文文本分类任务中,针对重要特征在中文文本中位置分布分散、稀疏的问题,以及不同文本特征对文本类别识别贡献不同的问题,该文提出一种基于语义理解的注意力神经网络、长短期记忆网络(LSTM)与卷积神经网络(CNN)的多元特征融合中文文本分类模型(3CLA)。模型首先通过文本预处理将中文文本分词、向量化。然后,通过嵌入层分别经过CNN通路、LSTM通路和注意力算法模型通路以提取不同层次、具有不同特点的文本特征。最终,文本特征经融合层融合后,由softmax分类器进行分类。基于中文语料进行了文本分类实验。实验结果表明,相较于CNN结构模型与LSTM结构模型,提出的算法模型对中文文本类别的识别能力最多提升约8%。     

基于多通道注意力机制的文本分类模型

为解决卷积神经网络(CNN)和循环神经网络(RNN)处理文本分类任务时,由于文本特征稀疏造成的关键特征信息丢失、模型性能不高和分类效果不佳等问题.提出一种基于多通道注意力机制的文本分类模型,首先利用字词融合的形式进行向量表示,然后利用CNN和BiLSTM提取文本的局部特征和上下文关联信息,接着以注意力机制对各通道的输出...     

融合词嵌入和通道注意力机制的网络安全态势预测

为了解决门控神经网络(Gated Recurrent Neural Network,GRU)不能很好地注意到关键词且训练速度较慢的问题,提出融合词嵌入和通道注意力机制的网络安全态势预测模型(E-SE-CNN)。首先,根据专家意见,采用正则表达式分割告警日志数据(网络安全设备捕捉到的Web攻击日志)构建多标签分类数据集;其次,利用Embedding进行词嵌入,获取词的分布式表示,使其能够充分地表示语义信息;再次,引入通道注意力机制(Squeeze and Excitation)和卷积神经网络(Convolutional Neural Networks,CNN),为不同的通道分配不同的权重,使得模型能够进一步获取文本的重要的语义信息;最后,将包含权重信息以及语义信息的特征图送入sigmoid层进行多标签的Web攻击分类。为了进一步验证E-SE-CNN模型的性能,设计了5组相关模型进行进一步验证。实验结果表明,在自建的Web攻击数据集上,在保证训练推理速度的同时,提出的模型在5种评估指标上均具有最佳的表现。     

基于深度学习的垃圾邮件文本分类方法

文中旨在研究基于深度学习的垃圾邮件文本分类方法，该方法结合了卷积神经网络（CNN）和循环神经网络（RNN）的模型，通过对邮件文本进行特征提取和分类，能高效、准确地对垃圾邮件进行分类。文中以卷积神经网络和循环神经网络为实验对象，提出了一种垃圾邮件文本分类方法，并在公开数据集上进行了实验。实验结果表明，该方法在垃圾邮件文本分类任务上具有较高的准确率和召回率。     

基于SWCNN和双向LSTM的文本情感特征分析方法

针对文本句子中语义角色重叠、高维度文本词向量训练中难以收敛等问题,将情感词标签与卷积神经网络相结合,采用结合情感词的卷积神经网络算法,将词语转为情感标签后与词向量拼接再输入卷积神经网络,将输出的特征再与双向长短期记忆神经网络所获取的特征进行融合,最后通过全连接网络输出情感分类结果。实验结果表明,在微博新冠疫情评论情绪数据集上,本研究所提出的算法模型文本情感特征识别精确度达到89.23%,比其他深度学习算法在准确率上至少提高1.95%,而且训练具有更快的收敛速度,能够为文本情感识别提供一种新的思路与方法。     

基于高效用神经网络的文本分类方法

现有的基于深度学习的文本分类方法没有考虑文本特征的重要性和特征之间的关联关系,影响了分类的准确率.针对此问题,本文提出一种基于高效用神经网络（High Utility Neural Networks,HUNN）的文本分类模型,可以有效地表示文本特征的重要性及其关联关系.利用高效用项集挖掘（Mining High Utility Itemsets,MHUI）算法获取数据集中各个特征的重要性以及共现频率.其中,共现频率在一定程度上反映了特征之间的关联关系.将MHUI作为HUNN的挖掘层,用于挖掘每个类别数据中重要性和关联性强的文本特征.然后将这些特征作为神经网络的输入,再经过卷积层进一步提炼类别表达能力更强的高层次文本特征,从而提高模型分类的准确率.通过在6个公开的基准数据集上进行实验分析,提出的算法优于卷积神经网络（Convolutional Neural Networks,CNN）,循环神经网络（Recurrent Neural Networks,RNN）,循环卷积神经网络（Recurrent Convolutional Neural Networks,RCNN）,快速文本分类（Fast Text Classifier,FAST）,分层注意力网络（Hierarchical Attention Networks,HAN）等5个基准算法.     

一种基于潜在语义分析和直推式谱图算法的文本分类方法LSASGT

 本文针对训练数据较少以及在基于图的分类算法中的文本表示问题,提出了一种基于潜在语义分析技术和直推式谱图算法的文本分类方法LSASGT,该方法将潜在语义分析技术和直推式谱图算法这两种基于谱分析理论的技术有机地结合在一起,对所有训练数据和测试数据进行统一建模,挖掘数据中潜在的多种结构信息.LSASGT引入潜在语义分析技术用于构造文本图表示模型,在能够反映人的分类标准的潜在语义特征空间中,描述文本之间的语义相关性;基于这样的文本表示,利用半监督的直推式谱图算法进行文本分类.在基准英文文本分类数据集Reuters21578和中文文本分类数据集Tan-Corp上的实验结果表明,本文给出的LSASGT文本分类方法获得了较好的分类结果.     

一种基于修正激活函数的CNN车载毫米波雷达目标检测方法

为了提高车载毫米波雷达在复杂城市道路环境中目标检测的抗杂波与干扰能力,本文利用卷积神经网络（CNN）特征参数提取和目标分类特性,提出了一种改进的基于CNN的车载毫米波雷达目标检测方法。该方法首先将毫米波雷达回波信号距离-多普勒二维数据运用滑窗进行分割,并采用CNN网络模型处理分割后的二维矩阵,训练二维CNN网络模型及其参数,使其具有提取回波特征并基于特征参数模型进行目标分类的能力,从而实现目标检测功能。通过对卷积神经网络模型结构进行优化,增加批量归一化层,优化Dropout层使得低权重特征失活,自适应地删减部分神经元节点修正该层非线性激活函数,进一步降低了CNN模型目标检测的虚警概率。实验结果表明,在相同虚警概率条件下,CNN网络检测方法目标发现概率优于传统的单元平均恒虚警检测方法,并且在低信噪比的条件下仍然能够保持较高的发现概率;在同等发现概率水平下,修正后CNN网络检测方法的虚警概率较修正前可提高约1个数量级。     

中文长短期记忆网络的情感分析

文中探讨了循环神经网络（RNN）在文本情感分析中的应用，并提出了一种新的混合模型——LSTM-CNN文本情感分析模型，它融合了长短时记忆网络（LSTM）和卷积神经网络（CNN）的优势，可以更好地处理文本数据中的上下文信息和局部特征。为了验证模型的有效性，文中使用开源爬虫工具抓取了《流浪地球2》的豆瓣评论构建数据集，然后对所提模型进行了训练和测试。结果表明，当词向量维度为100时，模型的性能达到最优，此时的精确率、召回率、F1值和准确率分别为84.2%，88.6%，86.2%和90.0%，证实了该模型在文本情感分类任务上的优越性。     

结合双预训练语言模型的中文文本分类模型

针对Word2Vec等模型所表示的词向量存在语义模糊从而导致的特征稀疏问题,提出一种结合自编码和广义自回归预训练语言模型的文本分类方法。首先,分别通过BERT、XLNet对文本进行特征表示,提取一词多义、词语位置及词间联系等语义特征;再分别通过双向长短期记忆网络(BiLSTM)充分提取上下文特征,最后分别使用自注意力机制(Self＿Attention)和层归一化(Layer Normalization)实现语义增强,并将两通道文本向量进行特征融合,获取更接近原文的语义特征,提升文本分类效果。将提出的文本分类模型与多个深度学习模型在3个数据集上进行对比,实验结果表明,相较于基于传统的Word2Vec以及BERT、XLNet词向量表示的文本分类模型,改进模型获得更高的准确率和F₁值,证明了改进模型的分类有效性。     

Research on image feature extraction and retrieval algorithms based on convolutional neural network

With the rapid development of mobile Internet and digital technology, people are more and more keen to share pictures on social networks, and online pictures have exploded. How to retrieve similar images from large-scale images has always been a hot issue in the field of image retrieval, and the selection of image features largely affects the performance of image retrieval. The Convolutional Neural Networks (CNN), which contains more hidden layers, has more complex network structure and stronger ability of feature learning and expression compared with traditional feature extraction methods. By analyzing the disadvantage that global CNN features cannot effectively describe local details when they act on image retrieval tasks, a strategy of aggregating low-level CNN feature maps to generate local features is proposed. The high-level features of CNN model pay more attention to semantic information, but the low-level features pay more attention to local details. Using the increasingly abstract characteristics of CNN model from low to high. This paper presents a probabilistic semantic retrieval algorithm, proposes a probabilistic semantic hash retrieval method based on CNN, and designs a new end-to-end supervised learning framework, which can simultaneously learn semantic features and hash features to achieve fast image retrieval. Using convolution network, the error rate is reduced to 14.41% in this test set. In three open image libraries, namely Oxford, Holidays and ImageNet, the performance of traditional SIFT-based retrieval algorithms and other CNN-based image retrieval algorithms in tasks are compared and analyzed. The experimental results show that the proposed algorithm is superior to other contrast algorithms in terms of comprehensive retrieval effect and retrieval time.     

一种面向隐含主题的上下文树核

该文针对上下文树核用于文本表示时缺乏语义信息的问题,提出了一种面向隐含主题的上下文树核构造方法。首先采用隐含狄利克雷分配将文本中的词语映射到隐含主题空间,然后以隐含主题为单位建立上下文树模型,最后利用模型间的互信息构造上下文树核。该方法以词的语义类别来定义文本的生成模型,解决了基于词的文本建模时所遇到的统计数据的稀疏性问题。在文本数据集上的聚类实验结果表明,文中提出的上下文树核能够更好地度量文本间主题的相似性,提高了文本聚类的性能。     

Deep global-attention based convolutional network with dense connections for text classification

Text classification is a classic task innatural language process (NLP). Convolutional neural networks (CNNs) have demonstrated its effectiveness in sentence and document modeling. However, most of existing CNN models are applied to the fixed-size convolution filters, thereby unable to adapt different local interdependency. To address this problem, a deep global-attention based convolutional network with dense connections (DGA-CCN) is proposed. In the framework, dense connections are applied to connect each convolution layer to each of the other layers which can accept information from all previous layers and get multiple sizes of local information. Then the local information extracted by the convolution layer is reweighted by deep global-attention to obtain a sequence representation with more valuable information of the whole sequence. A series of experiments are conducted on five text classification benchmarks, and the experimental results show that the proposed model improves upon the state of-the-art baselines on four of five datasets, which can show the effectiveness of our model for text classification.     

Meta-transfer-adjustment learning for few-shot learning

Deep neural network models with strong feature extraction capacity are prone to overfitting and fail to adapt quickly to new tasks with few samples. Gradient-based meta-learning approaches can minimize overfitting and adapt to new tasks fast, but they frequently use shallow neural networks with limited feature extraction capacity. We present a simple and effective approach called Meta-Transfer-Adjustment learning (MTA) in this paper, which enables deep neural networks with powerful feature extraction capabilities to be applied to few-shot scenarios while avoiding overfitting and gaining the capacity for quickly adapting to new tasks via training on numerous tasks. Our presented approach is classified into two major parts, the Feature Adjustment (FA) module, and the Task Adjustment (TA) module. The feature adjustment module (FA) helps the model to make better use of the deep network to improve feature extraction, while the task adjustment module (TA) is utilized for further improve the model’s fast response and generalization capabilities. The proposed model delivers good classification results on the benchmark small sample datasets MiniImageNet and Fewshot-CIFAR100, as proved experimentally.     

有监督主题模型的SLDA-TC文本分类新方法

本文提出了一种有监督主题模型的SLDA-TC（Supervised LDA-Text Categorization）文本分类方法,引入主题-类别概率分布参数,识别主题-类别的语义信息;提出SLDA-TC-Gibbs主题采样新方法,对每个词的隐含主题采样,只从该词所在文档的同类其它文档中采样,并给出了理论推导;另外,其主题数只需略大于类别数.实验表明,对比LDA-TC（LDA-Text Categorization）和SVM算法,本方法能提高分类精度和时间性能.     

A Review of FPGA-Based Custom Computing Architecture for Convolutional Neural Network Inference

Convolutional neural network (CNN) has been widely adopted in many tasks. Its inference process is usually applied on edge devices where the computing resources and power consumption are limited. At present, the performance of general processors cannot meet the requirement for CNN models with high computation complexity and large number of pa-rameters. Field-programmable gate array (FPGA)-based custom computing architecture is a promising solution to further enhance the CNN inference performance. The software/hardware co-design can effectively reduce the computing overhead, and improve the inference performance while ensuring accuracy. In this paper, the mainstream methods of CNN structure design, hardware-oriented model compression and FPGA-based custom architecture design are summarized, and the improvement of CNN inference performance is demonstrated through an example. Challenges and possible research directions in the future are concluded to foster research efforts in this domain.