Rough set based hybrid algorithm for text classification

Automatic classification of text documents, one of essential techniques for Web mining, has always been a hot topic due to the explosive growth of digital documents available on-line. In text classification community, k-nearest neighbor (kNN) is a simple and yet effective classifier. However, as being a lazy learning method without premodelling, kNN has a high cost to classify new documents when training set is large. Rocchio algorithm is another well-known and widely used technique for text classification. One drawback of the Rocchio classifier is that it restricts the hypothesis space to the set of linear separable hyperplane regions. When the data does not fit its underlying assumption well, Rocchio classifier suffers. In this paper, a hybrid algorithm based on variable precision rough set is proposed to combine the strength of both kNN and Rocchio techniques and overcome their weaknesses. An experimental evaluation of different methods is carried out on two common text corpora, i.e., the Reuters-21578 collection and the 20-newsgroup collection. The experimental results indicate that the novel algorithm achieves significant performance improvement.     

基于深度学习的中文文本分类方法

为提升中文文本的分类效率,改善梯度消失、梯度爆炸以及语义信息提取不准确等问题,提出基于深度学习的中文文本分类模型ResCNN-LSTMA.对残差网络和卷积神经网络的组合方式进行研究,发现二者组合能够避免梯度消失和梯度爆炸的情况.分析注意力机制与长短期记忆网络组合的形式对文本分类的效果,二者组合能够在保留上下文语义信息的...     

一种基于Canopy和粗糙集的CRS-KNN文本分类算法

针对KNN算法的分类效率随着训练集规模和特征维数的增加而逐渐降低的问题,提出了一种基于Canopy和粗糙集的CRS-KNN（Canopy Rough Set-KNN）文本分类算法。算法首先将待处理的文本数据通过Canopy进行聚类,然后对得到的每个类簇运用粗糙集理论进行上、下近似分割,对于分割得到的下近似区域无需再进行分类,而通过上、下近似作差所得的边界区域数据需要通过KNN算法确定其最终的类别。实验结果表明,该算法降低了KNN算法的数据计算规模,提高了分类效率。同时与传统的KNN算法和基于聚类改进的KNN文本分类算法相比,准确率、召回率和[F1]值都得到了一定的提高。     

融合后验概率校准训练的文本分类算法

用于文本表示的预训练语言模型在各种文本分类任务上实现了较高的准确率,但仍然存在以下问题：一方面,预训练语言模型在计算出所有类别的后验概率后选择后验概率最大的类别作为其最终分类结果,然而在很多场景下,后验概率的质量能比分类结果提供更多的可靠信息;另一方面,预训练语言模型的分类器在为语义相似的文本分配不同标签时会出现性能下降的情况。针对上述两个问题,提出一种后验概率校准结合负例监督的模型PosCal-negative。该模型端到端地在训练过程中动态地对预测概率和经验后验概率之间的差异进行惩罚,并在训练过程中利用带有不同标签的文本来实现对编码器的负例监督,从而为每个类别生成不同的特征向量表示。实验结果表明：PosCal-negative模型在两个中文母婴护理文本分类数据集MATINF-C-AGE和MATINF-C-TOPIC的分类准确率分别达到了91.55%和69.19%,相比ERNIE模型分别提高了1.13个百分点和2.53个百分点。     

基于选择迁移的baggi ng文本分类算法

针对目标域训练样本数量较少无法建立优质分类模型的问题,提出一种在迁移框架下基于集成bagging算法的跨领域分类方法。引入源域的数据并对其进行筛选,对混合数据集进行学习,建立基于集成bagging算法的分类模型,投票得出预测结果。仿真对比结果表明,采用基于贝叶斯个体分类器的集成bagging算法能够优化源域的迁移,提升目标域的分类准确率及泛化性能。分析源域的噪音数据数量,其结果表明,该算法可以部分规避负迁移。     

Few-shot learning for short text classification

Multimedia Tools and Applications - Due to the limited length and freely constructed sentence structures, it is a difficult classification task for short text classification. In this paper, a short...     

结合改进主动学习的SVD-CNN弹幕文本分类算法

为解决传统卷积神经网络（CNN）模型使用池化层进行文本特征降维会损失较多文本语义信息的问题,提出一种基于奇异值分解（SVD）算法的卷积神经网络模型（SVD-CNN）。首先,采用改进的基于密度中心点采样的主动学习算法（DBC-AL）选择对分类模型贡献率较高的样本进行标注,以低标注代价获得高质量模型训练集;然后,结合SVD算法建立SVD-CNN弹幕文本分类模型,使用奇异值分解的方法代替传统CNN模型池化层进行特征提取和降维,并在此基础上完成弹幕文本分类任务;最后,使用改进的梯度下降算法（PSGD）对模型参数进行优化。为了验证改进算法的有效性,使用多种弹幕数据样本集,对提出的模型与常用的文本分类模型进行对比实验。实验结果表明,改进的算法能够更好地保留文本语义特征,保证训练过程的稳定性并提高了模型的收敛速度,在不同的弹幕文本上较传统算法具有更好的分类性能。     

A Two-Stage Machine learning approach for temporally-robust text classification

One of the most relevant research topics in Information Retrieval is Automatic Document Classification (ADC). Several ADC algorithms have been proposed in the literature. However, the majority of these algorithms assume that the underlying data distribution does not change over time. Previous work has demonstrated evidence of the negative impact of three main temporal effects in representative datasets textual datasets, reflected by variations observed over time in the class distribution, in the pairwise class similarities and in the relationships between terms and classes [1]. In order to minimize the impact of temporal effects in ADC algorithms, we have previously introduced the notion of a temporal weighting function (TWF), which reflects the varying nature of textual datasets. We have also proposed a procedure to derive the TWF’s expression and parameters. However, the derivation of the TWF requires the running of explicit and complex statistical tests, which are very cumbersome or can not even be run in several cases. In this article, we propose a machine learning methodology to automatically learn the TWF without the need to perform any statistical tests. We also propose new strategies to incorporate the TWF into ADC algorithms, which we call temporally-aware classifiers. Experiments showed that the fully-automated temporally-aware classifiers achieved significant gains (up to 17%) when compared to their non-temporal counterparts, even outperforming some state-of-the-art algorithms (e.g., SVM) in most cases, with large reductions in execution time.     

面向文本分类的矩阵投影算法

对文本分类中降维技术、提高分类精度和效率的方法进行了研究,提出了一种基于矩阵投影运算的新型文本分类算法——Matrix Projection（MP）分类算法。矩阵运算将训练样例中表示文本特征的三维空间投影到二维空间上,得到归一化向量,有效地达到了降维与精确计算特征项权重的目的。与其他多种文本分类算法对比实验表明,MP算法的分类精度和时间性能都有明显提高,在两套数据集上的宏平均F1值分别达到92.29%和96.03%。     

基于余弦距离度量学习的伪K近邻文本分类算法

距离度量学习在分类领域有着广泛的应用,将其应用到文本分类时,由于一般采用的向量空间模型(VSM)中的TF*IDF算法在对文本向量表达时向量均是维度相同并且归一化的,这就导致传统距离度量学习过程中采用的欧式距离作为相似度判别标准在文本分类领域往往无法取得预期的效果,在距离度量学习中的LMNN算法的启发下提出一种余弦距离度量学习算法,使其适应于文本分类领域,称之为CS-LMNN.考虑到文本分类领域中样本类偏斜情况比较普遍,提出采用一种伪K近邻分类算法与CS-LMNN结合实现文本分类,该算法首先利用CS-LMNN算法对训练数据进行距离度量学习,根据训练结果对测试数据使用伪K近邻分类算法进行分类,实验结果表明,该算法可以有效的提高分类精度.     

文本分类中训练集相关数量指标的影响研究

针对训练集对分类性能的影响,从训练集的文本数、类别数以及特征项数这三项数量指标出发进行研究。使用多因素方差分析方法及多种语料库定量探讨该三项数量指标对分类性能的影响规律。结果发现特征项数对分类性能的影响在不同的文本数和类别数时是不同的,分类性能受训练集的这三项指标的交互影响,通过对训练集的这三项指标进行优化,提出了从分类算法、特征项选择法以外提高分类性能的途径。在真实数据上的实验结果表明,该方法可有效提高分类性能。     

基于SVM的Web文本快速增量分类算法*

针对基于支持向量机的Web文本分类效率低的问题,提出了一种基于支持向量机Web文本的快速增量分类FVI-SVM算法。算法保留增量训练集中违反KKT条件的Web文本特征向量,克服了Web文本训练集规模巨大,造成支持向量机训练效率低的缺点。算法通过计算支持向量的共享最近邻相似度,去除冗余支持向量,克服了在增量学习过程中不断加入相似文本特征向量而导致增量学习的训练时间消耗加大、分类效率下降的问题。实验结果表明,该方法在保证分类精度的前提下,有效提高了支持向量机的训练效率和分类效率。     

A scalable, incremental learning algorithm for classification problems

In this paper a novel data mining algorithm, Clustering and Classification Algorithm-Supervised (CCA-S), is introduced. CCA-S enables the scalable, incremental learning of a non-hierarchical cluster structure from training data. This cluster structure serves as a function to map the attribute values of new data to the target class of these data, that is, classify new data. CCA-S utilizes both the distance and the target class of training data points to derive the cluster structure. In this paper, we first present problems with many existing data mining algorithms for classification problems, such as decision trees, artificial neural networks, in scalable and incremental learning. We then describe CCA-S and discuss its advantages in scalable, incremental learning. The testing results of applying CCA-S to several common data sets for classification problems are presented. The testing results show that the classification performance of CCA-S is comparable to the other data mining algorithms such as decision trees, artificial neural networks and discriminant analysis.     

多级中文文本情感分类算法研究

针对文本情感分类准确率不高的问题,提出基于CCA-VSM分类器和KFD的多级文本情感分类方法。采用典型相关性分析对文档的权重特征向量和词性特征向量进行降维,在约简向量集上构建向量空间模型,根据模型之间的差异度设计VSM分类器,筛选出与测试文档差异度较小的R个模型作为核Fisher判别的输入,最终判别出文档的情感观点。实验结果表明:该方法比传统支持向量机有较高的分类准确率和较快的分类速度,权重特征和词性特征对分类准确率的影响较大。     

A fast learning algorithm for training feedforward neural networks

This paper deals with a fast and computationally simple Successive Over-relaxation Resilient Backpropagation (SORRPROP) learning algorithm which has been developed by modifying the Resilient Backpropagation (RPROP) algorithm. It uses latest computed values of weights between the hidden and output layers to update remaining weights. The modification does not add any extra computation in RPROP algorithm and maintains its computational simplicity. Classification and regression simulations examples have been used to compare the performance. From the test results for the examples undertaken it is concluded that SORRPROP has small convergence times and better performance in comparison to other first-order learning algorithms.     

自注意力机制的短文本分类算法

分析目前的短文本分类算法没有综合考虑文本中隐含的依赖关系和局部关键信息这一问题,提出基于自注意力机制(self-attention mechanism)的堆叠双向长短时记忆网络(stack bidirectional long short term memory)模型(简称Att-BLSTMs)。利用stack Bi-LSTMs捕获上下文隐藏依赖关系,优化短文本特征稀疏的问题;利用自注意力机制加大对短文本中局部关键信息的注意力,优化文本表示。在公开AG-news网页新闻的语料和DBpedia分类数据集中,进行丰富的对比实验。实验结果表明,该模型将文本中隐含依赖关系与局部关键信息综合考虑后,有效提高了短文本分类的准确性。     

A supervised learning algorithm for hierarchical classification of fuzzy patterns

The notion of “fuzzy separability” is introduced for fuzzy sets of patterns. A supervised learning algorithm is proposed for estimation of membership functions that yield hierarchical partitioning of the feature space for fuzzy separable pattern classes under confusion. Finally we present a methodology for the design of a classifier composed of hierarchical binary decision trees.     

Knowledge-based system for text classification using ID6NB algorithm

This paper presents a novel algorithm named ID6NB for extending decision tree induced by Quinlan’s non-incremental ID3 algorithm. The presented approach is aimed at suggesting the solutions for few unhandled exceptions of the Decision tree induction algorithms such as (i) the situation in which the majority voting makes incorrect decision (generating two different types of rules for same data), and (ii) in case of dimensionality reduction by decision tree induction algorithms, the determination of appropriate attribute at a node where two or more attributes have equal highest information gain. Exception due to majority voting is handled with the help of Naive Bayes algorithm and also novel solutions are given for dimensionality reduction. As a result, the classification accuracy has drastically improved. An extensive experimental evaluation on a number of real and synthetic databases shows that ID6NB is a state-of-the-art classification algorithm that outperforms well than other methods of decision tree learning.     

支持增量学习的文本单类别分类算法

目前的文本单类别分类算法在进行增量学习时需要进行大量的重复计算,提出了一种新的用于文本的单类别分类算法,在不降低分类效果的同时,有效地减少了加入新样本学习时所需的计算量,从而比较适合于需要进行增量学习的情况。该方法已进行了测试实验,获得了较好的实验结果。