A self-organizing HCMAC neural-network classifier

This paper presents a self-organizing hierarchical cerebellar model arithmetic computer (HCMAC) neural-network classifier, which contains a self-organizing input space module and an HCMAC neural network. The conventional CMAC can be viewed as a basis function network (BFN) with supervised learning, and performs well in terms of its fast learning speed and local generalization capability for approximating nonlinear functions. However, the conventional CMAC has an enormous memory requirement for resolving high-dimensional classification problems, and its performance heavily depends on the approach of input space quantization. To solve these problems, this paper presents a novel supervised HCMAC neural network capable of resolving high-dimensional classification problems well. Also, in order to reduce what is often trial-and-error parameter searching for constructing memory allocation automatically, proposed herein is a self-organizing input space module that uses Shannon's entropy measure and the golden-section search method to appropriately determine the input space quantization according to the various distributions of training data sets. Experimental results indicate that the self-organizing HCMAC indeed has a fast learning ability and low memory requirement. It is a better performing network than the conventional CMAC for resolving high-dimensional classification problems. Furthermore, the self-organizing HCMAC classifier has a better classification ability than other compared classifiers.     

基于堆叠模型的司法短文本多标签分类

司法文书短文本的语义多样性和特征稀疏性等特点,对短文本多标签分类精度提出了很大的挑战,传统单一模型的分类算法已无法满足业务需求。为此,提出一种融合深度学习与堆叠模型的多标签分类方法。该方法将分类器划分成两个层次,第一层使用BERT、卷积神经网络、门限循环单元等深度学习方法作为基础分类器,每个基础分类器模型通过K折交叉验证得到所有数据的多标签分类概率值,将此概率值数据进行融合形成元数据;第二层使用自定义的深度神经网络作为混合器,以第一层的元数据为输入,通过训练多标签概率矩阵获取模型参数。该方法将强分类器关联在一起,获得比单个分类器更加强大的性能。实验结果表明,深度学习堆叠模型实现了87%左右的短文本分类F1分数,优于BERT、卷积神经网络、循环神经网络及其他单个模型的性能。     

基于Yahoo的信息自动分类器的原理与设计

本文介绍了一种基于Ｙａｈｏｏ层次的自动分类器,此分类器主要是把基于文本数据的机器学习技巧用于Ｙａｈｏｏ的层次结构;讨论了分类文档过程的文档表示、功能选择和学习方法及相关的算法。     

基于Boruta-PSO-SVM的股票收益率研究

针对股票收益率的分类预测研究中支持向量机(SVM)存在的参数选择困难以及分类性能较差的问题,提出了一种基于特征选择(Boruta算法)和粒子群优化(PSO)算法SVM的新算法.通过Boruta算法对训练集进行特征选择,剔除无价值的特征以降低输入维度,同时引入PSO算法优化SVM核函数参数,从而提高SVM的分类性能.实验结果表明:相比决策树、神经网络及极限学习机算法,新算法取得了更高的分类精度,可以有效提高股票收益率的分类预测性能.     

Classifying web documents in a hierarchy of categories: a comprehensive study

Most of the research on text categorization has focused on classifying text documents into a set of categories with no structural relationships among them (flat classification). However, in many information repositories documents are organized in a hierarchy of categories to support a thematic search by browsing topics of interests. The consideration of the hierarchical relationship among categories opens several additional issues in the development of methods for automated document classification. Questions concern the representation of documents, the learning process, the classification process and the evaluation criteria of experimental results. They are systematically investigated in this paper, whose main contribution is a general hierarchical text categorization framework where the hierarchy of categories is involved in all phases of automated document classification, namely feature selection, learning and classification of a new document. An automated threshold determination method for classification scores is embedded in the proposed framework. It can be applied to any classifier that returns a degree of membership of a document to a category. In this work three learning methods are considered for the construction of document classifiers, namely centroid-based, naïve Bayes and SVM. The proposed framework has been implemented in the system WebClassIII and has been tested on three datasets (Yahoo, DMOZ, RCV1) which present a variety of situations in terms of hierarchical structure. Experimental results are reported and several conclusions are drawn on the comparison of the flat vs. the hierarchical approach as well as on the comparison of different hierarchical classifiers. The paper concludes with a review of related work and a discussion of previous findings vs. our findings.     

Two-level hierarchical feature learning for image classification

In some image classification tasks, similarities among different categories are different and the samples are usually misclassified as highly similar categories. To distinguish highly similar categories, more specific features are required so that the classifier can improve the classification performance. In this paper, we propose a novel two-level hierarchical feature learning framework based on the deep convolutional neural network (CNN), which is simple and effective. First, the deep feature extractors of different levels are trained using the transfer learning method that fine-tunes the pre-trained deep CNN model toward the new target dataset. Second, the general feature extracted from all the categories and the specific feature extracted from highly similar categories are fused into a feature vector. Then the final feature representation is fed into a linear classifier. Finally, experiments using the Caltech-256, Oxford Flower-102, and Tasmania Coral Point Count (CPC) datasets demonstrate that the expression ability of the deep features resulting from two-level hierarchical feature learning is powerful. Our proposed method effectively increases the classification accuracy in comparison with flat multiple classification methods.     

一种改进的基于神经网络的文本分类算法*

提出并实现了一种结合前馈型神经网络和K最近邻的文本分类算法。其中,在选取特征项时考虑到Web文本不同标签组所代表的意义和权重有所区别,采用了一种改进的TFIDF特征选择法。最后对设计的分类器进行了开放性测试,实验结果表明该分类器显著地提高了文本分类的查全率和查准率。     

基于分层筛选和动态更新的并行选择集成算法

提出一种选择性集成学习算法,该算法利用多线程并行优化基分类器的参数,通过多层筛选和动态更新筛选信息获取最优的候选基分类器集合,解决了以往在集成学习中选择分类器效率低下的问题。集成分类器采用分解合并的策略进行加权投票,通过使用二分法将大数据集的投票任务递归分解成多个子任务,并行运行子任务后合并投票结果以缩短集成分类器的投票运行时间。实验结果表明, 相对于传统方法, 所提出的算法在平均精度、F1-Measure以及AUC指标上都有着显著提升。     

Growing random forest on deep convolutional neural networks for scene categorization

Breakthrough performances have been achieved in computer vision by utilizing deep neural networks. In this paper we propose to use random forest to classify image representations obtained by concatenating multiple layers of learned features of deep convolutional neural networks for scene classification. Specifically, we first use deep convolutional neural networks pre-trained on the large-scale image database Places to extract features from scene images. Then, we concatenate multiple layers of features of the deep neural networks as image representations. After that, we use random forest as the classifier for scene classification. Moreover, to reduce feature redundancy in image representations we derived a novel feature selection method for selecting features that are suitable for random forest classification. Extensive experiments are conducted on two benchmark datasets, i.e. MIT-Indoor and UIUC-Sports. Obtained results demonstrated the effectiveness of the proposed method. The contributions of the paper are as follows. First, by extracting multiple layers of deep neural networks, we can explore more information of image contents for determining their categories. Second, we proposed a novel feature selection method that can be used to reduce redundancy in features obtained by deep neural networks for classification based on random forest. In particular, since deep learning methods can be used to augment expert systems by having the systems essentially training themselves, and the proposed framework is general, which can be easily extended to other intelligent systems that utilize deep learning methods, the proposed method provide a potential way for improving performances of other expert and intelligent systems.     

DSP-based hierarchical neural network modulation signal classification

This paper discusses a real-time digital signal processor (DSP)-based hierarchical neural network classifier capable of classifying both analog and digital modulation signals. A high-performance DSP processor, namely the TMS320C6701, is utilized to implement different kinds of classifiers including a hierarchical neural network classifier. A total of 31 statistical signal features are extracted and used to classify 11 modulation signals plus white noise. The modulation signals include CW, AM, FM, SSB, FSK2, FSK4, PSK2, PSK4, OOK, QAM16, and QAM32. A classification hierarchy is introduced and the genetic algorithm is employed to obtain the most effective set of features at each level of the hierarchy. The classification results and the number of operations on the DSP processor indicate the effectiveness of the introduced hierarchical neural network classifier in terms of both classification rate and processing time.     

基于动态径向基神经网络的人脸识别算法研究

提出了一个通用而且有效的方法来设计RBF神经网络分类器用于人脸识别。为了避免过拟合和减少计算量,用主元分析法和Fisher线性判别技术来降低维数,以提取人脸特征;利用一个混合的学习算法来训练RBF神经网络,使梯度下降法的搜索空间大大减少;采用一种基于训练样本类别信息的新的聚类算法,所有同类的数据可被聚集在一起,尽量减少不同类数据混杂在一起,同时选取结构尽可能紧凑的RBF神经网络分类器。在ORL数据库上进行了仿真,实验结果表明,该算法具有高效性和有效性。     

融合RoBERTa和特征提取的政务热线工单分类

政务热线承接了海量市民诉求,人工对工单分类耗时费力。现有工单分类方法大多基于机器学习或单一神经网络模型,难以有效理解上下文语义信息,且文本特征提取不全面。针对这一问题,本文提出一种融合RoBERTa和特征提取的政务热线工单分类方法。该方法首先通过基于RoBERTa预训练语言模型的语义编码层获取政务热线工单文本中的语义表征向量,然后通过由CNN-BiGRU-Self-Attention定义的特征提取层获取工单文本的局部特征和全局特征,并对全局特征进行处理以凸显重要性高的语义特征,最后将融合后的特征向量输入分类器来完成工单分类。实验结果表明,相较于其他基线分类方法,本文提出的方法能够取得更好的工单分类效果。     

Multiobjective genetic algorithm partitioning for hierarchicallearning of high-dimensional pattern spaces: alearning-follows-decomposition strategy

We present a novel approach to partitioning pattern spaces using a multiobjective genetic algorithm for identifying (near-)optimal subspaces for hierarchical learning. Our approach of "learning-follows-decomposition" is a generic solution to complex high-dimensional problems where the input space is partitioned prior to the hierarchical neural domain instead of by competitive learning. In this technique, clusters are generated on the basis of fitness of purpose. Results of partitioning pattern spaces are presented. This strategy of preprocessing the data and explicitly optimizing the partitions for subsequent mapping onto a hierarchical classifier is found both to reduce the learning complexity and the classification time with no degradation in overall classification error rate. The classification performance of various algorithms is compared and it is suggested that the neural modules are superior for learning the localized decision surfaces of such partitions and offer better generalization.     

基于 PCA 和 LDA 的方言辨识

针对PCA没有有效利用样本的类别信息而导致方言识别率低的问题,采用PCA和LDA组合方法进行特征提取。首先用PCA对普通话、上海话、广东话和闽南话四种方言进行降维,然后在降维后的空间中用LDA进一步特征提取,最后将该特征向量送入BP神经网络进行辨识。仿真实验结果表明,基于PCA和LDA的方言识别的平均识别率高达85%。     

层次化双注意力神经网络模型的情感分析研究

在篇章级的情感分类中由于篇章级文本较长,特征提取较普通句子级分析相对较难,大多方法使用层次化的模型进行篇章文本的情感分析,但目前的层次化模型多以循环神经网络和注意力机制为主,单一的循环神经网络结构提取的特征不够明显。本文针对篇章级的情感分类任务,提出一种层次化双注意力神经网络模型。首先对卷积神经网络进行改进,构建词注意力卷积神经网络。然后模型从两个层次依次提取篇章特征,第一层次使注意力卷积神经网络发现每个句子中的重要词汇,提取句子的词特征,构建句子特征向量;第二层次以循环神经网络获取整个篇章的语义表示,全局注意力机制发现篇章中每个句子的重要性,分配以不同的权重,最后构建篇章的整体语义表示。在IMDB、YELP 2013、YELP 2014数据集上的实验表明,模型较当前最好的模型更具优越性。     

Evolutionary strategy for classification problems and its application in fault diagnostics

Genetic algorithms (GAs) based evolutionary strategy is proposed for classification problems, which includes two aspects: evolutionary selection of the training samples and input features, and evolutionary construction of the neural network classifier. For the first aspect, the GA based k-means-type algorithm (GKMT) is proposed, which combines GA and k-means-type (KMT) to achieve the optimal selection of the training samples and input features simultaneously. By this algorithm, the “singular” samples will be eliminated according to the classification accuracy and the features that facilitate the classification will be enhanced. On the opposite, the useless features will be suppressed and even eliminated. For the second aspect, the hierarchical evolutionary strategy is proposed for the construction and training of the neural network classifier (HENN). This strategy uses the hierarchical chromosome to encode the structure and parameters of the neural network into control genes and parameter genes respectively, designs and trains the network simultaneously. Finally, the experimental study pertained to the fault diagnostics for the rotor-bearing system is given and the results presented show that the proposed evolutionary strategy for the classification problem is feasible and effective.     

基于深度神经网络的Android恶意软件检测方法

Android 系统正日益面临着恶意软件的攻击威胁。针对支持向量机等传统机器学习方法难以有效进行大样本多分类的恶意软件检测,提出一种基于深度神经网络的Android恶意软件检测与家族分类方法。该方法在全面提取应用组件、Intent Filter、权限、数据流等特征基础上,进行有效的特征选择以降低维度,基于深度神经网络进行面向恶意软件的大样本多分类检测。实验结果表明,该方法能够进行有效检测和分类,良性、恶意二分类精度为 97.73%,家族多分类精度可达到 93.54%,比其他机器学习算法有更好的分类效果。     

基于改进CNN的宫颈细胞自动分类算法

本文采用深度学习算法中的卷积神经网络对细胞图像进行识别, 实现对宫颈细胞图像的自动分类. 首先对宫颈细胞进行预处理, 通过细胞核裁剪解决图像输入尺寸不一的问题, 对图像进行翻转平移, 对数据集进行扩充, 并解决样本量不均衡的问题; 接着选取VGG-16网络进行改进, 使用改进后的VGG-16网络进行特征提取, 以及细胞分类; 并采用迁移学习的方法加载预训练网络参数, 进而加快参数收敛速度, 提高分类准确率; 最终通过对网络的训练, 得到了较好的分类结果, 将分类结果与人工提取特征设计分类器的方法相比, 分类的准确率有所提高, 二分类的准确率达97.3%, 七分类的准确率达89%. 实验结果表明: 卷积神经网络对宫颈细胞图像进行自动分类, 分类准确率相比较人工提取特征分类器效果较好, 且分类结果不受分割图像准确率的影响.     

基于标签语义相似的动态多标签文本分类算法

针对标签随着时间变化的动态多标签文本分类问题,提出了一种基于标签语义相似的动态多标签文本分类算法。该算法在训练阶段,首先按照标签固定训练得到一个基于卷积神经网络的多标签文本分类器,然后以该分类器的倒数第二层的输出为文本的特征向量。由于该特征向量是在有标签训练得到的,因而相对于基于字符串即文本内容而言,该特征向量含有标签语义信息。在测试阶段,将测试文档输入训练阶段的多标签文本分类器获取相应的特征向量,然后计算相似性,同时乘以时间衰减因子修正,使得时间越近的文本具有较高的相似性。最后,采用最近邻算法分类。实验结果表明,该算法在处理动态多标签文本分类问题上具有较优的性能。     

A fast approach for detection of erythemato-squamous diseases based on extreme learning machine with maximum relevance minimum redundancy feature selection

In this paper, a novel hybrid method, which integrates an effective filter maximum relevance minimum redundancy (MRMR) and a fast classifier extreme learning machine (ELM), has been introduced for diagnosing erythemato-squamous (ES) diseases. In the proposed method, MRMR is employed as a feature selection tool for dimensionality reduction in order to further improve the diagnostic accuracy of the ELM classifier. The impact of the type of activation functions, the number of hidden neurons and the size of the feature subsets on the performance of ELM have been investigated in detail. The effectiveness of the proposed method has been rigorously evaluated against the ES disease dataset, a benchmark dataset, from UCI machine learning database in terms of classification accuracy. Experimental results have demonstrated that our method has achieved the best classification accuracy of 98.89% and an average accuracy of 98.55% via 10-fold cross-validation technique. The proposed method might serve as a new candidate of powerful methods for diagnosing ES diseases.