Training pattern replication and weighted class allocation in artificial neural network classification

In some image classifications the importance of classes varies, and it is desirable to weight allocation to selected classes. Often the desire is to weight allocation in favour of classes that are abundant in the area represented by an image at the expense of the less abundant classes. If there is prior knowledge on the distribution of class occurrence, this weighting can be achieved with widely used statistical classifiers by setting appropriate a prioriprobabilities of class membership. With an artificial neural network, the incorporation of prior knowledge is more problematic. An approach to weight class allocation in an artificial neural network classification by replicating selected training patterns is presented. This investigation focuses on a series of classifications in which some classes were more abundant than others, but the same number of training cases were available for each class. By replicating the training patterns of abundant classes the representation of the abundant classes in the training set is increased, reflecting more closely the relative abundance of the classes in an image. Significant increases in classification accuracy were obtained by replicating the training patterns of abundant classes. Furthermore, in comparison against a discriminant analysis for the classification of synthetic aperture radar imagery, the results showed that training pattern replication could be used to weight class allocation with an effect similar to that of incorporating a prioriprobabilities of class membership into the discriminant analysis, and resulted in a significant 20.88%, increase in classification accuracy. This increase in classification accuracy was obtained without any new information, but was the result of making fuller use of what was available.     

Using discriminant analysis for multi-class classification: an experimental investigation

Many supervised machine learning tasks can be cast as multi-class classification problems. Support vector machines (SVMs) excel at binary classification problems, but the elegant theory behind large-margin hyperplane cannot be easily extended to their multi-class counterparts. On the other hand, it was shown that the decision hyperplanes for binary classification obtained by SVMs are equivalent to the solutions obtained by Fisher's linear discriminant on the set of support vectors. Discriminant analysis approaches are well known to learn discriminative feature transformations in the statistical pattern recognition literature and can be easily extend to multi-class cases. The use of discriminant analysis, however, has not been fully experimented in the data mining literature. In this paper, we explore the use of discriminant analysis for multi-class classification problems. We evaluate the performance of discriminant analysis on a large collection of benchmark datasets and investigate its usage in text categorization. Our experiments suggest that discriminant analysis provides a fast, efficient yet accurate alternative for general multi-class classification problems. Tao Li is currently an assistant professor in the School of Computer Science at Florida International University. He received his Ph.D. degree in Computer Science from University of Rochester in 2004. His primary research interests are: data mining, machine learning, bioinformatics, and music information retrieval. Shenghuo Zhu is currently a researcher in NEC Laboratories America, Inc. He received his B.E. from Zhejiang University in 1994, B.E. from Tsinghua University in 1997, and Ph.D degree in Computer Science from University of Rochester in 2003. His primary research interests include information retrieval, machine learning, and data mining. Mitsunori Ogihara received a Ph.D. in Information Sciences at Tokyo Institute of Technology in 1993. He is currently Professor and Chair of the Department of Computer Science at the University of Rochester. His primary research interests are data mining, computational complexity, and molecular computation.     

An adaptive PNN-DS approach to classification using multi-sensor information fusion

In this paper, an adaptive neural network approach to classification which combines modified probabilistic neural network and D-S evidence theory (PNN-DS) is proposed. It attempts to deal with the drawbacks of information uncertainty and imprecision using single classification algorithm. This PNN-DS approach firstly adopts a modified PNN to obtain posteriori probabilities and make a primary classification decision in feature-level fusion. Then posteriori probabilities are transformed to masses noting the evidence of the D-S evidential theory. Finally advanced D-S evidential theory is utilized to gain more accurate classification results in the last decision-level fusion. In order to implement PNN-DS, covariance matrices are firstly employed in the modified PNN module to replace the singular smoothing factor in the PNN’s kernel function, and linear function is utilized in the pattern of summation layer. Secondly, the whole scheme of the proposed approach is explained in depth. Thirdly, three classification experiments are carried out on the proposed approach and a large amount of comparable analyses are done to demonstrate the effectiveness and robustness of the proposed approach. Experiments reveal that the PNN-DS outperforms BPNN-DS, which provides encouraging results in terms of classification accuracy and the speed of learning convergence.     

多损失融合网络的中国书法字体与风格分类

目的 中国书法博大精深,是中国文化很重要的组成部分。书法字体与风格分类是书法领域的研究热点。目前书法字体和书法风格两个概念混淆,并且书法风格分类准确率不高,针对上述问题,本文将两个概念进行区分,并提出了一个融合多损失的孪生卷积神经网络,能同时解决中国书法字体以及风格分类问题。方法 提出的网络包含两个共享权重的分支,每个分支用于提取输入图像的特征。为了获得不同尺度下的特征表示,将Haar小波分解嵌入到每个网络分支中。与传统孪生神经网络不同的是,将网络的每个分支扩展为一个分类网络。网络训练时融合了两类不同的损失,即对比损失和分类损失,进而从两个角度同时对网络训练进行监督。具体来说,为了使来自同一类的两幅输入图像特征之间的距离尽可能小、使来自不同类的两幅输入图像特征之间的距离尽可能大,网络采用对比损失作为损失函数。此外,为了充分利用每幅输入图像的类别信息,在网络每个分支上采用交叉熵作为分类损失。结果 实验结果表明,本文方法在两个中国书法字体数据集和两个中国书法风格数据集上的分类准确率分别达到了99.90%、94.09%、99.38%和93.28%,高于对比方法。两种损失起到了良好的互补作用,Haar小波分解的引入在4个数据集上均提升了分类准确率,在风格数据集的提升效果更为明显。结论 本文方法在中国书法字体以及风格分类两个任务中取得了令人满意的效果,为书法领域研究工作提供了新思路。     

Conception of complex probabilistic neural network system for classification of partial discharge patterns using multifarious inputs

Pattern recognition has a long history within electrical engineering but has recently become much more widespread as the automated capture of signal and images has been cheaper. Very many of the application of neural networks are to classification, and so are within the field of pattern recognition and classification. In this paper, we explore how probabilistic neural networks fit into the earlier framework of pattern recognition of partial discharge patterns since the PD patterns are an important tool for diagnosis of HV insulation systems. Skilled humans can identify the possible insulation defects in various representations of partial discharge (PD) data. One of the most widely used representation is phase resolved PD (PRPD) patterns. Also this paper describes a method for the automated recognition of PRPD patterns using a novel complex probabilistic neural network system for the actual classification task. The efficacy of composite neural network developed using probabilistic neural network is examined.     

A new mathematical programming approach to multi-group classification problems

In this paper we introduce a goal programming formulation for the multi-group classification problem. Although a great number of mathematical programming models for two-group classification problems have been proposed in the literature, there are few mathematical programming models for multi-group classification problems. Newly proposed multi-group mathematical programming model is compared with other conventional multi-group methods by using different real data sets taken from the literature and simulation data. A comparative analysis on the real data sets and simulation data shows that our goal programming formulation may suggest efficient alternative to traditional statistical methods and mathematical programming formulations for the multi-group classification problem.     

Some decision network designs for pattern classification

This article is concerned with the problem of labelling an unidentified parameter vector as belonging to one of a number of given classes. A cluster-analytic approach to the design of binary decision trees is discussed, but the major part of the paper is devoted to the construction of binary features and the creation of a binary feature vector as a means of pattern classification. Complete algorithms are described and some worked examples are also presented.     

基于fcmpCNN模型的网络文本情感多分类标注

由于微博等网络文本所含的上下文信息有限,网络文本情感分析更具有挑战性。针对网络文本情感分析,提出了一种基于全卷积—多池化单元的卷积神经网络模型,实现情感多分类标注。无需手动指定多种上下文窗口大小和尽量保留文本的多层次语义,模型通过堆叠多级全卷积—多池化单元,提取出文本特征向量。该文本特征向量包含多个抽象级别、多种上下文窗口大小和不同层次语义的文本特征。模型最后基于此向量计算情感多分类标注。实验表明：模型的网络文本情感多分类标注正确率达到56.3%,与同类模型比较,提高了情感多分类标注的正确率。     

On detecting nonlinear patterns in discriminant problems

We propose a two-stage model for detecting nonlinear patterns in discriminant problems and for solving the problem. The model deploys a Linear Programming Based Discriminator (LPBD) in stage one for treating linear patterns and a Probabilistic Neural Network (PNN) in stage two for handling nonlinear patterns. The LPBD in stage one divides the decision space into a clear zone where observations are (almost) linearly separable and a gray zone where nonlinear patterns are more likely to occur. The PNN in stage two analyzes the gray zone and determines whether a significant nonlinear patterns exist in the observations. Our goal is to avoid using a nonlinear model unless the PNN strongly suggests so to maintain good interpretability and avoid overfitting. Our computational study demonstrates the effectiveness of the two-stage model in both classification accuracy and computational efficiency.     

一种图片分类的序列构造神经网络方法

针对图片数据的分类问题,从动态构造网络的思想出发,讨论了序列构造神经网络的基本方法,并且把其应用于图片分类中,最后给出标准测试集的分类测试结果,并对其进行了比对、分析和讨论,实验结果表明此方法适合多维数据分析,取得了满意的效果。     

基于模糊对向传播网络的水中目标分类识别

把对向传播（CP）网络的竞争层神经元输出函数定义为模糊隶属度函数,将模糊C-均值（FCM）算法和对向传播网络相结合,提出了一种改进的模糊对向传播（MFCP）网络。在MFCP网络中解决了模糊隶属度函数的自动生成问题。对海上实录的三类水中目标辐射噪声进行了调制解调谱（DEMON谱）的线谱和连续谱分析,并进行了对应的特征提取和神经网络分类识别实验,结果证明：MFCP网络的分类能力及对未训练目标的适应性优于CP网络和误差反向传播（BP）网络。     

Restricted Bayesian classification networks

Bayesian networks are graphical models that describe dependency relationships between variables, and are powerful tools for studying probability classifiers. At present, the causal Bayesian network learning method is used in constructing Bayesian network classifiers while the contribution of attribute to class is over-looked. In this paper, a Bayesian network specifically for classification-restricted Bayesian classification networks is proposed. Combining dependency analysis between variables, classification accuracy evaluation criteria and a search algorithm, a learning method for restricted Bayesian classification networks is presented. Experiments and analysis are done using data sets from UCI machine learning repository. The results show that the restricted Bayesian classification network is more accurate than other well-known classifiers.     

卷积神经网络在短文本情感多分类标注应用

情感多分类标注对文本信息的敏感性远高于二分类问题。为了有效利用语义依赖距离和语义多层次进行情感多分类,提出一种多窗口多池化层的卷积神经网络模型。首先使用多窗口的卷积层提取上下文局部语义,然后通过多池化层降低特征维度,同时保留不同层次的语义,由多层次语义构成文本特征向量,最后送入全连接层完成多分类标注。采用斯坦福情感树库数据集验证所提模型的多分类标注效果。实验结果表明,在训练集含短语和未包含短语两种设定下,模型的短文本情感多分类正确率分别达到54.6%和43.5%。     

一种基于信息熵的金融数据神经网络分类方法

讨论一种基于信息熵的神经网络数据分类方法,通过所有神经元的统计权重信息对输入数据进行投票分类.这种多层网络结构以及基于信息量的分割算法,使得它在数据分类问题上比现有的多数神经网络具有更好的表现.其并行的可扩展结构适合硬件实现,能够提高实际运算速度,适合用来处理金融方面高维度、复杂的海量数据问题.     

改进的概率神经网络的语义区域分类

针对彩色图像信息量大,分割效果自适应性差的问题,对图像语义区域的分割精度进行控制,提取图像的纹理特征值,再通过改进后的概率神经网络模型对测试样本做分类测试,达到提高图像语义提取和分类准确性的目的。实验表明,改进后的概率神经网络对彩色图像语义区域分类的正确性由原先的70%提高到90%,具有较好的分类效果。     

量型图像分类神经网络改进研究

当前普遍使用的轻量型神经网络仍然存在计算量与参数量过大的问题,导致算力较低的廉价移动设备无法快速完成图像分类任务.针对此问题提出了一种更适合于应用在算力较低的廉价移动设备上的轻量型神经网络,引入了代价较小的线性操作与特征图合并操作用于减少神经网络的计算量与参数量,还引入了改进的残差结构、注意力机制和标签平滑技术用于提高结果判断的准确率.基于PD-38数据集的实验表明,该神经网络相比传统的轻量型神经网络使用较小的计算量与参数量可以达到较高的分类准确率.在公共数据集CIFAR-10上的实验进一步表明该神经网络具有通用性.     

基于最优权重的神经网络集成文本分类研究

将神经网络集成思想引入 Web文本分类领域 ,提出了利用最小估计误差策略进行最优加权网络集成的方案。具体做法是根据各网络的分类性能、各网络同其他网络的相关程度给每个网络的后验概率估计赋予不同的权值 ,通过加权平均提高后验概率估计的准确程度 ,进而提高分类率。英文数据库的实验结果表明 ,与经典的 Bayes模型、 kNN模型相比 ,该模型具有更高的分类精度与更快的分类速度。     

基于异构动态图模型的社交网络节点分类方法

在机器学习领域,与传统的神经网络相比,图神经网络在社交推荐等任务中发挥着越来越重要的作用,但是目前工作中大多数都使用静态图.针对现有静态图神经网络方法难以考虑社交用户动态特性的问题,通过引入动态图模型提出了一种基于异构动态图模型的社交网络节点分类方法.该方法在动态图建模的基础上,通过基于点边交互的节点特征更新机制和基于循环神经网络的时序聚合方法,实现了高效的动态社交网络节点分类.在多个真实数据集上的实验结果表明,提出方法在动态社交网络数据的节点分类方面有较好的效果,对比静态图和动态图的基准方法有显著的提升.     

Self-supervised learning based on discriminative nonlinear features for image classification

For learning-based tasks such as image classification, the feature dimension is usually very high. The learning is afflicted by the curse of dimensionality as the search space grows exponentially with the dimension. Discriminant-EM (DEM) proposed a framework by applying self-supervised learning in a discriminating subspace. This paper extends the linear DEM to a nonlinear kernel algorithm, Kernel DEM (KDEM), and evaluates KDEM extensively on benchmark image databases and synthetic data. Various comparisons with other state-of-the-art learning techniques are investigated for several tasks of image classification. Extensive results show the effectiveness of our approach.     

Subclass Discriminant Analysis of morphological and textural features for HEp-2 staining pattern classification

Classifying HEp-2 fluorescence patterns in Indirect Immunofluorescence (IIF) HEp-2 cell imaging is important for the differential diagnosis of autoimmune diseases. The current technique, based on human visual inspection, is time-consuming, subjective and dependent on the operator's experience. Automating this process may be a solution to these limitations, making IIF faster and more reliable. This work proposes a classification approach based on Subclass Discriminant Analysis (SDA), a dimensionality reduction technique that provides an effective representation of the cells in the feature space, suitably coping with the high within-class variance typical of HEp-2 cell patterns. In order to generate an adequate characterization of the fluorescence patterns, we investigate the individual and combined contributions of several image attributes, showing that the integration of morphological, global and local textural features is the most suited for this purpose. The proposed approach provides an accuracy of the staining pattern classification of about 90%.