Parallel reduced multi-class contour preserving classification

Multi-class contour preserving classification is a contour conservancy technique that synthesizes two types of vectors; fundamental multi-class outpost vectors (FMCOVs) and additional multi-class outpost vectors (AMCOVs), at the judging border between classes of data to improve the classification accuracy of the feed-forward neural network. However, the number of both new vectors is tremendous, resulting in a significantly prolonged training time. Reduced multi-class contour preserving classification provides three practical methods to lessen the number of FMCOVs and AMCOVs. Nevertheless, the three reduced multi-class outpost vector methods are serial and therefore have limited applicability on modern machines with multiple CPU cores or processors. This paper presents the methodologies and the frameworks of the three parallel reduced multi-class outpost vector methods that can effectively utilize thread-level parallelism and process-level parallelism to (1) substantially lessen the number of FMCOVs and AMCOVs, (2) efficiently increase the speedups in execution times to be proportional to the number of available CPU cores or processors, and (3) significantly increase the classification performance (accuracy, precision, recall, and F1 score) of the feed-forward neural network. The experiments carried out on the balanced and imbalanced real-world multi-class data sets downloaded from the UCI machine learning repository confirmed the reduction performance, the speedups, and the classification performance aforementioned.     

A genetically optimized neural network model for multi-class classification

Multi-class classification is one of the major challenges in real world application. Classification algorithms are generally binary in nature and must be extended for multi-class problems. Therefore, in this paper, we proposed an enhanced Genetically Optimized Neural Network (GONN) algorithm, for solving multi-class classification problems. We used a multi-tree GONN representation which integrates multiple GONN trees; each individual is a single GONN classifier. Thus enhanced classifier is an integrated version of individual GONN classifiers for all classes. The integrated version of classifiers is evolved genetically to optimize its architecture for multi-class classification. To demonstrate our results, we had taken seven datasets from UCI Machine Learning repository and compared the classification accuracy and training time of enhanced GONN with classical Koza’s model and classical Back propagation model. Our algorithm gives better classification accuracy of almost 5% and 8% than Koza’s model and Back propagation model respectively even for complex and real multi-class data in lesser amount of time. This enhanced GONN algorithm produces better results than popular classification algorithms like Genetic Algorithm, Support Vector Machine and Neural Network which makes it a good alternative to the well-known machine learning methods for solving multi-class classification problems. Even for datasets containing noise and complex features, the results produced by enhanced GONN is much better than other machine learning algorithms. The proposed enhanced GONN can be applied to expert and intelligent systems for effectively classifying large, complex and noisy real time multi-class data.     

A class-modular feedforward neural network for handwriting recognition

Since the conventional feedforward neural networks for character recognition have been designed to classify a large number of classes with one large network structure, inevitably it poses the very complex problem of determining the optimal decision boundaries for all the classes involved in a high-dimensional feature space. Limitations also exist in several aspects of the training and recognition processes. This paper introduces the class modularity concept to the feedforward neural network classifier to overcome such limitations. In the class-modular concept, the original K-classification problem is decomposed into K 2-classification subproblems. A modular architecture is adopted which consists of K subnetworks, each responsible for discriminating a class from the other K−1 classes. The primary purpose of this paper is to prove the effectiveness of class-modular neural networks in terms of their convergence and recognition power. Several cases have been studied, including the recognition of handwritten numerals (10 classes), English capital letters (26 classes), touching numeral pairs (100 classes), and Korean characters in postal addresses (352 classes). The test results confirmed the superiority of the class-modular neural network and the interesting aspects on further investigations of the class modularity paradigm.     

Advanced classification approach for neuronal phoneme recognition system based on efficient constructive training algorithm

This paper introduces a neural network optimization procedure allowing the generation of multilayer perceptron (MLP) network topologies with few connections, low complexity and high classification performance for phoneme’s recognition. An efficient constructive algorithm with incremental training using a new proposed Frame by Frame Neural Networks (FFNN) classification approach for automatic phoneme recognition is thus proposed. It is based on a novel recruiting hidden neuron’s procedure for a single hidden-layer. After an initializing phase started with initial small number of hidden neurons, this algorithm allows the Neural Networks (NNs) to adjust automatically its parameters during the training phase. The modular FFNN classification method is then constructed and tested to recognize 5 broad phonetic classes extracted from the TIMIT database. In order to take into account the speech variability related to the coarticulation effect, a Context Window of Three Successive Frame’s (CWTSF) analysis is applied. Although, an important reduction of the computational training time is observed, this technique penalized the overall Phone Recognition Rate (PRR) and increased the complexity of the recognition system. To alleviate these limitations, two feature dimensionality reduction techniques respectively based on Principal Component Analysis (PCA) and Self Organizing Maps (SOM) are investigated. It is observed an important improvement in the performance of the recognition system when the PCA technique is applied. Optimal neuronal phone recognition architecture is finally derived according to the following criteria: best PRR, minimum computational training time and complexity of the BPNN architecture.     

一种大类别数分类的神经网络方法

神经网络是一种普遍使用的分类方法。当类别数目较大时,神经网络结构复杂、训练时间激增、分类性能下降。针对这些问题,基于Ｎ分类问题的两种类方法和树型分类器结构,对两分类子网络集进行排序,中给出了一种大类别分类的神经网络阵一结构和快速搜索方法并重点分析了网络阵列的分类性能。理论分析表明,使用网络阵列方法可降低平均分类错误率。该方法还使得网络结构简单灵活,易于扩充,网络的训练时间缩短,仿真实验表明,该方     

IIRNet: A lightweight deep neural network using intensely inverted residuals for image recognition

Deep neural networks have achieved great success in many tasks of pattern recognition. However, large model size and high cost in computation limit their applications in resource-limited systems. In this paper, our focus is to design a lightweight and efficient convolutional neural network architecture by directly training the compact network for image recognition. To achieve a good balance among classification accuracy, model size, and computation complexity, we propose a lightweight convolutional neural network architecture named IIRNet for resource-limited systems. The new architecture is built based on Intensely Inverted Residual block (IIR block) to decrease the redundancy of the convolutional blocks. By utilizing two new operations, intensely inverted residual and multi-scale low-redundancy convolutions, IIR block greatly reduces its model size and computational costs while matches the classification accuracy of the state-of-the-art networks. Experiments on CIFAR-10, CIFAR-100, and ImageNet datasets demonstrate the superior performance of IIRNet on the trade-offs among classification accuracy, computation complexity, and model size, compared to the mainstream compact network architectures.     

Neural Networks with Image Recognition by Pairs

Neural networks based on metric recognition methods have a strictly determined architecture. Number of neurons, connections, as well as weights and thresholds values are calculated analytically, based on the initial conditions of tasks: number of recognizable classes, number of samples, metric expressions used. This paper discusses the possibility of transforming these networks in order to apply classical learning algorithms to them without using analytical expressions that calculate weight values. In the received network, training is carried out by recognizing images in pairs. This approach simplifies the learning process and easily allows to expand the neural network by adding new images to the recognition task. The advantages of these networks, including such as: (1) network architecture simplicity and transparency; (2) training simplicity and reliability; (3) the possibility of using a large number of images in the recognition problem using a neural network; (4) a consistent increase in the number of recognizable classes without changing the previous values of weights and thresholds.     

Semi-supervised multi-class Adaboost by exploiting unlabeled data

Semi-supervised learning has attracted much attention in pattern recognition and machine learning. Most semi-supervised learning algorithms are proposed for binary classification, and then extended to multi-class cases by using approaches such as one-against-the-rest. In this work, we propose a semi-supervised learning method by using the multi-class boosting, which can directly classify the multi-class data and achieve high classification accuracy by exploiting the unlabeled data. There are two distinct features in our proposed semi-supervised learning approach: (1) handling multi-class cases directly without reducing them to multiple two-class problems, and (2) the classification accuracy of each base classifier requiring only at least 1/K or better than 1/K (K is the number of classes). Experimental results show that the proposed method is effective based on the testing of 21 UCI benchmark data sets.     

A review and analysis of backpropagation neural networks for classification of remotely-sensed multi-spectral imagery

A literature survey and analysis of the use of neural networks for the classification of remotely-sensed multi-spectral imagery is presented. As part of a brief mathematical review, the backpropagation algorithm, which is the most common method of training multi-layer networks, is discussed with an emphasis on its application to pattern recognition. The analysis is divided into five aspects of neural network classification: (1) input data preprocessing, structure, and encoding, (2) output encoding and extraction of classes, (3) network architecture, (4) training algorithms, and (5) comparisons to conventional classifiers. The advantages of the neural network method over traditional classifiers are its nonparametric nature, arbitrary decision boundary capabilities, easy adaptation to different types of data and input structures, fuzzy output values that can enhance classification, and good generalization for use with multiple images. The disadvantages of the method are slow training time, inconsistent results due to random initial weights, and the requirement of obscure initialization values (e.g., learning rate and hidden layer size). Possible techniques for ameliorating these problems are discussed. It is concluded that, although the neural network method has several unique capabilities, it will become a useful tool in remote sensing only if it is made faster, more predictable, and easier to use.     

Conception of complex probabilistic neural network system for classification of partial discharge patterns using multifarious inputs

Pattern recognition has a long history within electrical engineering but has recently become much more widespread as the automated capture of signal and images has been cheaper. Very many of the application of neural networks are to classification, and so are within the field of pattern recognition and classification. In this paper, we explore how probabilistic neural networks fit into the earlier framework of pattern recognition of partial discharge patterns since the PD patterns are an important tool for diagnosis of HV insulation systems. Skilled humans can identify the possible insulation defects in various representations of partial discharge (PD) data. One of the most widely used representation is phase resolved PD (PRPD) patterns. Also this paper describes a method for the automated recognition of PRPD patterns using a novel complex probabilistic neural network system for the actual classification task. The efficacy of composite neural network developed using probabilistic neural network is examined.     

一种新的基于构造型RBF神经元网络分类算法

依据RBF神经元模型的几何解释,提出一种新的构造型神经网络分类算法.首先从样本数据本身入手,通过引入一个密度估计函数来对样本数据进行聚类分析;然后在特征空间里构造超球面,以逼近样本点分布的几何轮廓,从而将神经网络训练问题转化为点集＂包含＂问题.该算法有效克服了传统神经网络训练时间长、学习复杂的缺陷,同时也考虑了神经网络规模的优化问题.实验证明了该算法的有效性.     

Multiple criteria mathematical programming for multi-class classification and application in network intrusion detection

Multi-class classification problems are harder to solve and less studied than binary classification problems. The goal of this paper is to present a multi-criteria mathematical programming (MCMP) model for multi-class classification. Furthermore, we introduce the concept of e-support vector to facilitate computation of large-scale applications. Instead of finding the optimal solution for a convex mathematical programming problem, the computation of optimal solution for the model requires only matrix computation. Using two network intrusion datasets, we demonstrate that the proposed model can achieve both high classification accuracies and low false alarm rates for multi-class network intrusion classification.     

基于注意力机制和图卷积的小样本分类网络

深度神经网络在有着大量标注数据的图像识别任务上已经占据了统治地位,但是在只有少量标注数据的数据集上训练一个好的网络仍然是一个据有挑战性的任务.如何从有限的标注数据中学习已经成为了一个有着很多应用场景的热点问题.目前有很多解决小样本分类任务的方法,但是仍然存在识别准确率低的问题,根本原因是在小样本学习中,神经网络只能接收少量有标签的数据,导致神经网络不能获取足够的用来识别的信息.因此,提出了一种基于注意力机制和图卷积网络的小样本分类模型.这个模型不仅能够更好地提取特征,而且能够充分利用提取的特征对目标图像进行分类.通过注意力机制,能够指导神经网络关注更有用的信息,而图卷积使得网络能够利用支撑集中其他类别的信息做出更准确的判断.经过大量的实验,证明了提出的模型在Omniglot数据集和mini-ImageNet数据集上的分类准确率都超过了基于传统神经网络的关系网络.     

Rotation, scale and translation invariant handwritten Devanagari numeral character recognition using general fuzzy neural network

In this paper a general fuzzy hyperline segment neural network is proposed [P.M. Patil, Pattern classification and clustering using fuzzy neural networks, Ph.D. Thesis, SRTMU, Nanded, India, January 2003]. It combines supervised and unsupervised learning in a single algorithm so that it can be used for pure classification, pure clustering and hybrid classification/clustering. The method is applied to handwritten Devanagari numeral character recognition and also to the Fisher Iris database. High recognition rates are achieved with less training and recall time per pattern. The algorithm is rotation, scale and translation invariant. The recognition rate with ring data features is found to be 99.5%.     

遥感图像飞机目标分类的卷积神经网络方法

目的 遥感图像飞机目标分类,利用可见光遥感图像对飞机类型进行有效区分,对提供军事作战信息有重要意义。针对该问题,目前存在一些传统机器学习方法,但这些方法需人工提取特征,且难以适应真实遥感图像的复杂背景。近年来,深度卷积神经网络方法兴起,网络能自动学习图像特征且泛化能力强,在计算机视觉各领域应用广泛。但深度卷积神经网络在遥感图像飞机分类问题上应用少见。本文旨在将深度卷积神经网络应用于遥感图像飞机目标分类问题。方法 在缺乏公开数据集的情况下,收集了真实可见光遥感图像中的8种飞机数据,按大致4∶1的比例分为训练集和测试集,并对训练集进行合理扩充。然后针对遥感图像与飞机分类的特殊性,结合深度学习卷积神经网络相关理论,有的放矢地设计了一个5层卷积神经网络。结果 首先,在逐步扩充的训练集上分别训练该卷积神经网络,并分别用同一测试集进行测试,实验表明训练集扩充有利于网络训练,测试准确率从72.4%提升至97.2%。在扩充后训练集上,分别对经典传统机器学习方法、经典卷积神经网络LeNet-5和本文设计的卷积神经网络进行训练,并在同一测试集上测试,实验表明该卷积神经网络的分类准确率高于其他两种方法,最终能在测试集上达到97.2%的准确率,其余两者准确率分别为82.3%、88.7%。结论 在少见使用深度卷积神经网络的遥感图像飞机目标分类问题上,本文设计了一个5层卷积神经网络加以应用。实验结果表明,该网络能适应图像场景,自动学习特征,分类效果良好。     

基于支持向量机的人脸识别方法

1.引言人脸是人类视觉中的常见模式,人脸识别在安全验证系统、公安(犯罪识别等)、医学、视频会议、交通量控制等方面有着广阔的应用前景。现有的基于生物特征的识别技术,包括语音识别、虹膜识别、指纹识别等,都已用于商业应用。然而最吸引人的还是人脸识别,因为从人机交互的方式来看,人脸识别更符合人们的理想。虽然人能毫不费力地识别出人脸及其表情,但人脸的机器自动识别仍然是一个具挑战性的研究领域。由于人脸结构的复杂性以及人脸表情的多样性、成像过     

Principal feature classification

The concept, structures, and algorithms of principal feature classification (PFC) are presented in this paper. PFC is intended to solve complex classification problems with large data sets. A PFC network is designed by sequentially finding principal features and removing training data which has already been correctly classified. PFC combines advantages of statistical pattern recognition, decision trees, and artificial neural networks (ANNs) and provides fast learning with good performance and a simple network structure. For the real-world applications of this paper, PFC provides better performance than conventional statistical pattern recognition, avoids the long training times of backpropagation and other gradient-descent algorithms for ANNs, and provides a low-complexity structure for realization.     

Hardness results for neural network approximation problems

We consider the problem of efficiently learning in two-layer neural networks. We investigate the computational complexity of agnostically learning with simple families of neural networks as the hypothesis classes. We show that it is NP-hard to find a linear threshold network of a fixed size that approximately minimizes the proportion of misclassified examples in a training set, even if there is a network that correctly classifies all of the training examples. In particular, for a training set that is correctly classified by some two-layer linear threshold network with k hidden units, it is NP-hard to find such a network that makes mistakes on a proportion smaller than c/k² of the examples, for some constant c. We prove a similar result for the problem of approximately minimizing the quadratic loss of a two-layer network with a sigmoid output unit.     

不平衡训练数据下的基于深度学习的文本分类

近几年来,随着词向量和各种神经网络模型在自然语言处理上的成功应用,基于神经网络的文本分类方法开始成为研究主流.但是当不同类别的训练数据不均衡时,训练得到的神经网络模型会由多数类所主导,分类结果往往倾向多数类,极大彩响了分类效果.针对这种情况,本文在卷积神经网络训练过程中,损失函数引入类别标签权重,强化少数类对模型参数的影响.在复旦大学文本分类数据集上进行测试,实验表明本文提出的方法相比于基线系统宏平均F1值提高了4.49%,较好地解决数据不平衡分类问题.     

基于时域特征提取的围栏入侵模式分类方法

基于无线传感网的防入侵应用领域中行为分类问题,提出一种基于时域特征提取的电子围栏入侵检测及异常入侵模式识别系统。由于频域处理方法计算量大、复杂度高、传感器采样率高,为减轻系统的传输负担并减少时延,首先对原始信号预处理提取时域特征,然后通过一个三层的BP神经网络对目标事件进行分类,最后对比了多种典型的分类器方法的准确率。仿真结果表明：相比于频域处理方法,该方法复杂度低、易于实现,多种分类器准确率达86%以上,其中BP神经网络测试集的准确率能够达到94%,并且训练集和测试集的准确率偏差较小。