Hybrid neural network classifiers for automatic target detection

Abstract: We describe a one-class classification approach to an automatic target detection problem, which involves distinguishing targets from clutter in diverse environments. We use only target statistics to construct the classifier. The classifier combines conventional and neural network methods. The classifier is a Parzen estimator, which requires storage and recall of all training points. To reduce the size of the training set, we apply two neural network learning algorithms: (1) we use a backpropagation network to approximate the Parzen estimator; (2) we apply the infomax learning principle to compress the size of the training set before constructing the Parzen estimator. We find that the results obtained with the infomax scheme approach those obtained with Parzen alone and are better than those obtained with backpropagation.     

A Feature-Based Serial Approach to Classifier Combination

: A new approach to the serial multi-stage combination of classifiers is proposed. Each classifier in the sequence uses a smaller subset of features than the subsequent classifier. The classification provided by a classifier is rejected only if its decision is below a predefined confidence level. The approach is tested on a two-stage combination of k-Nearest Neighbour classifiers. The features to be used by the first classifier in the combination are selected by two stand-alone algorithms (Relief and Info-Fuzzy Network, or IFN) and a hybrid method, called ‘IFN + Relief’. The feature-based approach is shown empirically to provide a substantial decrease in the computational complexity, while maintaining the accuracy level of a single-stage classifier or even improving it. Received: 24 November 2000, Received in revised form: 30 November 2001, Accepted: 05 June 2002 ID="A1" Correspondence and offprint requests to: M. Last, Department of Information Systems Engineering, Ben-Gurion University of the Negev, Beer-Sheva 84105, Israel. Email: mlast@bgumail.bgu.ac.il     

Work piece recognition based on the permutation neural classifier technique

This article describes a permutation neural classifier technique for the object recognition problem. Our research is aimed to help the automation of micromanufacturing and microassembly processes. In this article, we describe an object recognition system based on permutation of codes and neural classifier technique. This approach is called permutation code neural classifier (PCNC). In this work, we describe our experiments and results applying the PCNC in the recognition of micro work pieces. Two databases with different images were used for the experiments. The authors have published these databases and encourage the community to compare results. The best recognition rate obtained for the PCNC was of 97%.     

Speech recognition using hybrid hidden markov model and NN classifier

This paper discusses the use of an integrated HMM/NN classifier for speech recognition. The proposed classifier combines the time normalization property of the HMM classifier with the superior discriminative ability of the neural net (NN) classifier. Speech signals display a strong time varying characteristic. Although the neural net has been successful in many classification problems, its success (compared to HMM) is secondary to HMM in the field of speech recognition. The main reason is the lack of time normalization characteristics of most neural net structures (time-delay neural net is one notable exception but its structure is very complex). In the proposed integrated hybrid HMM/NN classifier, a left-to-right HMM module is used first to segment the observation sequence of every exemplar into a fixed number of states. Subsequently, all the frames belonging to the same state are replaced by one average frame. Thus, every exemplar, irrespective of its time scale variation, is transformed into a fixed number of frames, i.e., a static pattern. The multilayer perceptron (MLP) neural net is then used as the classifier for these time normalized exemplars. Some experimental results using telephone speech databases are presented to demonstrate the potential of this hybrid integrated classifier.     

Scale‐invariant segmentation of dynamic contrast‐enhanced perfusion MR images with inherent scale selection

Selection of the best set of scales is problematic when developing signal‐driven approaches for pixel‐based image segmentation. Often, different possibly conflicting criteria need to be fulfilled in order to obtain the best trade‐off between uncertainty (variance) and location accuracy. The optimal set of scales depends on several factors: the noise level present in the image material, the prior distribution of the different types of segments, the class‐conditional distributions associated with each type of segment as well as the actual size of the (connected) segments. We analyse, theoretically and through experiments, the possibility of using the overall and class‐conditional error rates as criteria for selecting the optimal sampling of the linear and morphological scale spaces. It is shown that the overall error rate is optimized by taking the prior class distribution in the image material into account. However, a uniform (ignorant) prior distribution ensures constant class‐conditional error rates. Consequently, we advocate for a uniform prior class distribution when an uncommitted, scale‐invariant segmentation approach is desired. Experiments with a neural net classifier developed for segmentation of dynamic magnetic resonance (MR) images, acquired with a paramagnetic tracer, support the theoretical results. Furthermore, the experiments show that the addition of spatial features to the classifier, extracted from the linear or morphological scale spaces, improves the segmentation result compared to a signal‐driven approach based solely on the dynamic MR signal. The segmentation results obtained from the two types of features are compared using two novel quality measures that characterize spatial properties of labelled images. Copyright © 2002 John Wiley & Sons, Ltd.     

应用BP神经网络分类器识别交通标志

介绍了神经网络特性和BP神经网络分类器的一般原理。针对交通标志识别需要处理的信息量大以及受天气道路等外界条件的影响存在噪声干扰的情况，提出了一种应用BP神经网络分类器识别交通标志的方法。识别分为图像数字处理、BP神经网络的训练、测试与对加入噪声图像进行识别3个步骤，经实验取得了良好的识别效果。     

Consistent specification of interface suites in UML

The paper motivates and describes a model oriented approach for consistent specification of interface suites in UML. An interface suite is a coherent collection of interfaces defining interactions that transcend component boundaries. The specification of interface suites contains diagrammatic views and documentation, but it is extended with templates for structured specifications deriving from the ISpec approach. To guarantee that the specification views, documentation and templates are consistent, a specification model has been constructed. The model contains both structural and behavioural information, represented in the form of sequences of carefully designed tuples. The model provides the underlying structure for the tool supporting the design process. The tool directs the designer to specify all elements of the model in a consistent way. The specification is collected both by customized specification templates and by diagrams. The documentation and the diagram elements – both derived from the template information – are automatically generated. This prevents errors and provides specification consistency. Initial submission: 15 February 2002 / Revised submission: 20 September 2002 Published online: 2 December 2002 RID="*" ID="*"Supported by PROGRESS grant EES.5141 and ITEA DESS grant IT990211.     

Handwriting style classification

This paper describes an independent handwriting style classifier that has been designed to select the best recognizer for a given style of writing. For this purpose a definition of handwriting legibility has been defined and a method implemented that can predict this legibility. The technique consists of two phases. In the feature-extraction phase, a set of 36 features is extracted from the image contour. In the classification phase, two nonparametric classification techniques are applied to the extracted features in order to compare their effectiveness in classifying words into legible, illegible, and middle classes. In the first method, a multiple discriminant analysis (MDA) is used to transform the space of extracted features (36 dimensions) into an optimal discriminant space for a nearest mean based classifier. In the second method, a probabilistic neural network (PNN) based on the Bayes strategy and nonparametric estimation of probability density function is used. The experimental results show that the PNN method gives superior classification results when compared with the MDA method. For the legible, illegible, and middle handwriting the method provides 86.5% (legible/illegible), 65.5% (legible/middle), and 90.5% (middle/illegible) correct classification for two classes. For the three-class legibility classification the rate of correct classification is 67.33% using a PNN classifier.Received: 6 September 2002, Accepted: 19 September 2002, Published online: 6 June 2003     

Adaptive resonance neural classifier for identification of gases/odours using an integrated sensor array

A new approach to intelligent gas sensor (IGS) design using a classifier based on adaptive resonance theory (ART) artificial neural network (ANN) is presented. Using published data of sensor arrays fabricated and characterised at our laboratory, we demonstrate excellent gas/odour identification performance of our classifier for a 4-gas, 4-sensor system to identify individual gas/odour. Since the ART neural network is a self-organising classifier trained in the unsupervised mode, it avoids the drawbacks associated with static feedforward neural networks trained with a locally optimal backpropagation-type training algorithms applied by researchers in the recent past. The ART neural network offers easy implementability and real time performance in addition to giving excellent classification accuracy as demonstrated by our experiments.     

基于多神经网络分类器组合的火焰图像分割

单一神经网络分类的性能很大程度上取决于网络参数的选择,设计一个性能最优的神经网络分类器是非常困难的。针对这一问题,本文提出了基于多个BP神经网络分类器组合的回转窑火焰图像分割方法。选取多组不同的训练样本对多个具有不同初始条件的BP网络进行训练,网络收敛后,用于火焰图像的分割,会产生多种分割结果,采用平均值法、投票表决法、最大统计概率法和神经网络4种方法对其进行组合,得到了最科的分割结果。实验结果表明,本文提出的方法具有分割效果好和可靠性高等优点,满了实际使用的要求。     

On an asymptotically optimal adaptive classifier design criterion

A new approach for estimating classification errors is presented. In the model, there are two types of classification error: empirical and generalization error. The first is the error observed over the training samples, and the second is the discrepancy between the error probability and empirical error. In this research, the Vapnik and Chervonenkis dimension (VCdim) is used as a measure for classifier complexity. Based on this complexity measure, an estimate for generalization error is developed. An optimal classifier design criterion (the generalized minimum empirical error criterion (GMEE)) is used. The GMEE criterion consists of two terms: the empirical and the estimate of generalization error. As an application, the criterion is used to design the optimal neural network classifier. A corollary to the Γ optimality of neural-network-based classifiers is proven. Thus, the approach provides a theoretic foundation for the connectionist approach to optimal classifier design. Experimental results to validate this approach     

Theoretical and experimental analysis of a two-stage system forclassification

We consider a popular approach to multicategory classification tasks: a two-stage system based on a first classifier with rejection followed by a nearest-neighbor classifier. Patterns which are not rejected by the first classifier are classified according to its output. Rejected patterns are passed to the nearest-neighbor classifier together with the top-h ranking classes returned by the first classifier. The nearest-neighbor classifier, looking at patterns in the top-h classes, classifies the rejected pattern. An editing strategy for the nearest-neighbor reference database, controlled by the first classifier, is also considered. We analyze this system. Moreover, we formally relate the response time of the system to the rejection rate of the first classifier and to the other system parameters. The error-response time trade-off is also discussed. Finally, we experimentally study two instances of the system applied to the recognition of handwritten digits. In one system, the first classifier is a fuzzy basis functions network, while in the second system it is a feed-forward neural network. Classification results as well as response times for different settings of the system parameters are reported for both systems     

Optimal linear combination of neural networks for improvingclassification performance

This paper presents a new method for linearly combining multiple neural network classifiers based on the statistical pattern recognition theory. In our approach, several neural networks are first selected based on which works best for each class in terms of minimizing classification errors. Then, they are linearly combined to form an ideal classifier that exploits the strengths of the individual classifiers. In this approach, the minimum classification error criterion is utilized to estimate the optimal linear weights. In this formulation, because the classification decision rule is incorporated into the cost function, a more suitable better combination of weights for the classification objective could be obtained. Experimental results using artificial and real data sets show that the proposed method can construct a better combined classifier that outperforms the best single classifier in terms of overall classification errors for test data     

一种基于神经网络覆盖构造法的模糊分类器

首先介绍了一种M-P模型几何表示,以及利用这种几何表示可将神经网络的训练问题转化为点集覆盖问题,并在此基础上分析了神经网络训练的一种几何方法.针对该方法可构造十分复杂的分类边界,但其时间复杂度很高.提出一种将神经网络覆盖算法与模糊集合思想相结合的方法,该分类器可改善训练速度、减少覆盖的球领域数目,即减少神经网络的隐结点数目.同时模糊化方法可方便地为大规模模式识别问题提供多选结果.用700类手写汉字的识别构造一个大规模模式识别问题测试提出的方法,实验结果表明,该方法对于大规模模式识别问题很有潜力.     

Fuzzy min-max neural networks. I. Classification.

A supervised learning neural network classifier that utilizes fuzzy sets as pattern classes is described. Each fuzzy set is an aggregate (union) of fuzzy set hyperboxes. A fuzzy set hyperbox is an n-dimensional box defined by a min point and a max point with a corresponding membership function. The min-max points are determined using the fuzzy min-max learning algorithm, an expansion-contraction process that can learn nonlinear class boundaries in a single pass through the data and provides the ability to incorporate new and refine existing classes without retraining. The use of a fuzzy set approach to pattern classification inherently provides a degree of membership information that is extremely useful in higher-level decision making. The relationship between fuzzy sets and pattern classification is described. The fuzzy min-max classifier neural network implementation is explained, the learning and recall algorithms are outlined, and several examples of operation demonstrate the strong qualities of this new neural network classifier.     

基于MPSO-BP神经网络方法的人体步态识别

为提高人体下肢步态相位识别准确率以实现外骨骼机器人控制,采用一种改进的粒子群优化MPSO-BP神经网络方法识别不同运动模式下的人体步态相位。通过自适应调整学习因子构造MPSO-BP神经网络分类器,以多种传感信息组成的特征向量样本集训练神经网络分类器,用于识别人体下肢在平地行走、上楼梯和起坐三种典型运动模式下的步态相位。实验结果表明,MPSO-BP神经网络分类器能有效识别三种不同运动模式的步态相位,识别准确率均达到96%以上,识别性能优于传统的BP神经网络模型和粒子群优化神经网络模型。     

A binary neural k-nearest neighbour technique

K-Nearest Neighbour (k-NN) is a widely used technique for classifying and clustering data. K-NN is effective but is often criticised for its polynomial run-time growth as k-NN calculates the distance to every other record in the data set for each record in turn. This paper evaluates a novel k-NN classifier with linear growth and faster run-time built from binary neural networks. The binary neural approach uses robust encoding to map standard ordinal, categorical and real-valued data sets onto a binary neural network. The binary neural network uses high speed pattern matching to recall the k-best matches. We compare various configurations of the binary approach to a conventional approach for memory overheads, training speed, retrieval speed and retrieval accuracy. We demonstrate the superior performance with respect to speed and memory requirements of the binary approach compared to the standard approach and we pinpoint the optimal configurations.     

Neural network-based analog fault diagnosis using testability analysis

A fault diagnosis procedure for analog linear circuits is presented. It uses an off-line trained neural network as a classifier. The innovative aspect of the proposed approach is the way the information provided by testability and ambiguity group determination is exploited when choosing the neural network architecture. The effectiveness of the proposed approach is shown by comparing with similar work that has already appeared in the literature.     

PERFORMANCE-DRIVEN AUTONOMOUS DESIGN OF PATTERN-RECOGNITION SYSTEMS

The closed-loop design experiment described in this paper demonstrates a three-phase automated design approach to pattern recognition. The experiment generates morphological feature detectors and then uses a novel application of genetic algorithms to select cooperative sets of features to pass to a neural net classifier. The self-organizing hybrid learning approach embodied in this closed-loop design methodology is complementary to conventional artificial intelligence (AI) expert systems that utilize rule-based approaches and a specific set of design elements. This experiment is part of a study directed to emulating the nondirected processes of biological evolution. The approach we discuss is semiautomatic in that initialization of computer programs requires human experience and expertise to select representations, develop search strategies, choose performance measures, and devise resource-allocation strategies. The hope is that these tasks will become easier with experience and will provide the means to exploit parallel processing without the need to analyze or program an entire design solution.