Some decision network designs for pattern classification

This article is concerned with the problem of labelling an unidentified parameter vector as belonging to one of a number of given classes. A cluster-analytic approach to the design of binary decision trees is discussed, but the major part of the paper is devoted to the construction of binary features and the creation of a binary feature vector as a means of pattern classification. Complete algorithms are described and some worked examples are also presented.     

Efficient clustering of radial basis perceptron neural network for pattern recognition

This paper studies how to train a new feed-forward neural network, radial basis perceptron (RBP) neural network, for distinguishing different sets in R^L. RBP neural network is based on radial basis function (RBF) neural network and perceptron neural network. It has two hidden layers where the nodes are not fully connected but use selective connection. A training algorithm corresponding to the structure of RBP network is presented. It adopts the input-output clustering (IOC) method to provide an efficient and powerful procedure for constructing a RBP network that generalizes very well. First, during the learning procedure, RBP neural network adopts IOC method to define the number of units of hidden layers and select centers. Second, the width parameter σ of centers is self-adjustable according to the information included in the learning samples. The effectiveness of this network is illustrated using an example taken from applications for component analysis of civil building materials. Simulation shows that RBP neural network can be used to predict the components of civil building materials successfully and gets good generalization ability.     

A neural network approach to characterize pattern parameters in process control charts

Abnormal patterns on manufacturing process control charts can reveal potential quality problems due to assignable causes at an early stage, helping to prevent defects and improve quality performance. In recent years, neural networks have been applied to the pattern recognition task for control charts. The emphasis has been on pattern detection and identification rather than more detailed pattern parameter information, such as shift magnitude, trend slope, etc., which is vital for effective assignable cause analysis. Moreover, the identification of concurrent patterns (where two or more patterns exist together) which are commonly encountered in practical manufacturing processes has not been reported. This paper proposes a neural network-based approach to recognize typical abnormal patterns and in addition to accurately identify key parameters of the specific patterns involved. Both single and concurrent patterns can be characterized using this approach. A sequential pattern analysis (SPA) design was adopted to tackle complexity and prevent interference between pattern categories. The performance of the model has been evaluated using a simulation approach, and numerical and graphical results are presented which demonstrate that the approach performs effectively in control chart pattern recognition and accurately identifies the key parameters of the recognized pattern(s) in both single and concurrent pattern circumstances.     

A flexible sequence alignment approach on pattern mining and matching for human activity recognition

This paper proposes a flexible sequence alignment approach for pattern mining and matching in the recognition of human activities. During pattern mining, the proposed sequence alignment algorithm is invoked to extract out the representative patterns which denote specific activities of a person from the training patterns. It features high performance and robustness on pattern diversity. Besides, the algorithm evaluates the appearance probability of each pattern as weight and allows adapting pattern length to various human activities. Both of them are able to improve the accuracy of activity recognition. In pattern matching, the proposed algorithm adopts a dynamic programming based strategy to evaluate the correlation degree between each representative activity pattern and the observed activity sequence. It can avoid the trouble on segmenting the observed sequence. Moreover, we are able to obtain recognition results continuously. Besides, the proposed matching algorithm favors recognition of concurrent human activities with parallel matching. The experimental result confirms the high accuracy of human activity recognition by the proposed approach.     

Hybrid evolutionary learning for synthesizing multi-class pattern recognition systems

This paper describes one aspect of a machine-learning system called HELPR that blends the best aspects of different evolutionary techniques to bootstrap-up a complete recognition system from primitive input data. HELPR uses a multi-faceted representation consisting of a growing sequence of non-linear mathematical expressions. Individual features are represented as tree structures and manipulated using the techniques of genetic programming. Sets of features are represented as list structures that are manipulated using genetic algorithms and evolutionary programming. Complete recognition systems are formed in this version of HELPR by attaching the evolved features to multiple perceptron discriminators. Experiments on datasets from the University of California at Irvine (UCI) machine-learning repository show that HELPR’s performance meets or exceeds accuracies previously published.     

Evolving transfer functions for artificial neural networks

The paper describes a methodology for constructing transfer functions for the hidden layer of a back-propagation network, which is based on evolutionary programming. The method allows the construction of almost any mathematical form. It is tested using four benchmark classification problems from the well-known machine intelligence problems repository maintained by the University of California, Irvine. It was found that functions other than the commonly used sigmoidal function could perform well when used as hidden layer transfer functions. Three of the four problems showed improved test results when these evolved functions were used.     

A unified approach to pattern recognition

The paper is an outline of a new approach to pattern recognition developed by the author. A fuller introduction to the approach will appear soon.⁽¹⁾ Within the proposed framework the two principal approaches to pattern recognition—vector and syntactic—are unified.     

A multilevel approach to pattern processing

This paper presents a methodology for describing multilevel pattern processing systems. It is suggested that any pattern processor can be adequately described in terms of multiple hierarchies of two types of fundamental mechanism: (1) a process which performs the pattern recognition functions of analysis and synthesis and (2) a process which performs the syntactic functions of parsing and generation. A computer implementation of these principles is outlined which enables a range of systems to be configured. Examples of speech and non-speech pattern processing are presented.     

Parallel double-conversion spiking neural network for world-centered recognition

The separation of patterns from ground is the necessary requirement for recognition. Most of neural recognition models are network-centered without the ability to extract patterns. As a result, some non-neural methods and the learning of pattern's variant positions are used to complete the task. This article presents a spiking double-conversion network (DCN) to search for patterns in input using the double conversions from the network-centered input vector to a time sequence and further from the sequence to pattern-centered vector. DCN is designed for network-centered recognition and cluster models to extend them to world-centered ones.     

Software support for the construction of syntactic pattern recognisers by a research engineer

Syntactic pattern recognition is introduced and it is suggested that a research engineer will probably be less familiar with the language theory underlying syntactic pattern recognition than with the statistical ideas connected with decision theoretic methods. For this reason application of syntactic pattern recognition will only develop if software support is provided. The steps of grammatical inference, recogniser construction and recogniser optimisation are outlined. They are included in a scheme by which, working interactively and iteratively, a syntactic pattern recogniser can be produced. Some comments on an implementation are given.     

A higher-order neural network for distortion invariant pattern recognition

Recently, it is shown that a single layer, higher-order neural network is effective for scale, rotation and shift invariance and in the training process it requires only one example for one category and a very small number of iterations. However, there are problems that scale invariance doesn't hold precisely and it is not so effective for distortion of unknown patterns. In this paper we present an idea to realize the scale invariance precisely and suggest a method that is available to distorted patterns. The experimental results are presented to show the feasibility of our approach.     

Conception of complex probabilistic neural network system for classification of partial discharge patterns using multifarious inputs

Pattern recognition has a long history within electrical engineering but has recently become much more widespread as the automated capture of signal and images has been cheaper. Very many of the application of neural networks are to classification, and so are within the field of pattern recognition and classification. In this paper, we explore how probabilistic neural networks fit into the earlier framework of pattern recognition of partial discharge patterns since the PD patterns are an important tool for diagnosis of HV insulation systems. Skilled humans can identify the possible insulation defects in various representations of partial discharge (PD) data. One of the most widely used representation is phase resolved PD (PRPD) patterns. Also this paper describes a method for the automated recognition of PRPD patterns using a novel complex probabilistic neural network system for the actual classification task. The efficacy of composite neural network developed using probabilistic neural network is examined.     

A net-structure learning system for pattern description

This paper discusses a computer program that recognizes and describes two-dimensional patterns composed of subpatterns. The program also recognizes all patterns in a scene consisting of several patterns.

Patterns are stored in a learned hierarchical, net-structure memory. Weighted links between memory nodes represent subpattern/pattern relationships. Both short term and permanent memories are used.

Pattern recognition is accomplished with a serial heuristic search algorithm, which attempts to search memory and compute input properties efficiently. Without special processing, the program can be asked to look for all occurrences of a specified pattern in a scene.     

A cooperative constructive method for neural networks for pattern recognition

In this paper, we propose a new constructive method, based on cooperative coevolution, for designing automatically the structure of a neural network for classification. Our approach is based on a modular construction of the neural network by means of a cooperative evolutionary process. This process benefits from the advantages of coevolutionary computation as well as the advantages of constructive methods. The proposed methodology can be easily extended to work with almost any kind of classifier.The evaluation of each module that constitutes the network is made using a multiobjective method. So, each new module can be evaluated in a comprehensive way, considering different aspects, such as performance, complexity, or degree of cooperation with the previous modules of the network. In this way, the method has the advantage of considering not only the performance of the networks, but also other features.The method is tested on 40 classification problems from the UCI machine learning repository with very good performance. The method is thoroughly compared with two other constructive methods, cascade correlation and GMDH networks, and other classification methods, namely, SVM, C4.5, and k nearest-neighbours, and an ensemble of neural networks constructed using four different methods.     

A bi-objective approach for shortest-path network interdiction

In the literature, solution approaches to the shortest-path network interdiction problem have been developed for optimizing a single figure-of-merit of the network configuration when considering limited amount of resources available to interdict network links. This paper presents a newly developed evolutionary algorithm that allows approximating the optimal Pareto set of network interdiction strategies when considering bi-objective shortest path problems. Thus, the paper considers the concurrent optimization of two objectives: (1) maximization of shortest-path length and (2) minimization of interdiction strategy cost. Also, the paper considers the transformation of the first objective into the minimization of the most reliable path reliability. To solve these multi-objective optimization problems, an evolutionary algorithm has been developed. This algorithm is based on Monte Carlo simulation, to generate potential network interdiction strategies, graph theory to analyze strategies’ shortest path or most reliable path and, an evolutionary search driven by the probability that a link will appear in the optimal Pareto set. Examples for different sizes of networks and network behavior are used throughout the paper to illustrate and validate the approach.     

A comparison study of pattern recognition algorithms implemented on a microcontroller for use in an electronic tongue for monitoring drinking waters

A portable electronic tongue has been developed using an array of eighteen thick-film electrodes of different materials forming a multi-electrode array. A microcontroller is used to implement the pattern recognition. The classification of drinking waters is carried out by a Microchip PIC18F4550 micro-controller and is based on neural networks algorithms. These algorithm are initially trained with the multi-electrode array on a Personal Computer (PC) using several samples of waters (still, sparkling and tap) to obtain the optimum architecture of the networks. Once it is trained, the computed data are programmed into the microcontroller, which then gives the water classification directly for new unknown water samples. A comparative study between a Fuzzy ARTMAP, a Multi-Layer Feed-Forward network (MLFF) and a Linear Discriminant Analysis (LDA) has been done in order to obtain the best implementation on a microcontroller.     

A hybrid machine learning approach to network anomaly detection

Zero-day cyber attacks such as worms and spy-ware are becoming increasingly widespread and dangerous. The existing signature-based intrusion detection mechanisms are often not sufficient in detecting these types of attacks. As a result, anomaly intrusion detection methods have been developed to cope with such attacks. Among the variety of anomaly detection approaches, the Support Vector Machine (SVM) is known to be one of the best machine learning algorithms to classify abnormal behaviors. The soft-margin SVM is one of the well-known basic SVM methods using supervised learning. However, it is not appropriate to use the soft-margin SVM method for detecting novel attacks in Internet traffic since it requires pre-acquired learning information for supervised learning procedure. Such pre-acquired learning information is divided into normal and attack traffic with labels separately. Furthermore, we apply the one-class SVM approach using unsupervised learning for detecting anomalies. This means one-class SVM does not require the labeled information. However, there is downside to using one-class SVM: it is difficult to use the one-class SVM in the real world, due to its high false positive rate. In this paper, we propose a new SVM approach, named Enhanced SVM, which combines these two methods in order to provide unsupervised learning and low false alarm capability, similar to that of a supervised SVM approach.We use the following additional techniques to improve the performance of the proposed approach (referred to as Anomaly Detector using Enhanced SVM): First, we create a profile of normal packets using Self-Organized Feature Map (SOFM), for SVM learning without pre-existing knowledge. Second, we use a packet filtering scheme based on Passive TCP/IP Fingerprinting (PTF), in order to reject incomplete network traffic that either violates the TCP/IP standard or generation policy inside of well-known platforms. Third, a feature selection technique using a Genetic Algorithm (GA) is used for extracting optimized information from raw internet packets. Fourth, we use the flow of packets based on temporal relationships during data preprocessing, for considering the temporal relationships among the inputs used in SVM learning. Lastly, we demonstrate the effectiveness of the Enhanced SVM approach using the above-mentioned techniques, such as SOFM, PTF, and GA on MIT Lincoln Lab datasets, and a live dataset captured from a real network. The experimental results are verified by m-fold cross validation, and the proposed approach is compared with real world Network Intrusion Detection Systems (NIDS).     

Improving the Fuzzy Min-Max neural network with a K-nearest hyperbox expansion rule for pattern classification

An improved Fuzzy Min-Max (FMM) neural network with a K-nearest hyperbox expansion rule is proposed in this paper. The aim is to reduce the FMM network complexity for undertaking pattern classification tasks. In the proposed model, a useful modification to overcome a number of identified limitations of the original FMM network and to improve its classification performance is derived. In particular, the K-nearest hyperbox expansion rule is formulated to reduce the network complexity by avoiding the creation of too many small hyperboxes within the vicinity of the winning hyperbox during the FMM learning stage. The effectiveness of the proposed model is evaluated using a number of benchmark data sets. The results compare favorably with those from various FMM variants and other existing classifiers.     

A tribe competition-based genetic algorithm for feature selection in pattern classification

Feature selection has always been a critical step in pattern recognition, in which evolutionary algorithms, such as the genetic algorithm (GA), are most commonly used. However, the individual encoding scheme used in various GAs would either pose a bias on the solution or require a pre-specified number of features, and hence may lead to less accurate results. In this paper, a tribe competition-based genetic algorithm (TCbGA) is proposed for feature selection in pattern classification. The population of individuals is divided into multiple tribes, and the initialization and evolutionary operations are modified to ensure that the number of selected features in each tribe follows a Gaussian distribution. Thus each tribe focuses on exploring a specific part of the solution space. Meanwhile, tribe competition is introduced to the evolution process, which allows the winning tribes, which produce better individuals, to enlarge their sizes, i.e. having more individuals to search their parts of the solution space. This algorithm, therefore, avoids the bias on solutions and requirement of a pre-specified number of features. We have evaluated our algorithm against several state-of-the-art feature selection approaches on 20 benchmark datasets. Our results suggest that the proposed TCbGA algorithm can identify the optimal feature subset more effectively and produce more accurate pattern classification.