Geno-mathematical identification of the multi-layer perceptron

In this paper, we will focus on the use of the three-layer backpropagation network in vector-valued time series estimation problems. The neural network provides a framework for noncomplex calculations to solve the estimation problem, yet the search for optimal or even feasible neural networks for stochastic processes is both time consuming and uncertain. The backpropagation algorithm—written in strict ANSI C—has been implemented as a standalone support library for the genetic hybrid algorithm (GHA) running on any sequential or parallel main frame computer. In order to cope with ill-conditioned time series problems, we extended the original backpropagation algorithm to a K nearest neighbors algorithm (K-NARX), where the number K is determined genetically along with a set of key parameters. In the K-NARX algorithm, the terminal solution at instant t can be used as a starting point for the next t, which tends to stabilize the optimization process when dealing with autocorrelated time series vectors. This possibility has proved to be especially useful in difficult time series problems. Following the prevailing research directions, we use a genetic algorithm to determine optimal parameterizations for the network, including the lag structure for the nonlinear vector time series system, the net structure with one or two hidden layers and the corresponding number of nodes, type of activation function (currently the standard logistic sigmoid, a bipolar transformation, the hyperbolic tangent, an exponential function and the sine function), the type of minimization algorithm, the number K of nearest neighbors in the K-NARX procedure, the initial value of the Levenberg–Marquardt damping parameter and the value of the neural learning (stabilization) coefficient α. We have focused on a flexible structure allowing addition of, e.g., new minimization algorithms and activation functions in the future. We demonstrate the power of the genetically trimmed K-NARX algorithm on a representative data set.     

Modeling manufacturing processes using a genetic programming-based fuzzy regression with detection of outliers

Fuzzy regression (FR) been demonstrated as a promising technique for modeling manufacturing processes where availability of data is limited. FR can only yield linear type FR models which have a higher degree of fuzziness, but FR ignores higher order or interaction terms and the influence of outliers, all of which usually exist in the manufacturing process data. Genetic programming (GP), on the other hand, can be used to generate models with higher order and interaction terms but it cannot address the fuzziness of the manufacturing process data. In this paper, genetic programming-based fuzzy regression (GP-FR), which combines the advantages of the two approaches to overcome the deficiencies of the commonly used existing modeling methods, is proposed in order to model manufacturing processes. GP-FR uses GP to generate model structures based on tree representation which can represent interaction and higher order terms of models, and it uses an FR generator based on fuzzy regression to determine outliers in experimental data sets. It determines the contribution and fuzziness of each term in the model by using experimental data excluding the outliers. To evaluate the effectiveness of GP-FR in modeling manufacturing processes, it was used to model a non-linear system and an epoxy dispensing process. The results were compared with those based on two commonly used FR methods, Tanka’s FR and Peters’ FR. The prediction accuracy of the models developed based on GP-FR was shown to be better than that of models based on the other two FR methods.     

Semi-supervised outlier detection based on fuzzy rough C-means clustering

This paper presents a fuzzy rough semi-supervised outlier detection (FRSSOD) approach with the help of some labeled samples and fuzzy rough C-means clustering. This method introduces an objective function, which minimizes the sum squared error of clustering results and the deviation from known labeled examples as well as the number of outliers. Each cluster is represented by a center, a crisp lower approximation and a fuzzy boundary by using fuzzy rough C-means clustering and only those points located in boundary can be further discussed the possibility to be reassigned as outliers. As a result, this method can obtain better clustering results for normal points and better accuracy for outlier detection. Experiment results show that the proposed method, on average, keep, or improve the detection precision and reduce false alarm rate as well as reduce the number of candidate outliers to be discussed.     

Genetic algorithms for outlier detection and variable selection in linear regression models

This article addresses some problems in outlier detection and variable selection in linear regression models. First, in outlier detection there are problems known as smearing and masking. Smearing means that one outlier makes another, non-outlier observation appear as an outlier, and masking that one outlier prevents another one from being detected. Detecting outliers one by one may therefore give misleading results. In this article a genetic algorithm is presented which considers different possible groupings of the data into outlier and non-outlier observations. In this way all outliers are detected at the same time. Second, it is known that outlier detection and variable selection can influence each other, and that different results may be obtained, depending on the order in which these two tasks are performed. It may therefore be useful to consider these tasks simultaneously, and a genetic algorithm for a simultaneous outlier detection and variable selection is suggested. Two real data sets are used to illustrate the algorithms, which are shown to work well. In addition, the scalability of the algorithms is considered with an experiment using generated data.I would like to thank Dr Tero Aittokallio and an anonymous referee for useful comments.     

Classification of weld flaws with imbalanced class data

This paper presents research results of our investigation of the imbalanced data problem in the classification of different types of weld flaws, a multi-class classification problem. The one-against-all scheme is adopted to carry out multi-class classification and three algorithms including minimum distance, nearest neighbors, and fuzzy nearest neighbors are employed as the classifiers. The effectiveness of 22 data preprocessing methods for dealing with imbalanced data is evaluated in terms of eight evaluation criteria to determine whether any method would emerge to dominate the others. The test results indicate that: (1) nearest neighbor classifiers outperform the minimum distance classifier; (2) some data preprocessing methods do not improve any criterion and they vary from one classifier to another; (3) the combination of using the AHC_KM data preprocessing method with the 1-NN classifier is the best because they together produce the best performance in six of eight evaluation criteria; and (4) the most difficult weld flaw type to recognize is crack.     

An effective and efficient algorithm for high-dimensional outlier detection

The outlier detection problem has important applications in the field of fraud detection, network robustness analysis, and intrusion detection. Most such applications are most important for high-dimensional domains in which the data can contain hundreds of dimensions. Many recent algorithms have been proposed for outlier detection that use several concepts of proximity in order to find the outliers based on their relationship to the other points in the data. However, in high-dimensional space, the data are sparse and concepts using the notion of proximity fail to retain their effectiveness. In fact, the sparsity of high-dimensional data can be understood in a different way so as to imply that every point is an equally good outlier from the perspective of distance-based definitions. Consequently, for high-dimensional data, the notion of finding meaningful outliers becomes substantially more complex and nonobvious. In this paper, we discuss new techniques for outlier detection that find the outliers by studying the behavior of projections from the data set.Received: 19 November 2002, Accepted: 6 February 2004, Published online: 19 August 2004Edited by: R. Ng.     

Ellipsoidal neighbourhood outlier factor for distributed anomaly detection in resource constrained networks

Anomaly detection in resource constrained wireless networks is an important challenge for tasks such as intrusion detection, quality assurance and event monitoring applications. The challenge is to detect these interesting events or anomalies in a timely manner, while minimising energy consumption in the network. We propose a distributed anomaly detection architecture, which uses multiple hyperellipsoidal clusters to model the data at each sensor node, and identify global and local anomalies in the network. In particular, a novel anomaly scoring method is proposed to provide a score for each hyperellipsoidal model, based on how remote the ellipsoid is relative to their neighbours. We demonstrate using several synthetic and real datasets that our proposed scheme achieves a higher detection performance with a significant reduction in communication overhead in the network compared to centralised and existing schemes.     

Stability of vector optimization problems with fuzzy weights in the objective functions and fuzzy matrix parameters in the constraints

The aim of this paper is to investigate the stability of multiobjective nonlinear programming problems with fuzzy weights in the objective functions and fuzzy matrix parameters in the constraints and represent, in addition, the related dual problems for which the set of feasible parameters and the solvability set are studied. These fuzzy weights and fuzzy matrix parameters are characterized by fuzzy numbers. The existing results concerning the basic notions parametric space in convex programs are redefined and analyzed qualitatively under the concept of α-Pareto optimality. An illustrative example is given to clarify the obtained results.     

On the strict logic foundation of fuzzy reasoning

This paper focuses on the logic foundation of fuzzy reasoning. At first, a new complete first-order fuzzy predicate calculus system K^* corresponding to the formal system L^* is built. Based on the many-sort system K_ms^* corresponding to K^*, the triple I methods of FMP and FMT for fuzzy reasoning and their consistency are formalized, thus fuzzy reasoning is put completely and rigorously into the logic framework of fuzzy logic.The author is indebted to anonymous referee for his useful comments which have helped to improve the paper.     

A fuzzy clustering algorithm based on evolutionary programming

In this paper, a fuzzy clustering method based on evolutionary programming (EPFCM) is proposed. The algorithm benefits from the global search strategy of evolutionary programming, to improve fuzzy c-means algorithm (FCM). The cluster validity can be measured by some cluster validity indices. To increase the convergence speed of the algorithm, we exploit the modified algorithm to change the number of cluster centers dynamically. Experiments demonstrate EPFCM can find the proper number of clusters, and the result of clustering does not depend critically on the choice of the initial cluster centers. The probability of trapping into the local optima will be very lower than FCM.     

一种求解旅行商问题的高效混合遗传算法

旅行商问题(TravellingSalesmanProblemTSP)是一个典型的组合优化难题,论文提出一种求解旅行商问题的高效混合遗传算法。该算法结合遗传算法和2-opt邻域搜索优化技术,并针对旅行商问题的特点,提出K近邻点集以缩减搜索空间从而加快求解速度。基于典型实例的仿真结果表明,此算法的求解效率比较高。     

Piecewise regression for fuzzy input–output data with automatic change-point detection by quadratic programming

To handle the large variation issues in fuzzy input–output data, the proposed quadratic programming (QP) method uses a piecewise approach to simultaneously generate the possibility and necessity models, as well as the change-points. According to Tanaka and Lee [H. Tanaka, H. Lee, Interval regression analysis by quadratic programming approach, IEEE Transactions on Fuzzy Systems 6 (1998) 473–481], the QP approach gives more diversely spread coefficients than linear programming (LP) does. However, their approach only deals with crisp input and fuzzy output data. Moreover, their method is weak in handling fluctuating data. So far, no method has been developed to cope with the large variation problems in fuzzy input–output data. Hence, we propose a piecewise regression for fuzzy input–output data with a QP approach. There are three advantages in our method. First, the QP technique gives a more diversely spread coefficient than does a linear programming technique. Second, the piecewise approach is used to detect the change-points in the estimated model automatically, and handle the large variation data such as outliers well. Third, the possibility and necessity models with better fitness in data processing are obtained at the same time. Two examples are presented to demonstrate the merits of the proposed method.     

A comparison of two learning mechanisms for the automatic design of fuzzy diagnosis systems for rotating machinery

In this paper, we approach the problem of automatically designing fuzzy diagnosis rules for rotating machinery, which can give an appropriate evaluation of the vibration data measured in the target machines. In particular, we explain the implementation to this aim and analyze the advantages and drawbacks of two soft computing techniques: knowledge-based networks (KBN) and genetic algorithms (GA). An application of both techniques is evaluated on the same case study, giving special emphasis to their performance in terms of classification success and computation time.A reduced version of this paper first appeared under the title “A comparative assessment on the application of knowledge-based networks and genetic algorithms to the design of fuzzy diagnosis systems for rotating machinery”, published in the book “Soft Computing in Industry—Recent Appliactions” (Springer Engineering).     

Credibility-based chance-constrained integer programming models for capital budgeting with fuzzy parameters

In this paper, we discuss a problem of capital budgeting in a fuzzy environment. Two types of models are proposed using credibility to measure confidence level. Since the proposed optimization problems are difficult to solve by traditional methods, a fuzzy simulation-based genetic algorithm is applied. Two numerical experiments demonstrate the effectiveness of the proposed algorithm.     

A two-phase approach for multi-objective programming problems with fuzzy coefficients

In this study, a two-phase procedure is introduced to solve multi-objective fuzzy linear programming problems. The procedure provides a practical solution approach, which is an integration of fuzzy parametric programming (FPP) and fuzzy linear programming (FLP), for solving real life multiple objective programming problems with all fuzzy coefficients. The interactive concept of the procedure is performed to reach simultaneous optimal solutions for all objective functions for different grades of precision according to the preferences of the decision-maker (DM). The procedure can be also performed to obtain lexicographic optimal and/or additive solutions if it is needed. In the first phase of the procedure, a family of vector optimization models is constructed by using FPP. Then in the second phase, each model is solved by FLP. The solutions are optimal and each one is an alternative decision plan for the DM.     

A fuzzy expert system for the early warning of accidents due to driver hypo-vigilance

In this paper a fuzzy expert system for the prediction of hypovigilance-related accidents is presented. The system uses physiological modalities in order to detect signs of extreme hypovigilance. An advantage of such a system is its extensibility regarding the physiological modalities and features that it can use as inputs. In that way, even though at present only eyelid-related features are exploited, in the future and for prototypes designed for professionals other physiological modalities, such as EEG can be easily integrated into the existing system in order to make it more robust and reliable.     

A Novel Approach to Noise Clustering for Outlier Detection

Noise clustering, as a robust clustering method, performs partitioning of data sets reducing errors caused by outliers. Noise clustering defines outliers in terms of a certain distance, which is called noise distance. The probability or membership degree of data points belonging to the noise cluster increases with their distance to regular clusters. The main purpose of noise clustering is to reduce the influence of outliers on the regular clusters. The emphasis is not put on exactly identifying outliers. However, in many applications outliers contain important information and their correct identification is crucial. In this paper we present a method to estimate the noise distance in noise clustering based on the preservation of the hypervolume of the feature space. Our examples will demonstrate the efficiency of this approach.     

A fuzzy reasoning design for fault detection and diagnosis of a computer-controlled system

A fuzzy reasoning and verification Petri nets (FRVPNs) model is established for an error detection and diagnosis mechanism applied to a complex fault-tolerant PC-controlled system. The inference accuracy can be improved through the hierarchical design of a two-level fuzzy rule decision tree and a Petri nets technique to transform the fuzzy rule into the FRVPNs model. Several simulation examples of the assumed failure events were carried out by using the FRVPNs and the Mamdani fuzzy method with MATLAB tools. The reasoning performance of the developed FRVPNs was verified by comparing the inference outcome to that of the Mamdani method. Both methods result in the same conclusions. Thus, the present study demonstrates that the proposed FRVPNs model is able to achieve the purpose of reasoning, and furthermore, determining of the failure event of the monitored application program.     

A novel encoding algorithm for vector quantization using transformed codebook

In this paper, a novel encoding algorithm for vector quantization is presented. Our method uses a set of transformed codewords and partial distortion rejection to determine the reproduction vector of an input vector. Experimental results show that our algorithm is superior to other methods in terms of the computing time and number of distance calculations. Compared with available approaches, our method can reduce the computing time and number of distance calculations significantly. Compared with the available best method of reducing the number of distance computations, our approach can reduce the number of distance calculations by 32.3-67.1%. Compared with the best encoding algorithm for vector quantization, our method can also further reduce the computing time by 19.7-23.9%. The performance of our method is better when a larger codebook is used and is weakly correlated to codebook size.