Geno-mathematical identification of the multi-layer perceptron

In this paper, we will focus on the use of the three-layer backpropagation network in vector-valued time series estimation problems. The neural network provides a framework for noncomplex calculations to solve the estimation problem, yet the search for optimal or even feasible neural networks for stochastic processes is both time consuming and uncertain. The backpropagation algorithm—written in strict ANSI C—has been implemented as a standalone support library for the genetic hybrid algorithm (GHA) running on any sequential or parallel main frame computer. In order to cope with ill-conditioned time series problems, we extended the original backpropagation algorithm to a K nearest neighbors algorithm (K-NARX), where the number K is determined genetically along with a set of key parameters. In the K-NARX algorithm, the terminal solution at instant t can be used as a starting point for the next t, which tends to stabilize the optimization process when dealing with autocorrelated time series vectors. This possibility has proved to be especially useful in difficult time series problems. Following the prevailing research directions, we use a genetic algorithm to determine optimal parameterizations for the network, including the lag structure for the nonlinear vector time series system, the net structure with one or two hidden layers and the corresponding number of nodes, type of activation function (currently the standard logistic sigmoid, a bipolar transformation, the hyperbolic tangent, an exponential function and the sine function), the type of minimization algorithm, the number K of nearest neighbors in the K-NARX procedure, the initial value of the Levenberg–Marquardt damping parameter and the value of the neural learning (stabilization) coefficient α. We have focused on a flexible structure allowing addition of, e.g., new minimization algorithms and activation functions in the future. We demonstrate the power of the genetically trimmed K-NARX algorithm on a representative data set.     

Modeling manufacturing processes using a genetic programming-based fuzzy regression with detection of outliers

Fuzzy regression (FR) been demonstrated as a promising technique for modeling manufacturing processes where availability of data is limited. FR can only yield linear type FR models which have a higher degree of fuzziness, but FR ignores higher order or interaction terms and the influence of outliers, all of which usually exist in the manufacturing process data. Genetic programming (GP), on the other hand, can be used to generate models with higher order and interaction terms but it cannot address the fuzziness of the manufacturing process data. In this paper, genetic programming-based fuzzy regression (GP-FR), which combines the advantages of the two approaches to overcome the deficiencies of the commonly used existing modeling methods, is proposed in order to model manufacturing processes. GP-FR uses GP to generate model structures based on tree representation which can represent interaction and higher order terms of models, and it uses an FR generator based on fuzzy regression to determine outliers in experimental data sets. It determines the contribution and fuzziness of each term in the model by using experimental data excluding the outliers. To evaluate the effectiveness of GP-FR in modeling manufacturing processes, it was used to model a non-linear system and an epoxy dispensing process. The results were compared with those based on two commonly used FR methods, Tanka’s FR and Peters’ FR. The prediction accuracy of the models developed based on GP-FR was shown to be better than that of models based on the other two FR methods.     

Semi-supervised outlier detection based on fuzzy rough C-means clustering

This paper presents a fuzzy rough semi-supervised outlier detection (FRSSOD) approach with the help of some labeled samples and fuzzy rough C-means clustering. This method introduces an objective function, which minimizes the sum squared error of clustering results and the deviation from known labeled examples as well as the number of outliers. Each cluster is represented by a center, a crisp lower approximation and a fuzzy boundary by using fuzzy rough C-means clustering and only those points located in boundary can be further discussed the possibility to be reassigned as outliers. As a result, this method can obtain better clustering results for normal points and better accuracy for outlier detection. Experiment results show that the proposed method, on average, keep, or improve the detection precision and reduce false alarm rate as well as reduce the number of candidate outliers to be discussed.     

Genetic algorithms for outlier detection and variable selection in linear regression models

This article addresses some problems in outlier detection and variable selection in linear regression models. First, in outlier detection there are problems known as smearing and masking. Smearing means that one outlier makes another, non-outlier observation appear as an outlier, and masking that one outlier prevents another one from being detected. Detecting outliers one by one may therefore give misleading results. In this article a genetic algorithm is presented which considers different possible groupings of the data into outlier and non-outlier observations. In this way all outliers are detected at the same time. Second, it is known that outlier detection and variable selection can influence each other, and that different results may be obtained, depending on the order in which these two tasks are performed. It may therefore be useful to consider these tasks simultaneously, and a genetic algorithm for a simultaneous outlier detection and variable selection is suggested. Two real data sets are used to illustrate the algorithms, which are shown to work well. In addition, the scalability of the algorithms is considered with an experiment using generated data.I would like to thank Dr Tero Aittokallio and an anonymous referee for useful comments.     

Research on outlier detection algorithm for express logistics

To detect the problems of time delay, path error and destination error in express logistics process effectively, a novel outlier detection algorithm for express logistics is proposed in this paper. To test the detection results, the express logistics system operating model is built to test the detection results. Experiment results show that the proposed algorithm is well applied to the express logistics data with multi-attribute characteristics, and can work well in detecting the abnormal conditions of express logistics.     

Classification of weld flaws with imbalanced class data

This paper presents research results of our investigation of the imbalanced data problem in the classification of different types of weld flaws, a multi-class classification problem. The one-against-all scheme is adopted to carry out multi-class classification and three algorithms including minimum distance, nearest neighbors, and fuzzy nearest neighbors are employed as the classifiers. The effectiveness of 22 data preprocessing methods for dealing with imbalanced data is evaluated in terms of eight evaluation criteria to determine whether any method would emerge to dominate the others. The test results indicate that: (1) nearest neighbor classifiers outperform the minimum distance classifier; (2) some data preprocessing methods do not improve any criterion and they vary from one classifier to another; (3) the combination of using the AHC_KM data preprocessing method with the 1-NN classifier is the best because they together produce the best performance in six of eight evaluation criteria; and (4) the most difficult weld flaw type to recognize is crack.     

An effective and efficient algorithm for high-dimensional outlier detection

The outlier detection problem has important applications in the field of fraud detection, network robustness analysis, and intrusion detection. Most such applications are most important for high-dimensional domains in which the data can contain hundreds of dimensions. Many recent algorithms have been proposed for outlier detection that use several concepts of proximity in order to find the outliers based on their relationship to the other points in the data. However, in high-dimensional space, the data are sparse and concepts using the notion of proximity fail to retain their effectiveness. In fact, the sparsity of high-dimensional data can be understood in a different way so as to imply that every point is an equally good outlier from the perspective of distance-based definitions. Consequently, for high-dimensional data, the notion of finding meaningful outliers becomes substantially more complex and nonobvious. In this paper, we discuss new techniques for outlier detection that find the outliers by studying the behavior of projections from the data set.Received: 19 November 2002, Accepted: 6 February 2004, Published online: 19 August 2004Edited by: R. Ng.     

A survey of outlier detection in high dimensional data streams

The rapid evolution of technology has led to the generation of high dimensional data streams in a wide range of fields, such as genomics, signal processing, and finance. The combination of the streaming scenario and high dimensionality is particularly challenging especially for the outlier detection task. This is due to the special characteristics of the data stream such as the concept drift, the limited time and space requirements, in addition to the impact of the well-known curse of dimensionality in high dimensional space. To the best of our knowledge, few studies have addressed these challenges simultaneously, and therefore detecting anomalies in this context requires a great deal of attention. The main objective of this work is to study the main approaches existing in the literature, to identify a set of comparison criteria, such as the computational cost and the interpretation of outliers, which will help us to reveal the different challenges and additional research directions associated with this problem. At the end of this study, we will draw up a summary report which summarizes the main limits identified and we will detail the different directions of research related to this issue in order to promote research for this community.     

适用于关联属性的样本自适应参数孤立点检测法

为解决数据集中关联属性之间的干扰问题,通过引进Mahalanobis距离,并对传统的k近邻孤立点检测方法进行了改进,提出了一种新的基于样本的参数选取方法。该方法通过训练数据集中的正常数据和孤立点数据,以获得最优的k距离值和阈值。实验仿真结果表明,提出的算法有更高的准确率,同时降低了误检率。     

Ellipsoidal neighbourhood outlier factor for distributed anomaly detection in resource constrained networks

Anomaly detection in resource constrained wireless networks is an important challenge for tasks such as intrusion detection, quality assurance and event monitoring applications. The challenge is to detect these interesting events or anomalies in a timely manner, while minimising energy consumption in the network. We propose a distributed anomaly detection architecture, which uses multiple hyperellipsoidal clusters to model the data at each sensor node, and identify global and local anomalies in the network. In particular, a novel anomaly scoring method is proposed to provide a score for each hyperellipsoidal model, based on how remote the ellipsoid is relative to their neighbours. We demonstrate using several synthetic and real datasets that our proposed scheme achieves a higher detection performance with a significant reduction in communication overhead in the network compared to centralised and existing schemes.     

时间序列异常模式的k-均距异常因子检测

提出了一种基于k-均距异常因子检测时间序列异常模式的算法（K-MDOF）。该算法首先利用边缘权重因子提取时间序列模式表示的边缘点,然后通过提取每一段子模式的四个特征值：模式长度、模式高度、模式均值和标准差将时间序列映射到特征空间,最后利用k-均距异常因子在该特征空间中检测时间序列的异常模式。从模式的角度检测时间序列的异常行为弥补了点异常检测的个体行为局限性,提高了异常检测的效率和准确性,在仿真数据集和真实数据集上的实验结果都证明了在时间序列异常检测中模式异常定义的合理性以及算法的有效性。     

Stability of vector optimization problems with fuzzy weights in the objective functions and fuzzy matrix parameters in the constraints

The aim of this paper is to investigate the stability of multiobjective nonlinear programming problems with fuzzy weights in the objective functions and fuzzy matrix parameters in the constraints and represent, in addition, the related dual problems for which the set of feasible parameters and the solvability set are studied. These fuzzy weights and fuzzy matrix parameters are characterized by fuzzy numbers. The existing results concerning the basic notions parametric space in convex programs are redefined and analyzed qualitatively under the concept of α-Pareto optimality. An illustrative example is given to clarify the obtained results.     

A multi-objective genetic algorithm with fuzzy c-means for automatic data clustering

This article presents a multi-objective genetic algorithm which considers the problem of data clustering. A given dataset is automatically assigned into a number of groups in appropriate fuzzy partitions through the fuzzy c-means method. This work has tried to exploit the advantage of fuzzy properties which provide capability to handle overlapping clusters. However, most fuzzy methods are based on compactness and/or separation measures which use only centroid information. The calculation from centroid information only may not be sufficient to differentiate the geometric structures of clusters. The overlap-separation measure using an aggregation operation of fuzzy membership degrees is better equipped to handle this drawback. For another key consideration, we need a mechanism to identify appropriate fuzzy clusters without prior knowledge on the number of clusters. From this requirement, an optimization with single criterion may not be feasible for different cluster shapes. A multi-objective genetic algorithm is therefore appropriate to search for fuzzy partitions in this situation. Apart from the overlap-separation measure, the well-known fuzzy J_m index is also optimized through genetic operations. The algorithm simultaneously optimizes the two criteria to search for optimal clustering solutions. A string of real-coded values is encoded to represent cluster centers. A number of strings with different lengths varied over a range correspond to variable numbers of clusters. These real-coded values are optimized and the Pareto solutions corresponding to a tradeoff between the two objectives are finally produced. As shown in the experiments, the approach provides promising solutions in well-separated, hyperspherical and overlapping clusters from synthetic and real-life data sets. This is demonstrated by the comparison with existing single-objective and multi-objective clustering techniques.     

On the strict logic foundation of fuzzy reasoning

This paper focuses on the logic foundation of fuzzy reasoning. At first, a new complete first-order fuzzy predicate calculus system K^* corresponding to the formal system L^* is built. Based on the many-sort system K_ms^* corresponding to K^*, the triple I methods of FMP and FMT for fuzzy reasoning and their consistency are formalized, thus fuzzy reasoning is put completely and rigorously into the logic framework of fuzzy logic.The author is indebted to anonymous referee for his useful comments which have helped to improve the paper.     

A hybrid fuzzy-GA algorithm for the integrated machine allocation problem with fuzzy demands

The integrated machine allocation and facility layout problem (IMALP) is a branch of the general facility layout problem in which, besides selecting machine locations, the processing route of each product is determined. Most research in this area suppose that the flow of material is certain and exact, which is an unrealistic assumption in today's dynamic and uncertain business environment. Therefore, in this paper the demand volume has been assumed as fuzzy numbers with different membership functions. To solve this problem, the deterministic model is first integrated with a fuzzy implication via the expected value model, and thereafter an intelligent hybrid algorithm, including a genetic algorithm and a fuzzy simulation approach has been applied. Finally, the efficiency of the proposed algorithm is evaluated with a set of numerical examples. The results show the effectiveness of the hybrid algorithm in finding the IMALP solutions.     

A fuzzy clustering algorithm based on evolutionary programming

In this paper, a fuzzy clustering method based on evolutionary programming (EPFCM) is proposed. The algorithm benefits from the global search strategy of evolutionary programming, to improve fuzzy c-means algorithm (FCM). The cluster validity can be measured by some cluster validity indices. To increase the convergence speed of the algorithm, we exploit the modified algorithm to change the number of cluster centers dynamically. Experiments demonstrate EPFCM can find the proper number of clusters, and the result of clustering does not depend critically on the choice of the initial cluster centers. The probability of trapping into the local optima will be very lower than FCM.     

一种求解旅行商问题的高效混合遗传算法

旅行商问题(TravellingSalesmanProblemTSP)是一个典型的组合优化难题,论文提出一种求解旅行商问题的高效混合遗传算法。该算法结合遗传算法和2-opt邻域搜索优化技术,并针对旅行商问题的特点,提出K近邻点集以缩减搜索空间从而加快求解速度。基于典型实例的仿真结果表明,此算法的求解效率比较高。     

Piecewise regression for fuzzy input–output data with automatic change-point detection by quadratic programming

To handle the large variation issues in fuzzy input–output data, the proposed quadratic programming (QP) method uses a piecewise approach to simultaneously generate the possibility and necessity models, as well as the change-points. According to Tanaka and Lee [H. Tanaka, H. Lee, Interval regression analysis by quadratic programming approach, IEEE Transactions on Fuzzy Systems 6 (1998) 473–481], the QP approach gives more diversely spread coefficients than linear programming (LP) does. However, their approach only deals with crisp input and fuzzy output data. Moreover, their method is weak in handling fluctuating data. So far, no method has been developed to cope with the large variation problems in fuzzy input–output data. Hence, we propose a piecewise regression for fuzzy input–output data with a QP approach. There are three advantages in our method. First, the QP technique gives a more diversely spread coefficient than does a linear programming technique. Second, the piecewise approach is used to detect the change-points in the estimated model automatically, and handle the large variation data such as outliers well. Third, the possibility and necessity models with better fitness in data processing are obtained at the same time. Two examples are presented to demonstrate the merits of the proposed method.     

A comparison of two learning mechanisms for the automatic design of fuzzy diagnosis systems for rotating machinery

In this paper, we approach the problem of automatically designing fuzzy diagnosis rules for rotating machinery, which can give an appropriate evaluation of the vibration data measured in the target machines. In particular, we explain the implementation to this aim and analyze the advantages and drawbacks of two soft computing techniques: knowledge-based networks (KBN) and genetic algorithms (GA). An application of both techniques is evaluated on the same case study, giving special emphasis to their performance in terms of classification success and computation time.A reduced version of this paper first appeared under the title “A comparative assessment on the application of knowledge-based networks and genetic algorithms to the design of fuzzy diagnosis systems for rotating machinery”, published in the book “Soft Computing in Industry—Recent Appliactions” (Springer Engineering).