Hybrid robust approach for TSK fuzzy modeling with outliers

This study proposes a hybrid robust approach for constructing Takagi–Sugeno–Kang (TSK) fuzzy models with outliers. The approach consists of a robust fuzzy C-regression model (RFCRM) clustering algorithm in the coarse-tuning phase and an annealing robust back-propagation (ARBP) learning algorithm in the fine-tuning phase. The RFCRM clustering algorithm is modified from the fuzzy C-regression models (FCRM) clustering algorithm by incorporating a robust mechanism and considering input data distribution and robust similarity measure into the FCRM clustering algorithm. Due to the use of robust mechanisms and the consideration of input data distribution, the fuzzy subspaces and the parameters of functions in the consequent parts are simultaneously identified by the proposed RFCRM clustering algorithm and the obtained model will not be significantly affected by outliers. Furthermore, the robust similarity measure is used in the clustering process to reduce the redundant clusters. Consequently, the RFCRM clustering algorithm can generate a better initialization for the TSK fuzzy models in the coarse-tuning phase. Then, an ARBP algorithm is employed to obtain a more precise model in the fine-tuning phase. From our simulation results, it is clearly evident that the proposed robust TSK fuzzy model approach is superior to existing approaches in learning speed and in approximation accuracy.     

Improving the interpretability of TSK fuzzy models by combiningglobal learning and local learning

The fuzzy inference system proposed by Takagi, Sugeno, and Kang, known as the TSK model in fuzzy system literature, provides a powerful tool for modeling complex nonlinear systems. Unlike conventional modeling where a single model is used to describe the global behavior of a system, TSK modeling is essentially a multimodel approach in which simple submodels (typically linear models) are combined to describe the global behavior of the system. Most existing learning algorithms for identifying the TSK model are based on minimizing the square of the residual between the overall outputs of the real system and the identified model. Although these algorithms can generate a TSK model with good global performance (i.e., the model is capable of approximating the given system with arbitrary accuracy, provided that sufficient rules are used and sufficient training data are available), they cannot guarantee the resulting model to have a good local performance. Often, the submodels in the TSK model may exhibit an erratic local behavior, which is difficult to interpret. Since one of the important motivations of using the TSK model (also other fuzzy models) is to gain insights into the model, it is important to investigate the interpretability issue of the TSK model. We propose a new learning algorithm that integrates global learning and local learning in a single algorithmic framework. This algorithm uses the idea of local weighed regression and local approximation in nonparametric statistics, but remains the component of global fitting in the existing learning algorithms. The algorithm is capable of adjusting its parameters based on the user's preference, generating models with good tradeoff in terms of global fitting and local interpretation. We illustrate the performance of the proposed algorithm using a motorcycle crash modeling example     

Nature-inspired optimal tuning of input membership functions of Takagi-Sugeno-Kang fuzzy models for Anti-lock Braking Systems

This paper suggests a synergy of fuzzy logic and nature-inspired optimization in terms of the nature-inspired optimal tuning of the input membership functions of a class of Takagi-Sugeno-Kang (TSK) fuzzy models dedicated to Anti-lock Braking Systems (ABSs). A set of TSK fuzzy models is proposed by a novel fuzzy modeling approach for ABSs. The fuzzy modeling approach starts with the derivation of a set of local state-space models of the nonlinear ABS process by the linearization of the first-principle process model at ten operating points. The TSK fuzzy model structure and the initial TSK fuzzy models are obtained by the modal equivalence principle in terms of placing the local state-space models in the rule consequents of the TSK fuzzy models. An operating point selection algorithm to guide modeling is proposed, formulated on the basis of ranking the operating points according to their importance factors, and inserted in the third step of the fuzzy modeling approach. The optimization problems are defined such that to minimize the objective functions expressed as the average of squared modeling errors over the time horizon, and the variables of these functions are a part of the parameters of the input membership functions. Two representative nature-inspired algorithms, namely a Simulated Annealing (SA) algorithm and a Particle Swarm Optimization (PSO) algorithm, are implemented to solve the optimization problems and to obtain optimal TSK fuzzy models. The validation and the comparison of SA and PSO and of the new TSK fuzzy models are carried out for an ABS laboratory equipment. The real-time experimental results highlight that the optimized TSK fuzzy models are simple and consistent with both training data and validation data and that these models outperform the initial TSK fuzzy models.     

Fuzzy and possibilistic shell clustering algorithms and theirapplication to boundary detection and surface approximation. I

Traditionally, prototype-based fuzzy clustering algorithms such as the Fuzzy C Means (FCM) algorithm have been used to find “compact” or “filled” clusters. Recently, there have been attempts to generalize such algorithms to the case of hollow or “shell-like” clusters, i.e., clusters that lie in subspaces of feature space. The shell clustering approach provides a powerful means to solve the hitherto unsolved problem of simultaneously fitting multiple curves/surfaces to unsegmented, scattered and sparse data. In this paper, we present several fuzzy and possibilistic algorithms to detect linear and quadric shell clusters. We also introduce generalizations of these algorithms in which the prototypes represent sets of higher-order polynomial functions. The suggested algorithms provide a good trade-off between computational complexity and performance, since the objective function used in these algorithms is the sum of squared distances, and the clustering is sensitive to noise and outliers. We show that by using a possibilistic approach to clustering, one can make the proposed algorithms robust     

A fuzzy modeling method via Enhanced Objective Cluster Analysis for designing TSK model

This paper proposes a fuzzy modeling method via Enhanced Objective Cluster Analysis to obtain the compact and robust approximate TSK fuzzy model. In our approach, the Objective Cluster Analysis algorithm is introduced. In order to obtain more compact and more robust fuzzy rule prototypes, this algorithm is enhanced by introducing the Relative Dissimilarity Measure and the new consistency criterion to represent the similarity degree between the clusters. By these additional criteria, the redundant clusters caused by iterations are avoided; the subjective influence from human judgment for clustering is weakened. Moreover the clustering results including the number of clusters and the cluster centers are considered as the initial condition of the premise parameters identification. Thus the traditional iteration modeling procedure for determining the number of rules and identifying parameters is changed into one-off modeling, which significantly reduces the burden of computation. Furthermore the decomposition errors and the approximation errors resulted from premise parameters identification by Fuzzy c-Means clustering are decreased. For the consequence parameters identification, the Stable Kalman Filter algorithm is adopted. The performance of the proposed modeling method is evaluated by the example of Box–Jenkins gas furnace. The simulation results demonstrate the power of our model.     

基于快速聚类分析的支持向量数据描述算法

针对数据中存在的噪声对数据描述建模的影响, 提出一种基于快速聚类分析的支持向量数据描述算法. 该算法通过快速聚类分析算法对所要建模的数据进行预处理, 通过预处理快速剔除数据中存在的影响建模的噪声; 然后再将基于??NN算法计算获得的权重值加权在每一个数据上, 进行支持向量数据描述算法的建模. 在标准数据集上的实验分析表明, 所提出的支持向量数据描述算法较传统的支持向量数据描述算法和密度驱动支持向量数据描述算法在准确度上具有较明显的提升.

     

Multibody grouping by inference of multiple subspaces from high-dimensional data using oriented-frames

Recently, subspace constraints have been widely exploited in many computer vision problems such as multibody grouping. Under linear projection models, feature points associated with multiple bodies reside in multiple subspaces. Most existing factorization-based algorithms can segment objects undergoing independent motions. However, intersections among the correlated motion subspaces will lead most previous factorization-based algorithms to erroneous segmentation. To overcome this limitation, in this paper, we formulate the problem of multibody grouping as inference of multiple subspaces from a high-dimensional data space. A novel and robust algorithm is proposed to capture the configuration of the multiple subspace structure and to find the segmentation of objects by clustering the feature points into these inferred subspaces, no matter whether they are independent or correlated. In the proposed method, an oriented-frame (OF), which is a multidimensional coordinate frame, is associated with each data point indicating the point's preferred subspace configuration. Based on the similarity between the subspaces, novel mechanisms of subspace evolution and voting are developed. By filtering the outliers due to their structural incompatibility, the subspace configurations will emerge. Compared with most existing factorization-based algorithms that cannot correctly segment correlated motions, such as motions of articulated objects, the proposed method has a robust performance in both independent and correlated motion segmentation. A number of controlled and real experiments show the effectiveness of the proposed method. However, the current approach does not deal with transparent motions and motion subspaces of different dimensions.     

A novel pruning approach for robust data clustering

In this paper, we make an effort to overcome the sensitivity of traditional clustering algorithms to noisy data points (noise and outliers). A novel pruning method, in terms of information theory, is therefore proposed to phase out noisy points for robust data clustering. This approach identifies and prunes the noisy points based on the maximization of mutual information against input data distributions such that the resulting clusters are least affected by noise and outliers, where the degree of robustness is controlled through a separate parameter to make a trade-off between rejection of noisy points and optimal clustered data. The pruning approach is general, and it can improve the robustness of many existing traditional clustering methods. In particular, we apply the pruning approach to improve the robustness of fuzzy c-means clustering and its extensions, e.g., fuzzy c-spherical shells clustering and kernel-based fuzzy c-means clustering. As a result, we obtain three clustering algorithms that are the robust versions of the existing ones. The effectiveness of the proposed pruning approach is supported by experimental results.     

Bayesian zero-order TSK fuzzy system modeling

Different from the existing TSK fuzzy system modeling methods, a novel zero-order TSK fuzzy modeling method called Bayesian zero-order TSK fuzzy system (B-ZTSK-FS) is proposed from the perspective of Bayesian inference in this paper. The proposed method B-ZTSK-FS constructs zero-order TSK fuzzy system by using the maximum a posteriori (MAP) framework to maximize the corresponding posteriori probability. First, a joint likelihood model about zero-order TSK fuzzy system is defined to derive a new objective function which can assure that both antecedents and consequents of fuzzy rules rather than only their antecedents of the most existing TSK fuzzy systems become interpretable. The defined likelihood model is composed of three aspects: clustering on the training set for antecedents of fuzzy rules, the least squares (LS) error for consequent parameters of fuzzy rules, and a Dirichlet prior distribution for fuzzy cluster memberships which is considered to not only automatically match the “sum-to-one” constraints on fuzzy cluster memberships, but also make the proposed method B-ZTSK-FS scalable for large-scale datasets by appropriately setting the Dirichlet index. This likelihood model indeed indicates that antecedent and consequent parameters of fuzzy rules can be linguistically interpreted and simultaneously optimized by the proposed method B-ZTSK-FS which is based on the MAP framework with the iterative sampling algorithm, which in fact implies that fuzziness and probability can co-jointly work for TSK fuzzy system modeling in a collaborative rather than repulsive way. Finally, experimental results on 28 synthetic and real-world datasets are reported to demonstrate the effectiveness of the proposed method B-ZTSK-FS in the sense of approximation accuracy, interpretability and scalability.     

A robust learning algorithm based on support vector regression and robust fuzzy cerebellar model articulation controller

For real-world applications, the obtained data are always subject to noise or outliers. The learning mechanism of cerebellar model articulation controller (CMAC), a neurological model, is to imitate the cerebellum of human being. CMAC has an attractive property of learning speed in which a small subset addressed by the input space determines output instantaneously. For fuzzy cerebellar model articulation controller (FCMAC), the concept of fuzzy is incorporated into CMAC to improve the accuracy problem. However, the distributions of errors into the addressed hypercubes may cause unacceptable learning performance for input data with noise or outliers. For robust fuzzy cerebellar model articulation controller (RFCMAC), the robust learning of M-estimator can be embedded into FCMAC to degrade noise or outliers. Meanwhile, support vector machine (SVR) is a machine learning theory based algorithm which has been applied successfully to a number of regression problems when noise or outliers exist. Unfortunately, the practical application of SVR is limited to defining a set of parameters for obtaining admirable performance by the user. In this paper, a robust learning algorithm based on support SVR and RFCMAC is proposed. The proposed algorithm has both the advantage of SVR, the ability to avoid corruption effects, and the advantage of RFCMAC, the ability to obtain attractive properties of learning performance and to increase accurate approximation. Additionally, particle swarm optimization (PSO) is applied to obtain the best parameters setting for SVR. From simulation results, it shows that the proposed algorithm outperforms other algorithms.     

The annealing robust backpropagation (ARBP) learning algorithm

Multilayer feedforward neural networks are often referred to as universal approximators. Nevertheless, if the used training data are corrupted by large noise, such as outliers, traditional backpropagation learning schemes may not always come up with acceptable performance. Even though various robust learning algorithms have been proposed in the literature, those approaches still suffer from the initialization problem. In those robust learning algorithms, the so-called M-estimator is employed. For the M-estimation type of learning algorithms, the loss function is used to play the role in discriminating against outliers from the majority by degrading the effects of those outliers in learning. However, the loss function used in those algorithms may not correctly discriminate against those outliers. In the paper, the annealing robust backpropagation learning algorithm (ARBP) that adopts the annealing concept into the robust learning algorithms is proposed to deal with the problem of modeling under the existence of outliers. The proposed algorithm has been employed in various examples. Those results all demonstrated the superiority over other robust learning algorithms independent of outliers. In the paper, not only is the annealing concept adopted into the robust learning algorithms but also the annealing schedule k/t was found experimentally to achieve the best performance among other annealing schedules, where k is a constant and t is the epoch number.     

Online identification of evolving Takagi–Sugeno–Kang fuzzy models for crane systems

This paper suggests new evolving Takagi–Sugeno–Kang (TSK) fuzzy models dedicated to crane systems. A set of evolving TSK fuzzy models with different numbers of inputs are derived by the novel relatively simple and transparent implementation of an online identification algorithm. An input selection algorithm to guide modeling is proposed on the basis of ranking the inputs according to their important factors after the first step of the online identification algorithm. The online identification algorithm offers rule bases and parameters which continuously evolve by adding new rules with more summarization power and by modifying existing rules and parameters. The potentials of new data points are used with this regard. The algorithm is applied in the framework of the pendulum–crane system laboratory equipment. The evolving TSK fuzzy models are tested against the experimental data and a comparison with other TSK fuzzy models and modeling approaches is carried out. The comparison points out that the proposed evolving TSK fuzzy models are simple and consistent with both training data and testing data and that these models outperform other TSK fuzzy models.     

Hybrid robust support vector machines for regression with outliers

In this study, a hybrid robust support vector machine for regression is proposed to deal with training data sets with outliers. The proposed approach consists of two stages of strategies. The first stage is for data preprocessing and a support vector machine for regression is used to filter out outliers in the training data set. Since the outliers in the training data set are removed, the concept of robust statistic is not needed for reducing the outliers’ effects in the later stage. Then, the training data set except for outliers, called as the reduced training data set, is directly used in training the non-robust least squares support vector machines for regression (LS-SVMR) or the non-robust support vector regression networks (SVRNs) in the second stage. Consequently, the learning mechanism of the proposed approach is much easier than that of the robust support vector regression networks (RSVRNs) approach and of the weighted LS-SVMR approach. Based on the simulation results, the performance of the proposed approach with non-robust LS-SVMR is superior to the weighted LS-SVMR approach when the outliers exist. Moreover, the performance of the proposed approach with non-robust SVRNs is also superior to the RSVRNs approach.     

基于模糊子空间聚类的0 阶岭回归TSK 模糊系统

经典数据驱动型TSK模糊系统在利用高维数据训练模型时,由于规则前件采用的特征过多,导致规则的解释性和简洁性下降.对此,根据模糊子空间聚类算法的子空间特性,为TSK模型添加特征抽取机制,并进一步利用岭回归实现后件的学习,提出一种基于模糊子空间聚类的0阶岭回归TSK模型构建方法.该方法不仅能为规则抽取出重要子空间特征,而且可为不同规则抽取不同的特征.在模拟和真实数据集上的实验结果验证了所提出方法的优势.     

Robust clustering by pruning outliers

In many applications of C-means clustering, the given data set often contains noisy points. These noisy points will affect the resulting clusters, especially if they are far away from the data points. In this paper, we develop a pruning approach for robust C-means clustering. This approach identifies and prunes the outliers based on the sizes and shapes of the clusters so that the resulting clusters are least affected by the outliers. The pruning approach is general, and it can improve the robustness of many existing C-means clustering methods. In particular, we apply the pruning approach to improve the robustness of hard C-means clustering, fuzzy C-means clustering, and deterministic-annealing C-means clustering. As a result, we obtain three clustering algorithms that are the robust versions of the existing ones. In addition, we integrate the pruning approach with the fuzzy approach and the possibilistic approach to design two new algorithms for robust C-means clustering. The numerical results demonstrate that the pruning approach can achieve good robustness.     

Mining Projected Clusters in High-Dimensional Spaces

Clustering high-dimensional data has been a major challenge due to the inherent sparsity of the points. Most existing clustering algorithms become substantially inefficient if the required similarity measure is computed between data points in the full-dimensional space. To address this problem, a number of projected clustering algorithms have been proposed. However, most of them encounter difficulties when clusters hide in subspaces with very low dimensionality. These challenges motivate our effort to propose a robust partitional distance-based projected clustering algorithm. The algorithm consists of three phases. The first phase performs attribute relevance analysis by detecting dense and sparse regions and their location in each attribute. Starting from the results of the first phase, the goal of the second phase is to eliminate outliers, while the third phase aims to discover clusters in different subspaces. The clustering process is based on the K-means algorithm, with the computation of distance restricted to subsets of attributes where object values are dense. Our algorithm is capable of detecting projected clusters of low dimensionality embedded in a high-dimensional space and avoids the computation of the distance in the full-dimensional space. The suitability of our proposal has been demonstrated through an empirical study using synthetic and real datasets.     

Alpha-cut implemented fuzzy clustering algorithms and switching regressions.

In the fuzzy c-means (FCM) clustering algorithm, almost none of the data points have a membership value of 1. Moreover, noise and outliers may cause difficulties in obtaining appropriate clustering results from the FCM algorithm. The embedding of FCM into switching regressions, called the fuzzy c-regressions (FCRs), still has the same drawbacks as FCM. In this paper, we propose the alpha-cut implemented fuzzy clustering algorithms, referred to as FCMalpha, which allow the data points being able to completely belong to one cluster. The proposed FCMalpha algorithms can form a cluster core for each cluster, where data points inside a cluster core will have a membership value of 1 so that it can resolve the drawbacks of FCM. On the other hand, the fuzziness index m plays different roles for FCM and FCMalpha. We find that the clustering results obtained by FCMalpha are more robust to noise and outliers than FCM when a larger m is used. Moreover, the cluster cores generated by FCMalpha are workable for various data shape clusters, so that FCMalpha is very suitable for embedding into switching regressions. The embedding of FCMalpha into switching regressions is called FCRalpha. The proposed FCRalpha provides better results than FCR for environments with noise or outliers. Numerical examples show the robustness and the superiority of our proposed methods.     

Enhanced neural gas network for prototype-based clustering

In practical cluster analysis tasks, an efficient clustering algorithm should be less sensitive to parameter configurations and tolerate the existence of outliers. Based on the neural gas (NG) network framework, we propose an efficient prototype-based clustering (PBC) algorithm called enhanced neural gas (ENG) network. Several problems associated with the traditional PBC algorithms and original NG algorithm such as sensitivity to initialization, sensitivity to input sequence ordering and the adverse influence from outliers can be effectively tackled in our new scheme. In addition, our new algorithm can establish the topology relationships among the prototypes and all topology-wise badly located prototypes can be relocated to represent more meaningful regions. Experimental results¹on synthetic and UCI datasets show that our algorithm possesses superior performance in comparison to several PBC algorithms and their improved variants, such as hard c-means, fuzzy c-means, NG, fuzzy possibilistic c-means, credibilistic fuzzy c-means, hard/fuzzy robust clustering and alternative hard/fuzzy c-means, in static data clustering tasks with a fixed number of prototypes.