Rough Neural Network Based on Bottom-Up Fuzzy Rough Data Analysis

Based on bottom-up fuzzy rough data analysis, a new rough neural network decision-making model is proposed. Through supervised Gaustafason–Kessel (G–K) clustering algorithm, proper fuzzy clusters are found to partition the input data space. At the same time cluster number is searched by monotone increasing process. If the cluster number matches with that exactly exist in data sets then excellent fuzzy rough data modeling (FRDM) model can be built. And by integrating it with neural network technique, corresponding rough neural network is constructed. Our method overcomes the defects of conventional top-down based rough logic neural network (RLNN) method, and it also achieves adaptive learning ability and comprehensive soft decision-making ability compared with FRDM model. The experiment results indicate that our method has stronger generalization ability and more compact network structure than conventional RLNN.     

Extension neural network-type 2 and its applications

A supervised learning pattern classifier, called the extension neural network (ENN), has been described in a recent paper. In this sequel, the unsupervised learning pattern clustering sibling called the extension neural network type 2 (ENN-2) is proposed. This new neural network uses an extension distance (ED) to measure the similarity between data and the cluster center. It does not require an initial guess of the cluster center coordinates, nor of the initial number of clusters. The clustering process is controlled by a distanced parameter and by a novel extension distance. It shows the same capability as human memory systems to keep stability and plasticity characteristics at the same time, and it can produce meaningful weights after learning. Moreover, the structure of the proposed ENN-2 is simpler and the learning time is shorter than traditional neural networks. Experimental results from five different examples, including three benchmark data sets and two practical applications, verify the effectiveness and applicability of the proposed work.     

Clustering of Binary Market Research Data

This paper details research carried out into the clustering of large market research data sets. The aim of the analysis was to find clusters of vectors in the data with maximum similarity expressed in the form of identical question answers. A novel paradigm (the interrogative memory structure) has been developed and compared with current unsupervised artificial neural network and statistical clustering techniques. The interrogative memory structure is a bi-directional network, which, when utilised with the controlling algorithm detailed in this paper, has produced encouraging results with initial tests. A detailed account of the paradigm is given along with an analysis of the initial results.     

Transfer learning using the online Fuzzy Min–Max neural network

In this paper, we present an empirical analysis on transfer learning using the Fuzzy Min–Max (FMM) neural network with an online learning strategy. Three transfer learning benchmark data sets, i.e., 20 Newsgroups, WiFi Time, and Botswana, are used for evaluation. In addition, the data samples are corrupted with white Gaussian noise up to 50 %, in order to assess the robustness of the online FMM network in handling noisy transfer learning tasks. The results are analyzed and compared with those from other methods. The outcomes indicate that the online FMM network is effective for undertaking transfer learning tasks in noisy environments.     

Fuzzy basis functions: comparisons with other basis functions

Fuzzy basis functions (FBF's) which have the capability of combining both numerical data and linguistic information, are compared with other basis functions. Because a FBF network is different from other networks in that it is the only one that can combine numerical and linguistic information, comparisons are made when only numerical data is available. In particular, a FBF network is compared with a radial basis function (RBF) network from the viewpoint of function approximation. Their architectural interrelationships are discussed. Additionally, a RBF network, which is implemented using a regularization technique, is compared with a FBF network from the viewpoint of overcoming ill-posed problems. A FBF network is also compared with Specht's probabilistic neural network and his general regression neural network (GRNN) from an architectural point of view. A FBF network is also compared with a Gaussian sum approximation in which Gaussian functions play a central role. Finally, we summarize the architectural relationships between all the networks discussed in this paper     

A self-organizing network for hyperellipsoidal clustering (HEC)

We propose a self-organizing network for hyperellipsoidal clustering (HEC). It consists of two layers. The first employs a number of principal component analysis subnetworks to estimate the hyperellipsoidal shapes of currently formed clusters. The second performs competitive learning using the cluster shape information from the first. The network performs partitional clustering using the proposed regularized Mahalanobis distance, which was designed to deal with the problems in estimating the Mahalanobis distance when the number of patterns in a cluster is less than or not considerably larger than the dimensionality of the feature space during clustering. This distance also achieves a tradeoff between hyperspherical and hyperellipsoidal cluster shapes so as to prevent the HEC network from producing unusually large or small clusters. The significance level of the Kolmogorov-Smirnov test on the distribution of the Mahalanobis distances of patterns in a cluster to the cluster center under the Gaussian cluster assumption is used as a compactness measure. The HEC network has been tested on a number of artificial data sets and real data sets, We also apply the HEC network to texture segmentation problems. Experiments show that the HEC network leads to a significant improvement in the clustering results over the K-means algorithm with Euclidean distance. Our results on real data sets also indicate that hyperellipsoidal shaped clusters are often encountered in practice.     

A new multiobjective clustering technique based on the concepts of stability and symmetry

Most clustering algorithms operate by optimizing (either implicitly or explicitly) a single measure of cluster solution quality. Such methods may perform well on some data sets but lack robustness with respect to variations in cluster shape, proximity, evenness and so forth. In this paper, we have proposed a multiobjective clustering technique which optimizes simultaneously two objectives, one reflecting the total cluster symmetry and the other reflecting the stability of the obtained partitions over different bootstrap samples of the data set. The proposed algorithm uses a recently developed simulated annealing-based multiobjective optimization technique, named AMOSA, as the underlying optimization strategy. Here, points are assigned to different clusters based on a newly defined point symmetry-based distance rather than the Euclidean distance. Results on several artificial and real-life data sets in comparison with another multiobjective clustering technique, MOCK, three single objective genetic algorithm-based automatic clustering techniques, VGAPS clustering, GCUK clustering and HNGA clustering, and several hybrid methods of determining the appropriate number of clusters from data sets show that the proposed technique is well suited to detect automatically the appropriate number of clusters as well as the appropriate partitioning from data sets having point symmetric clusters. The performance of AMOSA as the underlying optimization technique in the proposed clustering algorithm is also compared with PESA-II, another evolutionary multiobjective optimization technique.     

A NON-PARAMETRIC APPROACH TO SIMPLICITY CLUSTERING

The simplicity principle—an updating of Ockham's razor to take into account modern information theory—states that the preferred theory for a set of data is the one that allows for the most efficient encoding of the data. We consider this in the context of classification, or clustering, as a data reduction technique that helps describe a set of objects by dividing the objects into groups. The simplicity model we present favors clusters such that the similarity of the items in the clusters is maximal, while the similarity of items between clusters is minimal. Several novel features of our clustering criterion make it especially appropriate for clustering of data derived from, psychological procedures (e.g., similarity ratings): It is non-parametric, and may be applied in situations where the metric axioms are violated without requiring (information-forgetting) transformation procedures. We illustrate the use of the criterion with a selection of data sets. A distinctive aspect of this research is that it motivates a clustering algorithm from psychological principles.     

A feed forward deep neural network model using feature selection for cloud intrusion detection system

The rapid advancement and growth of technology have rendered cloud computing services indispensable to our activities. Threats and intrusions have since multiplied exponentially across a range of industries. In such a scenario, the intrusion detection system, or simply the IDS, is deployed on the network to monitor and detect any attacks. The paper proposes a feed-forward deep neural network (FFDNN) method based on deep learning methodology using a filter-based feature selection model. The feature selection strategy aims to determine and select the most highly relevant subset of attributes from the feature importance score for training the deep learning model. Three benchmark data sets were used to assess the experiment: CIC-IDS 2017, UNSW-NB15, and NSL-KDD. In order to justify the proposed technique, a comparison was done using other learning algorithms ranging from classical machine learning to ensemble learning methods that can detect various attacks. The experiments showed that the FFDNN model with reduced feature subsets gave the highest accuracy of 99.53% and 94.45% in the NSL-KDD and UNSW-NB15 data sets, while the ensemble-based XGBoost model performed better in the CIC-IDS 2017 data set. In addition, the results show that the overall accuracy, recall, and F1 score of the deep learning algorithm are generally better for all the data sets.     

Parameterized attribute reduction with Gaussian kernel based fuzzy rough sets

Fuzzy rough sets are considered as an effective tool to deal with uncertainty in data analysis, and fuzzy similarity relations are used in fuzzy rough sets to calculate similarity between objects. On the other hand in kernel tricks, a kernel maps data into a higher dimensional feature space where the resulting structure of the learning task is linearly separable, while the kernel is the inner product of this feature space and can also be viewed as a similarity function. It has been reported there is an overlap between family of kernels and collection of fuzzy similarity relations. This fact motivates the idea in this paper to use some kernels as fuzzy similarity relations and develop kernel based fuzzy rough sets. First, we consider Gaussian kernel and propose Gaussian kernel based fuzzy rough sets. Second we introduce parameterized attribute reduction with the derived model of fuzzy rough sets. Structures of attribute reduction are investigated and an algorithm with discernibility matrix to find all reducts is developed. Finally, a heuristic algorithm is designed to compute reducts with Gaussian kernel fuzzy rough sets. Several experiments are provided to demonstrate the effectiveness of the idea.     

A new neural network for cluster-detection-and-labeling

We propose an unsupervised neural net which clusters a set of experimental data according to a given generic interpoint similarity measure, and then assigns to each new input its appropriate cluster label. The network can do this for clusters of any shape, and without knowing in advance the number of clusters to be created. We call this two-layer net a cluster-detection-and-labeling (CDL) net. In it, the concept of similarity and closeness with regard to distance are combined. Specifically, clusters are represented by a set of prototypes, and the similarities between an input vector and these prototypes are calculated as inner products of these vectors compared to some thresholds. These thresholds, which depend on the distance between the input vector and the prototype, are calculated in a separate threshold calculating unit. The data are cycled through the network several times. At the end of each cycle the clusters are evaluated, and only those with more than a specified number of samples are retained. The others are fed back through the updated network. This process terminates according to a suitable criterion, such as when a prespecified portion of the data are classified. The performance of the CDL network has been compared with that of the winner-take-all (WTA) network for several different cluster structures, since the latter is widely used in cluster analysis applications. These studies demonstrate that the new network performs well for all the tested cluster shapes, also for those cases where the WTA network completely fails.     

用神经网络驱动的模糊推理入侵检测方法

提出了神经网络驱动模糊推理的入侵检测方法,利用神经网络的学习能力,对不清楚规则的复杂系统的输入输出特性进行适当的非线性划分,自动形成舰则集和相应的隶属关系,克服了在多维空间上经验性的确定隶属函数的困难。对于神经网络的训练数据,采用人工数据,克服了神经网络监督学习和获取已知输出的训练数据的困难。试验证明,这种技术具有很好的灵敏度和鲁棒性,而且,能够检测出未知的入侵行为。     

A novel soft cluster neural network for the classification of suspicious areas in digital mammograms

This paper presents a novel soft cluster neural network technique for the classification of suspicious areas in digital mammograms. The technique introduces the concept of soft clusters within a neural network layer and combines them with least squares for optimising neural network weights. The idea of soft clusters is proposed in order to increase the generalisation ability of the neural network by providing a mechanism to more aptly depict the relationship between the input features and the subsequent classification as either a benign or malignant class. Soft clusters with least squares make the training process faster and avoid iterative processes which have many problems. The proposed neural network technique has been tested on the DDSM benchmark database. The results are analysed and discussed in this paper.     

An automatic shape independent clustering technique

This article describes a clustering technique that can automatically detect any number of well-separated clusters which may be of any shape, convex and/or non-convex. This is in contrast to most other techniques which assume a value for the number of clusters and/or a particular cluster structure. The proposed technique is based on an iterative partitioning of the relative neighborhood graph, coupled with a post-processing step for merging small clusters. Techniques for improving the efficiency of the proposed scheme are implemented. The clustering scheme is able to detect outliers in data. It is also able to indicate the inherent hierarchical nature of the clusters present in a data set. Moreover, the proposed technique is also able to identify the situation when the data do not have any natural clusters at all. Results demonstrating the effectiveness of the clustering scheme are provided for several data sets.     

CoBi: Pattern Based Co-Regulated Biclustering of Gene Expression Data

Co-regulation is a common phenomenon in gene expression. Finding positively and negatively co-regulated gene clusters from gene expression data is a real need. Existing techniques based on global similarity are unable to detect true up- and down-regulated gene clusters. This paper presents an expression pattern based biclustering technique, CoBi, for grouping both positively and negatively regulated genes from microarray expression data. Regulation pattern and similarity in degree of fluctuation are accounted for while computing similarity between two genes. Unlike traditional biclustering techniques, which use greedy iterative approaches, it uses a BiClust tree that needs single pass over the entire dataset to find a set of biologically relevant biclusters. Biclusters determined from different gene expression datasets by the technique show highly enriched functional categories.     

A symmetry based multiobjective clustering technique for automatic evolution of clusters

In this paper the problem of automatic clustering a data set is posed as solving a multiobjective optimization (MOO) problem, optimizing a set of cluster validity indices simultaneously. The proposed multiobjective clustering technique utilizes a recently developed simulated annealing based multiobjective optimization method as the underlying optimization strategy. Here variable number of cluster centers is encoded in the string. The number of clusters present in different strings varies over a range. The points are assigned to different clusters based on the newly developed point symmetry based distance rather than the existing Euclidean distance. Two cluster validity indices, one based on the Euclidean distance, XB-index, and another recently developed point symmetry distance based cluster validity index, Sym-index, are optimized simultaneously in order to determine the appropriate number of clusters present in a data set. Thus the proposed clustering technique is able to detect both the proper number of clusters and the appropriate partitioning from data sets either having hyperspherical clusters or having point symmetric clusters. A new semi-supervised method is also proposed in the present paper to select a single solution from the final Pareto optimal front of the proposed multiobjective clustering technique. The efficacy of the proposed algorithm is shown for seven artificial data sets and six real-life data sets of varying complexities. Results are also compared with those obtained by another multiobjective clustering technique, MOCK, two single objective genetic algorithm based automatic clustering techniques, VGAPS clustering and GCUK clustering.     

Neural Learning from Unbalanced Data

This paper describes the result of our study on neural learning to solve the classification problems in which data is unbalanced and noisy. We conducted the study on three different neural network architectures, multi-layered Back Propagation, Radial Basis Function, and Fuzzy ARTMAP using three different training methods, duplicating minority class examples, Snowball technique and multidimensional Gaussian modeling of data noise. Three major issues are addressed: neural learning from unbalanced data examples, neural learning from noisy data, and making intentional biased decisions. We argue that by properly generated extra training data examples around the noise densities, we can train a neural network that has a stronger capability of generalization and better control of the classification error of the trained neural network. In particular, we focus on problems that require a neural network to make favorable classification to a particular class such as classifying normal(pass)/abnormal(fail) vehicles in an assembly plant. In addition, we present three methods that quantitatively measure the noise level of a given data set. All experiments were conducted using data examples downloaded directly from test sites of an automobile assembly plant. The experimental results showed that the proposed multidimensional Gaussian noise modeling algorithm was very effective in generating extra data examples that can be used to train a neural network to make favorable decisions for the minority class and to have increased generalization capability.     

DCT-Yager FNN: a novel Yager-based fuzzy neural network with the discrete clustering technique.

Earlier clustering techniques such as the modified learning vector quantization (MLVQ) and the fuzzy Kohonen partitioning (FKP) techniques have focused on the derivation of a certain set of parameters so as to define the fuzzy sets in terms of an algebraic function. The fuzzy membership functions thus generated are uniform, normal, and convex. Since any irregular training data is clustered into uniform fuzzy sets (Gaussian, triangular, or trapezoidal), the clustering may not be exact and some amount of information may be lost. In this paper, two clustering techniques using a Kohonen-like self-organizing neural network architecture, namely, the unsupervised discrete clustering technique (UDCT) and the supervised discrete clustering technique (SDCT), are proposed. The UDCT and SDCT algorithms reduce this data loss by introducing nonuniform, normal fuzzy sets that are not necessarily convex. The training data range is divided into discrete points at equal intervals, and the membership value corresponding to each discrete point is generated. Hence, the fuzzy sets obtained contain pairs of values, each pair corresponding to a discrete point and its membership grade. Thus, it can be argued that fuzzy membership functions generated using this kind of a discrete methodology provide a more accurate representation of the actual input data. This fact has been demonstrated by comparing the membership functions generated by the UDCT and SDCT algorithms against those generated by the MLVQ, FKP, and pseudofuzzy Kohonen partitioning (PFKP) algorithms. In addition to these clustering techniques, a novel pattern classifying network called the Yager fuzzy neural network (FNN) is proposed in this paper. This network corresponds completely to the Yager inference rule and exhibits remarkable generalization abilities. A modified version of the pseudo-outer product (POP)-Yager FNN called the modified Yager FNN is introduced that eliminates the drawbacks of the earlier network and yi- elds superior performance. Extensive experiments have been conducted to test the effectiveness of these two networks, using various clustering algorithms. It follows that the SDCT and UDCT clustering algorithms are particularly suited to networks based on the Yager inference rule.     

Quantum and impulse noise filtering from breast mammogram images

Recent advances in the field of image processing have shown that level of noise highly affect the quality and accuracy of classification when working with mammographic images. In this paper, we have proposed a method that consists of two major modules: noise detection and noise filtering. For detection purpose, neural network is used which effectively detect the noise from highly corrupted images. Pixel values of the window and some other features are used as feature for the training of neural network. For noise removal, three filters are used. The weighted average value of these three filters is filled on noisy pixels. The proposed technique has been tested on salt & pepper and quantum noise present in mammogram images. Peak signal to noise ratio (PSNR) and structural similarity index measure (SSIM) are used for comparison of proposed technique with different existing techniques. Experiments shows that proposed technique produce better results as compare to existing methods.     

A generalized automatic clustering algorithm in a multiobjective framework

In this paper a new multiobjective (MO) clustering technique (GenClustMOO) is proposed which can automatically partition the data into an appropriate number of clusters. Each cluster is divided into several small hyperspherical subclusters and the centers of all these small sub-clusters are encoded in a string to represent the whole clustering. For assigning points to different clusters, these local sub-clusters are considered individually. For the purpose of objective function evaluation, these sub-clusters are merged appropriately to form a variable number of global clusters. Three objective functions, one reflecting the total compactness of the partitioning based on the Euclidean distance, the other reflecting the total symmetry of the clusters, and the last reflecting the cluster connectedness, are considered here. These are optimized simultaneously using AMOSA, a newly developed simulated annealing based multiobjective optimization method, in order to detect the appropriate number of clusters as well as the appropriate partitioning. The symmetry present in a partitioning is measured using a newly developed point symmetry based distance. Connectedness present in a partitioning is measured using the relative neighborhood graph concept. Since AMOSA, as well as any other MO optimization technique, provides a set of Pareto-optimal solutions, a new method is also developed to determine a single solution from this set. Thus the proposed GenClustMOO is able to detect the appropriate number of clusters and the appropriate partitioning from data sets having either well-separated clusters of any shape or symmetrical clusters with or without overlaps. The effectiveness of the proposed GenClustMOO in comparison with another recent multiobjective clustering technique (MOCK), a single objective genetic algorithm based automatic clustering technique (VGAPS-clustering), K-means and single linkage clustering techniques is comprehensively demonstrated for nineteen artificial and seven real-life data sets of varying complexities. In a part of the experiment the effectiveness of AMOSA as the underlying optimization technique in GenClustMOO is also demonstrated in comparison to another evolutionary MO algorithm, PESA2.