A NEW FUZZY-DATA MINING METHOD FOR PATTERN CLASSIFICATION BY PRINCIPAL COMPONENT ANALYSIS

ABSTRACT

Data mining techniques can be used to discover useful information by exploring and analyzing data. The aim of this article is to propose a new fuzzy-data mining method to find a compact set consisting of fuzzy if-then classification rules with high classification capability using the genetic algorithm. Furthermore, for not reducing the usefulness of the proposed method for classification problems with high dimensional feature space, the curse dimensionality resulting from the grid partition is overcome in the proposed method by employing the principal component analysis to reduce the dimensions. Through computer simulations, it can be seen that the proposed method is comparable to the other fuzzy classification methods on the well-known iris data, the appendicitis data, and the cancer data.     

基于数值型数据的模糊规则快速挖掘方法

针对数值型数据能准确反应现实世界,但难以理解的问题,为了从数值型数据中挖掘出易于理解的知识,提出了基于数值型数据的模糊规则快速挖掘方法。该方法能从数值型数据中挖掘出一个零阶的Sugeno模糊规则,并采用一种启发式方法将这个零阶的Sugeno模糊规则的数值结论转变为两个带置信度的语言变量,并给出了规则库的存储结构。最后通过实例证明了这种快速模糊规则挖掘方法能避免复杂的数值型计算和能有效逼近非线性函数的优点．     

Performance evaluation of fuzzy rule-based classification systems obtained by multi-objective genetic algorithms

In this paper, we examine the classification performance of fuzzy if-then rules selected by a GA-based multi-objective rule selection method. This rule selection method can be applied to high-dimensional pattern classification problems with many continuous attributes by restricting the number of antecedent conditions of each candidate fuzzy if-then rule. As candidate rules, we only use fuzzy if-then rules with a small number of antecedent conditions. Thus it is easy for human users to understand each rule selected by our method. Our rule selection method has two objectives: to minimize the number of selected fuzzy if-then rules and to maximize the number of correctly classified patterns. In our multi-objective fuzzy rule selection problem, there exist several solutions (i.e., several rule sets) called “non-dominated solutions” because two conflicting objectives are considered. In this paper, we examine the performance of our GA-based rule selection method by computer simulations on a real-world pattern classification problem with many continuous attributes. First we examine the classification performance of our method for training patterns by computer simulations. Next we examine the generalization ability for test patterns. We show that a fuzzy rule-based classification system with an appropriate number of rules has high generalization ability.     

Performance evaluation of fuzzy classifier systems formultidimensional pattern classification problems

We examine the performance of a fuzzy genetics-based machine learning method for multidimensional pattern classification problems with continuous attributes. In our method, each fuzzy if-then rule is handled as an individual, and a fitness value is assigned to each rule. Thus, our method can be viewed as a classifier system. In this paper, we first describe fuzzy if-then rules and fuzzy reasoning for pattern classification problems. Then we explain a genetics-based machine learning method that automatically generates fuzzy if-then rules for pattern classification problems from numerical data. Because our method uses linguistic values with fixed membership functions as antecedent fuzzy sets, a linguistic interpretation of each fuzzy if-then rule is easily obtained. The fixed membership functions also lead to a simple implementation of our method as a computer program. The simplicity of implementation and the linguistic interpretation of the generated fuzzy if-then rules are the main characteristic features of our method. The performance of our method is evaluated by computer simulations on some well-known test problems. While our method involves no tuning mechanism of membership functions, it works very well in comparison with other classification methods such as nonfuzzy machine learning techniques and neural networks.     

An adaptive neural fuzzy filter and its applications

A new kind of nonlinear adaptive filter, the adaptive neural fuzzy filter (ANFF), based upon a neural network's learning ability and fuzzy if-then rule structure, is proposed in this paper. The ANFF is inherently a feedforward multilayered connectionist network which can learn by itself according to numerical training data or expert knowledge represented by fuzzy if-then rules. The adaptation here includes the construction of fuzzy if-then rules (structure learning), and the tuning of the free parameters of membership functions (parameter learning). In the structure learning phase, fuzzy rules are found based on the matching of input-output clusters. In the parameter learning phase, a backpropagation-like adaptation algorithm is developed to minimize the output error. There are no hidden nodes (i.e., no membership functions and fuzzy rules) initially, and both the structure learning and parameter learning are performed concurrently as the adaptation proceeds. However, if some linguistic information about the design of the filter is available, such knowledge can be put into the ANFF to form an initial structure with hidden nodes. Two major advantages of the ANFF can thus be seen: 1) a priori knowledge can be incorporated into the ANFF which makes the fusion of numerical data and linguistic information in the filter possible; and 2) no predetermination, like the number of hidden nodes, must be given, since the ANFF can find its optimal structure and parameters automatically     

Selecting fuzzy if-then rules for classification problems usinggenetic algorithms

This paper proposes a genetic-algorithm-based method for selecting a small number of significant fuzzy if-then rules to construct a compact fuzzy classification system with high classification power. The rule selection problem is formulated as a combinatorial optimization problem with two objectives: to maximize the number of correctly classified patterns and to minimize the number of fuzzy if-then rules. Genetic algorithms are applied to this problem. A set of fuzzy if-then rules is coded into a string and treated as an individual in genetic algorithms. The fitness of each individual is specified by the two objectives in the combinatorial optimization problem. The performance of the proposed method for training data and test data is examined by computer simulations on the iris data of Fisher     

Modeling Connectionist Neuro-Fuzzy network and Applications

The main objective of this paper is to propose a Neuro-Fuzzy network, which can model a system from input–output data by automatically dividing the input–output space and extracting fuzzy if-then rules from numerical data. The structure of the network is simple with input, rule and output layers only. The connections between input and rule layer is used to derive the membership functions of the fuzzified part. Kohonens self-organizing learning algorithm is applied to partition the pattern space. Using this algorithm, similar rules are mapped close by and extraction of if-then rules is made easy. It can also adapt to a number of rules automatically. The proposed network is verified for three benchmark applications. Experimental results show that the adaptive method discussed in this paper not only trains in a few hundred iterations but also provides better performance measures when compared to conventional methods.     

An approach to fuzzy default reasoning for function approximation

This paper discusses fuzzy reasoning for approximately realizing nonlinear functions by a small number of fuzzy if-then rules with different specificity levels. Our fuzzy rule base is a mixture of general and specific rules, which overlap with each other in the input space. General rules work as default rules in our fuzzy rule base. First, we briefly describe existing approaches to the handling of default rules in the framework of possibility theory. Next, we show that standard interpolation-based fuzzy reasoning leads to counterintuitive results when general rules include specific rules with different consequents. Then, we demonstrate that intuitively acceptable results are obtained from a non-standard inclusion-based fuzzy reasoning method. Our approach is based on the preference for more specific rules, which is a commonly used idea in the field of default reasoning. When a general rule includes a specific rule and they are both compatible with an input vector, the weight of the general rule is discounted in fuzzy reasoning. We also discuss the case where general rules do not perfectly but partially include specific rules. Then we propose a genetics-based machine learning (GBML) algorithm for extracting a small number of fuzzy if-then rules with different specificity levels from numerical data using our inclusion-based fuzzy reasoning method. Finally, we describe how our approach can be applied to the approximate realization of fuzzy number-valued nonlinear functions     

A neural fuzzy system with fuzzy supervised learning

A neural fuzzy system learning with fuzzy training data (fuzzy if-then rules) is proposed in this paper. This system is able to process and learn numerical information as well as linguistic information. At first, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use alpha-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, a fuzzy supervised learning algorithm is developed for the proposed system. It extends the normal supervised learning techniques to the learning problems where only linguistic teaching signals are available. The fuzzy supervised learning scheme can train the proposed system with desired fuzzy input-output pairs which are fuzzy numbers instead of the normal numerical values. With fuzzy supervised learning, the proposed system can be used for rule base concentration to reduce the number of rules in a fuzzy rule base. Simulation results are presented to illustrate the performance and applicability of the proposed system.     

Mining fuzzy specific rare itemsets for education data

Association rule mining is an important data analysis method for the discovery of associations within data. There have been many studies focused on finding fuzzy association rules from transaction databases. Unfortunately, in the real world, one may have available relatively infrequent data, as well as frequent data. From infrequent data, we can find a set of rare itemsets that will be useful for teachers to find out which students need extra help in learning. While the previous association rules discovery techniques are able to discover some rules based on frequency, this is insufficient to determine the importance of a rule composed of frequency-based data items. To remedy this problem, we develop a new algorithm based on the Apriori approach to mine fuzzy specific rare itemsets from quantitative data. Finally, fuzzy association rules can be generated from these fuzzy specific rare itemsets. The patterns are useful to discover learning problems. Experimental results show that the proposed approach is able to discover interesting and valuable patterns from the survey data.     

Mining fuzzy association rules from questionnaire data

Association rule mining is one of most popular data analysis methods that can discover associations within data. Association rule mining algorithms have been applied to various datasets, due to their practical usefulness. Little attention has been paid, however, on how to apply the association mining techniques to analyze questionnaire data. Therefore, this paper first identifies the various data types that may appear in a questionnaire. Then, we introduce the questionnaire data mining problem and define the rule patterns that can be mined from questionnaire data. A unified approach is developed based on fuzzy techniques so that all different data types can be handled in a uniform manner. After that, an algorithm is developed to discover fuzzy association rules from the questionnaire dataset. Finally, we evaluate the performance of the proposed algorithm, and the results indicate that our method is capable of finding interesting association rules that would have never been found by previous mining algorithms.     

SIGNIFICANT FUZZY RULES EXTRACTION BY AN SVD–QR-BASED APPROACH

ABSTRACT

In this article, an SVD–QR-based approach is proposed to extract the important fuzzy rules from a rule base with several fuzzy rule tables to design an appropriate fuzzy system directly from some input-output data of the identified system. A fuzzy system with fuzzy rule tables is defined to approach the input-output pairs of an identified system. In the rule base of the defined fuzzy system, each fuzzy rule table corresponds to a partition of an input space. In order to extract the important fuzzy rules from the rule base of the defined fuzzy system, a firing strength matrix determined by the membership functions of the premise fuzzy sets is constructed. According to the firing strength matrix, the number of important fuzzy rules is determined by the Singular Value Decomposition SVD, and the important fuzzy rules are selected by the SVD–QR-based method. Consequently, a reconstructed fuzzy rule base composed of significant fuzzy rules is determined by the firing strength matrix. Furthermore, the recursive least-squares method is applied to determine the consequent part of the reconstructed fuzzy system according to the gathered input-output data so that a fine fuzzy system is determined by the proposed method. Finally, three nonlinear systems illustrate the efficiency of the proposed method.     

FUZZY PNN ALGORITHM AND ITSAPPLICATION TO NONLINEARPROCESSES

Abstract

In this paper, a fuzzy Polynomial Neural Network (PNN) algorithm is proposed to estimate the structure and parameters of fuzzy model, using the PNN based on Group Method of Data Handling (GMDH) algorithm. The new algorithm uses PNN algorithm and fuzzy reasoning in order to identify the premise structure and parameter of fuzzy implications rules, and the least square method in order to identify the optimal consequence parameters. Both time series data for the gas furnace and data for the NO_x emission process of gas turbine power plants are used for the purpose of evaluating the performance of the fuzzy PNN. The simulation results show that the proposed technique can produce the fuzzy model with higher accuracy and feasibility than other works achieved previously. This algorithm will be applied to limited data processes with several inputs.     

数据库中的模糊规则获取方法

分析了规则化的模糊逻辑原理。介绍了模糊化的规则获取和推理方法,由分析可以看出,基于模糊逻辑的规则获取和推理方法能在数据库中找出其规则,并且在实际系统中有着一定的意义。     

Mining fuzzy association rules from uncertain data

Association rule mining is an important data analysis method that can discover associations within data. There are numerous previous studies that focus on finding fuzzy association rules from precise and certain data. Unfortunately, real-world data tends to be uncertain due to human errors, instrument errors, recording errors, and so on. Therefore, a question arising immediately is how we can mine fuzzy association rules from uncertain data. To this end, this paper proposes a representation scheme to represent uncertain data. This representation is based on possibility distributions because the possibility theory establishes a close connection between the concepts of similarity and uncertainty, providing an excellent framework for handling uncertain data. Then, we develop an algorithm to mine fuzzy association rules from uncertain data represented by possibility distributions. Experimental results from the survey data show that the proposed approach can discover interesting and valuable patterns with high certainty.     

Neural networks that learn from fuzzy if-then rules

An architecture for neural networks that can handle fuzzy input vectors is proposed, and learning algorithms that utilize fuzzy if-then rules as well as numerical data in neural network learning for classification problems and for fuzzy control problems are derived. The learning algorithms can be viewed as an extension of the backpropagation algorithm to the case of fuzzy input vectors and fuzzy target outputs. Using the proposed methods, linguistic knowledge from human experts represented by fuzzy if-then rules and numerical data from measuring instruments can be integrated into a single information processing system (classification system or fuzzy control system). It is shown that the scheme works well for simple examples     

A multi-objective genetic algorithm approach to rule mining for affective product design

A novel multi-objective genetic algorithm (GA)-based rule-mining method for affective product design is proposed to discover a set of rules relating design attributes with customer evaluation based on survey data. The proposed method can generate approximate rules to consider the ambiguity of customer assessments. The generated rules can be used to determine the lower and upper limits of the affective effect of design patterns. For a rule-mining problem, the proposed multi-objective GA approach could simultaneously consider the accuracy, comprehensibility, and definability of approximate rules. In addition, the proposed approach can deal with categorical attributes and quantitative attributes, and determine the interval of quantitative attributes. Categorical and quantitative attributes in affective product design should be considered because they are commonly used to define the design profile of a product. In this paper, a two-stage rule-mining approach is proposed to generate rules with a simple chromosome design in the first stage of rule mining. In the second stage of rule mining, entire rule sets are refined to determine solutions considering rule interaction. A case study on mobile phones is used to demonstrate and validate the performance of the proposed rule-mining method. The method can discover rule sets with good support and coverage rates from the survey data.     

Elicitation of classification rules by fuzzy data mining

Data mining techniques can be used to find potentially useful patterns from data and to ease the knowledge acquisition bottleneck in building prototype rule-based systems. Based on the partition methods presented in simple-fuzzy-partition-based method (SFPBM) proposed by Hu et al. (Comput. Ind. Eng. 43(4) (2002) 735), the aim of this paper is to propose a new fuzzy data mining technique consisting of two phases to find fuzzy if–then rules for classification problems: one to find frequent fuzzy grids by using a pre-specified simple fuzzy partition method to divide each quantitative attribute, and the other to generate fuzzy classification rules from frequent fuzzy grids. To improve the classification performance of the proposed method, we specially incorporate adaptive rules proposed by Nozaki et al. (IEEE Trans. Fuzzy Syst. 4(3) (1996) 238) into our methods to adjust the confidence of each classification rule. For classification generalization ability, the simulation results from the iris data demonstrate that the proposed method may effectively derive fuzzy classification rules from training samples.     

Mining fuzzy association rules for classification problems

The effective development of data mining techniques for the discovery of knowledge from training samples for classification problems in industrial engineering is necessary in applications, such as group technology. This paper proposes a learning algorithm, which can be viewed as a knowledge acquisition tool, to effectively discover fuzzy association rules for classification problems. The consequence part of each rule is one class label. The proposed learning algorithm consists of two phases: one to generate large fuzzy grids from training samples by fuzzy partitioning in each attribute, and the other to generate fuzzy association rules for classification problems by large fuzzy grids. The proposed learning algorithm is implemented by scanning training samples stored in a database only once and applying a sequence of Boolean operations to generate fuzzy grids and fuzzy rules; therefore, it can be easily extended to discover other types of fuzzy association rules. The simulation results from the iris data demonstrate that the proposed learning algorithm can effectively derive fuzzy association rules for classification problems.     

Applying cluster-based fuzzy association rules mining framework into EC environment

During electronic commerce (EC) environment, how to effectively mine the useful transaction information will be an important issue to be addressed in designing the marketing strategy for most enterprises. Especially, the relationships between different databases (e.g., the transaction and online browsing database) may have the unknown and potential knowledge of business intelligence. Two important issues of mining association rules were mentioned to address EC application in this study. The first issue is the discovery of generalized fuzzy association rules in the transaction database. The second issue is to discover association rules from the web usage data and the large itemsets identified in the transaction database. A cluster-based fuzzy association rules (CBFAR) mining architecture is then proposed to simultaneously address such two issues in this study. Three contributions were achieved as: (a) an efficient fuzzy association rule miner based on cluster-based fuzzy-sets tables is presented to identify all the large fuzzy itemsets; (b) this approach requires less contrast to generate large itemsets; (3) a fuzzy rule mining approach is used to compute the confidence values for discovering the relationships between transaction database and browsing information database. Finally, a simulated example during EC environment is provided to demonstrate the rationality and feasibility of the proposed approach.