Mining fuzzy association rules for classification problems

The effective development of data mining techniques for the discovery of knowledge from training samples for classification problems in industrial engineering is necessary in applications, such as group technology. This paper proposes a learning algorithm, which can be viewed as a knowledge acquisition tool, to effectively discover fuzzy association rules for classification problems. The consequence part of each rule is one class label. The proposed learning algorithm consists of two phases: one to generate large fuzzy grids from training samples by fuzzy partitioning in each attribute, and the other to generate fuzzy association rules for classification problems by large fuzzy grids. The proposed learning algorithm is implemented by scanning training samples stored in a database only once and applying a sequence of Boolean operations to generate fuzzy grids and fuzzy rules; therefore, it can be easily extended to discover other types of fuzzy association rules. The simulation results from the iris data demonstrate that the proposed learning algorithm can effectively derive fuzzy association rules for classification problems.     

Selecting fuzzy if-then rules for classification problems usinggenetic algorithms

This paper proposes a genetic-algorithm-based method for selecting a small number of significant fuzzy if-then rules to construct a compact fuzzy classification system with high classification power. The rule selection problem is formulated as a combinatorial optimization problem with two objectives: to maximize the number of correctly classified patterns and to minimize the number of fuzzy if-then rules. Genetic algorithms are applied to this problem. A set of fuzzy if-then rules is coded into a string and treated as an individual in genetic algorithms. The fitness of each individual is specified by the two objectives in the combinatorial optimization problem. The performance of the proposed method for training data and test data is examined by computer simulations on the iris data of Fisher     

面向不均衡分类的隶属度加权模糊支持向量机

针对不均衡分类问题,提出了一种基于隶属度加权的模糊支持向量机模型。使用传统支持向量机对样本进行训练,并通过样本点与所得分类超平面之间的距离构造模糊隶属度,这不仅能够消除噪点和野值点的影响,而且可以在一定程度上约减样本;利用正负类的平均隶属度和样本数量求得平衡调节因子,消除数据不平衡时造成的分类超平面的偏移现象;通过实验结果验证了该算法的可行性和有效性。实验结果表明,该算法能有效提高分类精度,特别是对不平衡数据效果更加明显,在训练速度和分类性能上比传统支持向量机和模糊支持向量机有进一步的提升。     

Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions

Multiobjective genetic fuzzy rule selection is based on the generation of a set of candidate fuzzy classification rules using a preestablished granularity or multiple fuzzy partitions with different granularities for each attribute. Then, a multiobjective evolutionary algorithm is applied to perform fuzzy rule selection. Since using multiple granularities for the same attribute has been sometimes pointed out as to involve a potential interpretability loss, a mechanism to specify appropriate single granularities at the rule extraction stage has been proposed to avoid it but maintaining or even improving the classification performance. In this work, we perform a statistical study on this proposal and we extend it by combining the single granularity-based approach with a lateral tuning of the membership functions, i.e., complete contexts learning. In this way, we analyze in depth the importance of determining the appropriate contexts for learning fuzzy classifiers. To this end, we will compare the single granularity-based approach with the use of multiple granularities with and without tuning. The results show that the performance of the obtained classifiers can be even improved by obtaining the appropriate variable contexts, i.e., appropriate granularities and membership function parameters.     

Genetic algorithm-based strategy for identifying association rules without specifying actual minimum support

We design a genetic algorithm-based strategy for identifying association rules without specifying actual minimum support. In this approach, an elaborate encoding method is developed, and the relative confidence is used as the fitness function. With genetic algorithm, a global search can be performed and system automation is implemented, because our model does not require the user-specified threshold of minimum support. Furthermore, we expand this strategy to cover quantitative association rule discovery. For efficiency, we design a generalized FP-tree to implement this algorithm. We experimentally evaluate our approach, and demonstrate that our algorithms significantly reduce the computation costs and generate interesting association rules only.     

基于多最小支持度的多层模糊关联规则挖掘

为了在事务数据库中发现关联规则,在现实挖掘应用中,经常采用不同的标准去判断不同项目的重要性,管理项目之间的分类关系和处理定量数据集这3个方法去处理问题,因此提出一个在定量事务数据库中采用多最小支持度,在项目集中获取隐含知识的多层模糊关联规则挖掘算法。该挖掘算法使用两种支持度约束和至上而下逐步细化的方法推导出频繁项集,同时可以发现交叉层次的模糊关联规则。通过实例证明了该挖掘算法在多最小支持度约束下推导出的多层模糊关联规则是易于理解和有意义的,具有很好的效率和伸缩性。     

An improved approach to find membership functions and multiple minimum supports in fuzzy data mining

Fuzzy mining approaches have recently been discussed for deriving fuzzy knowledge. Since items may have their own characteristics, different minimum supports and membership functions may be specified for different items. In the past, we proposed a genetic-fuzzy data-mining algorithm for extracting minimum supports and membership functions for items from quantitative transactions. In that paper, minimum supports and membership functions of all items are encoded in a chromosome such that it may be not easy to converge. In this paper, an enhanced approach is proposed, which processes the items in a divide-and-conquer strategy. The approach is called divide-and-conquer genetic-fuzzy mining algorithm for items with Multiple Minimum Supports (DGFMMS), and is designed for finding minimum supports, membership functions, and fuzzy association rules. Possible solutions are evaluated by their requirement satisfaction divided by their suitability of derived membership functions. The proposed GA framework maintains multiple populations, each for one item’s minimum support and membership functions. The final best minimum supports and membership functions in all the populations are then gathered together to be used for mining fuzzy association rules. Experimental results also show the effectiveness of the proposed approach.     

Maximum entropy membership functions for discrete fuzzy variables

Due to the deficiency of information, the membership function of a fuzzy variable cannot be obtained explicitly. It is a challenging work to find an appropriate membership function when certain partial information about a fuzzy variable is given, such as expected value or moments. This paper solves such problems for discrete fuzzy variables via maximum entropy principle and proves some maximum entropy theorems with certain constraints. A genetic algorithm is designed to solve the general maximum entropy model for discrete fuzzy variables, which is illustrated by some numerical experiments.     

Boosting fuzzy rules in classification problems under single‐winner inference

In previous studies, we have shown that an Adaboost‐based fitness can be successfully combined with a Genetic Algorithm to iteratively learn fuzzy rules from examples in classification problems. Unfortunately, some restrictive constraints in the implementation of the logical connectives and the inference method were assumed. Alas, the knowledge bases Adaboost produces are only compatible with an inference based on the maximum sum of votes scheme, and they can only use the t‐norm product to model the “and” operator. This design is not optimal in terms of linguistic interpretability. Using the sum to aggregate votes allows many rules to be combined, when the class of an example is being decided. Because it can be difficult to isolate the contribution of individual rules to the knowledge base, fuzzy rules produced by Adaboost may be difficult to understand linguistically. In this point of view, single‐winner inference would be a better choice, but it implies dropping some nontrivial hypotheses. In this work we introduce our first results in the search for a boosting‐based genetic method able to learn weighted fuzzy rules that are compatible with this last inference method. © 2007 Wiley Periodicals, Inc. Int J Int Syst 22: 1021–1034, 2007.     

Generating fuzzy rules from training instances for fuzzy classification systems

In recent years, many methods have been proposed to generate fuzzy rules from training instances for handling the Iris data classification problem. In this paper, we present a new method to generate fuzzy rules from training instances for dealing with the Iris data classification problem based on the attribute threshold value α, the classification threshold value β and the level threshold value γ, where α  [0, 1], β  [0, 1] and γ  [0, 1]. The proposed method gets a higher average classification accuracy rate than the existing methods.     

Mining fuzzy association rules and fuzzy frequency episodes for intrusion detection

Lee, Stolfo, and Mok 1 previously reported the use of association rules and frequency episodes for mining audit data to gain knowledge for intrusion detection. The integration of association rules and frequency episodes with fuzzy logic can produce more abstract and flexible patterns for intrusion detection, since many quantitative features are involved in intrusion detection and security itself is fuzzy. We present a modification of a previously reported algorithm for mining fuzzy association rules, define the concept of fuzzy frequency episodes, and present an original algorithm for mining fuzzy frequency episodes. We add a normalization step to the procedure for mining fuzzy association rules in order to prevent one data instance from contributing more than others. We also modify the procedure for mining frequency episodes to learn fuzzy frequency episodes. Experimental results show the utility of fuzzy association rules and fuzzy frequency episodes for intrusion detection. © 2000 John Wiley & Sons, Inc.     

Representing membership functions as points in high-dimensionalspaces for fuzzy interpolation and extrapolation

This paper introduces a new approach for fuzzy interpolation and extrapolation of sparse rule base comprising of membership functions with finite number of characteristic points. The approach calls for representing membership functions as points in high-dimensional Cartesian spaces using the locations of their characteristic points as coordinates. Hence, a fuzzy rule base can be viewed as a set of mappings between the antecedent and consequent spaces and the interpolation and extrapolation problem becomes searching for an image in the consequent space upon given an antecedent observation. The present approach divides observations into two groups: 1) observations within the antecedent spanning set contain the same geometric properties as the given antecedents; and 2) observations lying outside the antecedent spanning set contain new geometric properties beyond those of the given rules. Heuristic reasoning must therefore be applied. In this case, a two-step approach with certain flexibility to accommodate additional criteria and design objectives is formulated     

Supervised Pseudo Self-Evolving Cerebellar algorithm for generating fuzzy membership functions

The proper generation of fuzzy membership function is of fundamental importance in fuzzy applications. The effectiveness of the membership functions in pattern classifications can be objectively measured in terms of interpretability and classification accuracy in the conformity of the decision boundaries to the inherent probabilistic decision boundaries of the training data. This paper presents the Supervised Pseudo Self-Evolving Cerebellar (SPSEC) algorithm that is bio-inspired from the two-stage development process of the human nervous system whereby the basic architecture are first laid out without any activity-dependent processes and then refined in activity-dependent ways. SPSEC first constructs a cerebellar-like structure in which neurons with high trophic factors evolves to form membership functions that relate intimately to the probability distributions of the data and concomitantly reconcile with defined semantic properties of linguistic variables. The experimental result of using SPSEC to generate fuzzy membership functions is reported and compared with a selection of algorithms using a publicly available UCI Sonar dataset to illustrate its effectiveness.     

A relationship between membership functions and approximationaccuracy in fuzzy systems

This paper presents a relationship between membership functions and approximation accuracy in fuzzy systems. This relationship suggests an idea to design membership functions such that the approximation accuracy of fuzzy systems is improved.     

一种有效的模糊关联规则挖掘的形式化方法

模糊关联规则用于处理数据库中的不精确信息,并提供一个知识发现的良好表示。利用约束级别表示理论将GUHA模型泛化用于模糊关联规则,通过约束级别管理模糊规则,并给出一个扩展的验证度量过程。使用形式化方法的挖掘算法,在不同的约束级别上并行化挖掘过程,总结得到的结果。算法的复杂度分析以及实验结果表明该形式化方法是有效可行的,从而确立了模糊关联规则表示和评价的逻辑基础。     

基于模糊聚类的模糊关联规则在流程企业中的应用

采用数据挖掘中的模糊聚类分析了流程企业中历史数据的区间值,然后用模糊关联规则挖掘出有用的规则。首先阐述了模糊聚类的RFCM算法和关联规则的Apriori算法的内容,分析了实现模糊关联规则的Fuzzy_ClustApriori算法流程,并用RFCM算法对实际数据进行分析,得到不同类别的模糊数。根据Fuzzy_ClustApriori算法的步骤对模糊化的参数点进行处理,得到了有价值的模糊规则,为流程企业的生产优化提供了理论依据。     

Wavelet domain association rules for efficient texture classification

The wavelet domain association rules method is proposed for efficient texture characterization. The concept of association rules to capture the frequently occurring local intensity variation in textures. The frequency of occurrence of these local patterns within a region is used as texture features. Since texture is basically a multi-scale phenomenon, multi-resolution approaches such as wavelets, are expected to perform efficiently for texture analysis. Thus, this study proposes a new algorithm which uses the wavelet domain association rules for texture classification. Essentially, this work is an extension version of an early work of the Rushing et al. [10], [11], where the generation of intensity domain association rules generation was proposed for efficient texture characterization. The wavelet domain and the intensity domain (gray scale) association rules were generated for performance comparison purposes. As a result, Rushing et al. [10], [11] demonstrated that intensity domain association rules performs much more accurate results than those of the methods which were compared in the Rushing et al. work. Moreover, the performed experimental studies showed the effectiveness of the wavelet domain association rules than the intensity domain association rules for texture classification problem. The overall success rate is about 97%.     

A new method for constructing membership functions and fuzzy rulesfrom training examples

To extract knowledge from a set of numerical data and build up a rule-based system is an important research topic in knowledge acquisition and expert systems. In recent years, many fuzzy systems that automatically generate fuzzy rules from numerical data have been proposed. In this paper, we propose a new fuzzy learning algorithm based on the alpha-cuts of equivalence relations and the alpha-cuts of fuzzy sets to construct the membership functions of the input variables and the output variables of fuzzy rules and to induce the fuzzy rules from the numerical training data set. Based on the proposed fuzzy learning algorithm, we also implemented a program on a Pentium PC using the MATLAB development tool to deal with the Iris data classification problem. The experimental results show that the proposed fuzzy learning algorithm has a higher average classification ratio and can generate fewer rules than the existing algorithm.     

Mining fuzzy association rules in a bank-account database

This paper describes how we applied a fuzzy technique to a data-mining task involving a large database that was provided by an international bank with offices in Hong Kong. The database contains the demographic data of over 320,000 customers and their banking transactions, which were collected over a six-month period. By mining the database, the bank would like to be able to discover interesting patterns in the data. The bank expected that the hidden patterns would reveal different characteristics about different customers so that they could better serve and retain them. To help the bank achieve its goal, we developed a fuzzy technique, called fuzzy association rule mining II (FARM II). FARM II is able to handle both relational and transactional data. It can also handle fuzzy data. The former type of data allows FARM II to discover multidimensional association rules, whereas the latter data allows some of the patterns to be more easily revealed and expressed. To effectively uncover the hidden associations in the bank-account database, FARM II performs several steps which are described in detail in this paper. With FARM II, the bank discovered that they had identified some interesting characteristics about the customers who had once used the bank's loan services but then decided later to cease using them. The bank translated what they discovered into actionable items by offering some incentives to retain their existing customers.