Learning fuzzy rules for similarity assessment in case-based reasoning

Fundamental to case-based reasoning is the assumption that similar problems have similar solutions. The meaning of the concept of “similarity” can vary in different situations and remains an issue. This paper proposes a novel similarity model consisting of fuzzy rules to represent the semantics and evaluation criteria for similarity. We believe that fuzzy if-then rules present a more powerful and flexible means to capture domain knowledge for utility oriented similarity modeling than traditional similarity measures based on feature weighting. Fuzzy rule-based reasoning is utilized as a case matching mechanism to determine whether and to which extent a known case in the case library is similar to a given problem in query. Further, we explain that such fuzzy rules for similarity assessment can be learned from the case library using genetic algorithms. The key to this is pair-wise comparisons of cases with known solutions in the case library such that sufficient training samples can be derived for genetic-based fuzzy rule learning. The evaluations conducted have shown the superiority of the proposed method in similarity modeling over traditional schemes as well as the feasibility of learning fuzzy similarity rules from a rather small case base while still yielding competent system performance.     

Transferring Case Knowledge To Adaptation Knowledge: An Approach for Case-Base Maintenance

In this article we propose a case-base maintenance methodology based on the idea of transferring knowledge between knowledge containers in a case-based reasoning (CBR) system. A machine-learning technique, fuzzy decision-tree induction, is used to transform the case knowledge to adaptation knowledge. By learning the more sophisticated fuzzy adaptation knowledge, many of the redundant cases can be removed. This approach is particularly useful when the case base consists of a large number of redundant cases and the retrieval efficiency becomes a real concern of the user. The method of maintaining a case base from scratch, as proposed in this article, consists of four steps. First, an approach to learning feature weights automatically is used to evaluate the importance of different features in a given case base. Second, clustering of cases is carried out to identify different concepts in the case base using the acquired feature-weights knowledge. Third, adaptation rules are mined for each concept using fuzzy decision trees. Fourth, a selection strategy based on the concepts of case coverage and reachability is used to select representative cases. In order to demonstrate the effectiveness of this approach as well as to examine the relationship between compactness and performance of a CBR system, experimental testing is carried out using the Traveling and the Rice Taste data sets. The results show that the testing case bases can be reduced by 36 and 39 percent, respectively, if we complement the remaining cases by the adaptation rules discovered using our approach. The overall accuracies of the two smaller case bases are 94 and 90 percent, respectively, of the originals.     

KNOWLEDGEABLE LEARNING USING MOBAL: A MEDICAL CASE STUDY

Building up a knowledge base is a complex task in which theoretical knowledge needs to be integrated with practical experience. This integration can be supported by a system that can manage linking between rules, representing experts or textbook or theoretical knowledge and facts (or data), representing cases from practice. Conflicts between rules and real-world cases can have diverse causes. Case data can be noisy or inconsistent or both. We use rules to filter cases and confine noise or inconsistency. However, rules can be overly general and can classify more cases than intended. Then machine learning can be used to find additional restrictions for the rules. If, however, no significant similarities can be determined between the misclassified cases, we seek additional expert input to support effective implementation of learning tools. Using a case subset, we capture expert input and formulate a set of “enriched” cases. We then use the enriched cases to learn additional rules and to introduce practical features to validate and refine the original rule base. In this paper, we discuss the role of machine learning as a vehicle for supporting     

A two-stage evolutionary process for designing TSK fuzzy rule-basedsystems

Nowadays, fuzzy rule-based systems are successfully applied to many different real-world problems. Unfortunately, relatively few well-structured methodologies exist for designing and, in many cases, human experts are not able to express the knowledge needed to solve the problem in the form of fuzzy rules. Takagi-Sugeno-Kang (TSK) fuzzy rule-based systems were enunciated in order to solve this design problem because they are usually identified using numerical data. In this paper we present a two-stage evolutionary process for designing TSK fuzzy rule-based systems from examples combining a generation stage based on a (mu, lambda)-evolution strategy, in which the fuzzy rules with different consequents compete among themselves to form part of a preliminary knowledge base, and a refinement stage in which both the antecedent and consequent parts of the fuzzy rules in this previous knowledge base are adapted by a hybrid evolutionary process composed of a genetic algorithm and an evolution strategy to obtain the final Knowledge base whose rules cooperate in the best possible way. Some aspects make this process different from others proposed until now: the design problem is addressed in two different stages, the use of an angular coding of the consequent parameters that allows us to search across the whole space of possible solutions, and the use of the available knowledge about the system under identification to generate the initial populations of the Evolutionary Algorithms that causes the search process to obtain good solutions more quickly. The performance of the method proposed is shown by solving two different problems: the fuzzy modeling of some three-dimensional surfaces and the computing of the maintenance costs of electrical medium line in Spanish towns. Results obtained are compared with other kind of techniques, evolutionary learning processes to design TSK and Mamdani-type fuzzy rule-based systems in the first case, and classical regression and neural modeling in the second.     

Knowledge acquisition in incomplete fuzzy information systems via the rough set approach

Abstract: Machine learning can extract desired knowledge from training examples and ease the development bottleneck in building expert systems. Most learning approaches derive rules from complete and incomplete data sets. If attribute values are known as possibility distributions on the domain of the attributes, the system is called an incomplete fuzzy information system. Learning from incomplete fuzzy data sets is usually more difficult than learning from complete data sets and incomplete data sets. In this paper, we deal with the problem of producing a set of certain and possible rules from incomplete fuzzy data sets based on rough sets. The notions of lower and upper generalized fuzzy rough approximations are introduced. By using the fuzzy rough upper approximation operator, we transform each fuzzy subset of the domain of every attribute in an incomplete fuzzy information system into a fuzzy subset of the universe, from which fuzzy similarity neighbourhoods of objects in the system are derived. The fuzzy lower and upper approximations for any subset of the universe are then calculated and the knowledge hidden in the information system is unravelled and expressed in the form of decision rules.     

An approach to fuzzy default reasoning for function approximation

This paper discusses fuzzy reasoning for approximately realizing nonlinear functions by a small number of fuzzy if-then rules with different specificity levels. Our fuzzy rule base is a mixture of general and specific rules, which overlap with each other in the input space. General rules work as default rules in our fuzzy rule base. First, we briefly describe existing approaches to the handling of default rules in the framework of possibility theory. Next, we show that standard interpolation-based fuzzy reasoning leads to counterintuitive results when general rules include specific rules with different consequents. Then, we demonstrate that intuitively acceptable results are obtained from a non-standard inclusion-based fuzzy reasoning method. Our approach is based on the preference for more specific rules, which is a commonly used idea in the field of default reasoning. When a general rule includes a specific rule and they are both compatible with an input vector, the weight of the general rule is discounted in fuzzy reasoning. We also discuss the case where general rules do not perfectly but partially include specific rules. Then we propose a genetics-based machine learning (GBML) algorithm for extracting a small number of fuzzy if-then rules with different specificity levels from numerical data using our inclusion-based fuzzy reasoning method. Finally, we describe how our approach can be applied to the approximate realization of fuzzy number-valued nonlinear functions     

Neural networks that learn from fuzzy if-then rules

An architecture for neural networks that can handle fuzzy input vectors is proposed, and learning algorithms that utilize fuzzy if-then rules as well as numerical data in neural network learning for classification problems and for fuzzy control problems are derived. The learning algorithms can be viewed as an extension of the backpropagation algorithm to the case of fuzzy input vectors and fuzzy target outputs. Using the proposed methods, linguistic knowledge from human experts represented by fuzzy if-then rules and numerical data from measuring instruments can be integrated into a single information processing system (classification system or fuzzy control system). It is shown that the scheme works well for simple examples     

Mining fuzzy association rules for classification problems

The effective development of data mining techniques for the discovery of knowledge from training samples for classification problems in industrial engineering is necessary in applications, such as group technology. This paper proposes a learning algorithm, which can be viewed as a knowledge acquisition tool, to effectively discover fuzzy association rules for classification problems. The consequence part of each rule is one class label. The proposed learning algorithm consists of two phases: one to generate large fuzzy grids from training samples by fuzzy partitioning in each attribute, and the other to generate fuzzy association rules for classification problems by large fuzzy grids. The proposed learning algorithm is implemented by scanning training samples stored in a database only once and applying a sequence of Boolean operations to generate fuzzy grids and fuzzy rules; therefore, it can be easily extended to discover other types of fuzzy association rules. The simulation results from the iris data demonstrate that the proposed learning algorithm can effectively derive fuzzy association rules for classification problems.     

Fuzzy hypercubes: Linguistic learning/reasoning systems for intelligent control and identification

This paper introduces a new tool for intelligent control and identification. A robust and reliable learning and reasoning mechanism is addressed based upon fuzzy set theory and fuzzy associative memories. The mechanism storesa priori an initial knowledge base via approximate learning and utilizes this information for identification and control via fuzzy inferencing. This architecture parallels a well-known scheme in which memory implicative rules are stored disjunctively. We call this process afuzzy hypercube. Fuzzy hypercubes can be applied to a class of complex and highly nonlinear systems which suffer from vagueness uncertainty and incomplete information such as fuzziness and ignorance. Evidential aspects of a fuzzy hypercube are treated to assess the degree of certainty or reliability. The implementation issue using fuzzy hypercubes is raised, and finally, a fuzzy hypercube is applied to fuzzy linguistic control.     

Knowledge Incorporation into Neural Networks From Fuzzy Rules

The incorporation of prior knowledge into neural networks can improve neural network learning in several respects, for example, a faster learning speed and better generalization ability. However, neural network learning is data driven and there is no general way to exploit knowledge which is not in the form of data input-output pairs. In this paper, we propose two approaches for incorporating knowledge into neural networks from fuzzy rules. These fuzzy rules are generated based on expert knowledge or intuition. In the first approach, information from the derivative of the fuzzy system is used to regularize the neural network learning, whereas in the second approach the fuzzy rules are used as a catalyst. Simulation studies show that both approaches increase the learning speed significantly.     

Case-base maintenance by conceptual clustering of graphs

Case-base maintenance typically involves the addition, removal or revision of cases, but can also include changes to the retrieval knowledge. In this paper, we consider the learning of the retrieval knowledge (organization) as well as the prototypes and the cases as case-based maintenance. We address this problem based on cases that have a structural case representation. Such representations are common in computer vision and image interpretation, building design, timetabling or gene-nets. In this paper we propose a similarity measure for an attributed structural representation and an algorithm that incrementally learns the organizational structure of a case base. This organization schema is based on a hierarchy and can be updated incrementally as soon as new cases are available. The tentative underlying conceptual structure of the case base is visually presented to the user. We describe two approaches for organizing the case base. Both are based on approximate graph subsumption. The first approach is based on a divide-and-conquer strategy whereas the second one is based on a split-and-merge strategy which better allows to fit the hierarchy to the actual structure of the application but takes more complex operations.     

Adaptive knowledge base with attribute weight and threshold adjustments for Japanese-to-Braille translations

This paper proposes a method for generating an adaptive knowledge base (AKB) involving two knowledge representations: rule and case. Combining rules and cases makes it possible to solve problems accurately and quickly, and to acquire new cases from problem-solving results. In general case-based problem-solving methods, the similarity metric must be defined for each problem domain. In previous work using rules and cases, a threshold of negative case applications had to be adjusted. The proposed AKB does not require manual adjustment of the threshold and the similarity metric. This paper also proposes a Japanese-to-Braille translation system which uses the proposed AKB. Experimental results have showed that the case acquisition and similarity weight adjustment can reduce errors, and that the threshold adjustment significantly reduces segmentation errors. This work was presented in part at the 12th International Symposium on Artificial Life and Robotics, Oita, Japan, January 25–27, 2007     

A neural fuzzy system with fuzzy supervised learning

A neural fuzzy system learning with fuzzy training data (fuzzy if-then rules) is proposed in this paper. This system is able to process and learn numerical information as well as linguistic information. At first, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use alpha-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, a fuzzy supervised learning algorithm is developed for the proposed system. It extends the normal supervised learning techniques to the learning problems where only linguistic teaching signals are available. The fuzzy supervised learning scheme can train the proposed system with desired fuzzy input-output pairs which are fuzzy numbers instead of the normal numerical values. With fuzzy supervised learning, the proposed system can be used for rule base concentration to reduce the number of rules in a fuzzy rule base. Simulation results are presented to illustrate the performance and applicability of the proposed system.     

Learning Premises of Fuzzy Rules for Knowledge Acquisition in Classification Problems

A key issue in building fuzzy classification systems is the specification of rule conditions, which determine the structure of a knowledge base. This paper presents a new approach to automatically extract classification knowledge from numerical data by means of premise learning. A genetic algorithm is employed to search for premise structure in combination with parameters of membership functions of input fuzzy sets to yield optimal conditions of classification rules. The major advantage of our work is that a parsimonious knowledge base with a low number of rules can be achieved. The practical applicability of the proposed method is examined by computer simulations on two well-known benchmark problems of Iris Data and Cancer Data classification. Received 11 February 1999 / Revised 13 January 2001 / Accepted in revised form 13 February 2001     

A fuzzy connectionist expert system for visual pattern classification

In this paper, a connectionist model to integrate knowledge-based techniques into neural network approaches for visual pattern classification is presented. We propose a new structure of connectionist model which has rule-following capability as well as instance-based learning capability. Each node of the proposed network is doubly linked by two types of connections: positive connection and negative connection. Such connectionism provides a methodology to construct the classifier from the rule base and allows the expert knowledge to be utilized for the effective learning. For visual pattern classification, we present the techniques for knowledge representation and utilization using the concepts of fuzzy rules and fuzzy relations. We also discuss in this paper some advantageous characteristics of the model: result explanation capability and rule refinement capability. From the experimental results of the handwritten digit classification, the feasibility of the proposed model is evaluated.     

Towards Automated Case Knowledge Discovery in the M 2 Case-Based Reasoning System

In this paper we present the M ² Case-Based Reasoning (CBR) system. The M ² system addresses a number of issues that present methodologies for CBR systems have shied away from. We discuss techniques for removing the knowledge acquisition bottleneck when acquiring case knowledge. Here, case knowledge refers to the complementary knowledge structures, cases (more specific in nature) and adaptation rules (more general). We address the use of negative cases for updating the case knowledge as well as for refining the similarity measures. In particular we discuss in detail, showing experimental results, the use of Data Mining within the M ² system to build the case base from a database containing operational data, and discover adaptation rules. A methodology to monitor the competence of the CBR system and to utilize negative cases for updating the CBR system to enhance its competence is also discussed. The M ² CBR system also employs Rough Set and Fuzzy Set theories to further enhance its capabilities within real-world applications as well as providing a richer and truer model of human reasoning.     

基于数值型数据的模糊规则快速挖掘方法

针对数值型数据能准确反应现实世界,但难以理解的问题,为了从数值型数据中挖掘出易于理解的知识,提出了基于数值型数据的模糊规则快速挖掘方法。该方法能从数值型数据中挖掘出一个零阶的Sugeno模糊规则,并采用一种启发式方法将这个零阶的Sugeno模糊规则的数值结论转变为两个带置信度的语言变量,并给出了规则库的存储结构。最后通过实例证明了这种快速模糊规则挖掘方法能避免复杂的数值型计算和能有效逼近非线性函数的优点．     

Integrating fuzzy knowledge by genetic algorithms

We propose a genetic algorithm-based fuzzy knowledge integration framework that can simultaneously integrate multiple fuzzy rule sets and their membership function sets. The proposed approach consists of two phases: fuzzy knowledge encoding and fuzzy knowledge integration. In the encoding phase, each fuzzy rule set with its associated membership functions is first transformed into an intermediary representation and then further encoded as a string. The combined strings form an initial knowledge population, which is then ready for integration. In the knowledge-integration phase, a genetic algorithm is used to generate an optimal or nearly optimal set of fuzzy rules and membership functions from the initial knowledge population. Two application domains, the hepatitis diagnosis and the sugarcane breeding prediction, were used to show the performance of the proposed knowledge-integration approach. Results show that the fuzzy knowledge base derived using our approach performs better than every individual knowledge base     

Checking the coherence and redundancy of fuzzy knowledge bases

Checking the coherence of a set of rules is an important step in knowledge base validation. Coherence is also needed in the field of fuzzy systems. Indeed, rules are often used regardless of their semantics, and it sometimes leads to sets of rules that make no sense. Avoiding redundancy is also of interest in real-time systems for which the inference engine is time consuming. A knowledge base is potentially inconsistent or incoherent if there exists a piece of input data that respects integrity constraints and that leads to logical inconsistency when added to the knowledge base. We more particularly consider knowledge bases composed of parallel fuzzy rules. Then, coherence means that the projection on the input variables of the conjunctive combination of the possibility distributions representing the fuzzy rules leaves these variables completely unrestricted (i.e., any value for these variables is possible) or, at least, not more restrictive than integrity constraints. Fuzzy rule representations can be implication-based or conjunction-based; we show that only implication-based models may lead to coherence problems. However, unlike conjunction-based models, they allow to design coherence checking processes. Some conditions that a set of parallel rules has to satisfy in order to avoid inconsistency problems are given for certainty or gradual rules. The problem of redundancy, which is also of interest for fuzzy knowledge bases validation, is addressed for these two kinds of rules