Piecewise-linear neural networks and their relationship to rule extraction from data

This article addresses the topic of extracting logical rules from data by means of artificial neural networks. The approach based on piecewise linear neural networks is revisited, which has already been used for the extraction of Boolean rules in the past, and it is shown that this approach can be important also for the extraction of fuzzy rules. Two important theoretical properties of piecewise-linear neural networks are proved, allowing an elaboration of the basic ideas of the approach into several variants of an algorithm for the extraction of Boolean rules. That algorithm has already been used in two real-world applications. Finally, a connection to the extraction of rules of the ?ukasiewicz logic is established, relying on recent results about rational McNaughton functions. Based on one of the constructive proofs of the McNaughton theorem, an algorithm is formulated that in principle allows extracting a particular kind of formulas of the ?ukasiewicz predicate logic from piecewise-linear neural networks trained with rational data.     

A neural fuzzy system with fuzzy supervised learning

A neural fuzzy system learning with fuzzy training data (fuzzy if-then rules) is proposed in this paper. This system is able to process and learn numerical information as well as linguistic information. At first, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use alpha-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, a fuzzy supervised learning algorithm is developed for the proposed system. It extends the normal supervised learning techniques to the learning problems where only linguistic teaching signals are available. The fuzzy supervised learning scheme can train the proposed system with desired fuzzy input-output pairs which are fuzzy numbers instead of the normal numerical values. With fuzzy supervised learning, the proposed system can be used for rule base concentration to reduce the number of rules in a fuzzy rule base. Simulation results are presented to illustrate the performance and applicability of the proposed system.     

A probabilistic fuzzy approach to modeling nonlinear systems

Recently, the study of incorporating probability theory and fuzzy logic has received much interest. To endow the traditional fuzzy rule-based systems (FRBs) with probabilistic features to handle randomness, this paper presents a probabilistic fuzzy neural network (ProFNN) by introducing the probability of input linguistic terms and providing linguistic meaning into the connectionist architecture. ProFNN integrates the probabilistic information of fuzzy rules into the antecedent parts and quantifies the impacts of the rules on the consequent parts using mutual subsethood, which work in conjunction with volume defuzzification in a gradient descent learning frame work. Despite the increase in the number of parameters, ProFNN provides a promising solution to deal with randomness and fuzziness in a single frame. To evaluate the performance and applicability of the proposed approach, ProFNN is carried out on various benchmarking problems and compared with other existing models with a performance better than most of them.     

A combination of hidden Markov model and fuzzy model for stock market forecasting

This paper presents a novel combination of the hidden Markov model (HMM) and the fuzzy models for forecasting stock market data. In a previous study we used an HMM to identify similar data patterns from the historical data and then used a weighted average to generate a ‘one-day-ahead’ forecast. This paper uses a similar approach to identify data patterns by using the HMM and then uses fuzzy logic to obtain a forecast value. The HMM's log-likelihood for each of the input data vectors is used to partition the dataspace. Each of the divided dataspaces is then used to generate a fuzzy rule. The fuzzy model developed from this approach is tested on stock market data drawn from different sectors. Experimental results clearly show an improved forecasting accuracy compared to other forecasting models such as, ARIMA, artificial neural network (ANN) and another HMM-based forecasting model.     

On some idea of a neuro-fuzzy controller

The paper presents a neuro-fuzzy technique for the design of controllers. This technique can effectively deal with two main types of knowledge which usually describe the control strategy for complex systems, that is, a qualitative, linguistic, fuzzy knowledge usually expressed in the form of linguistic rules, and a quantitative, nonfuzzy information in the form of measurements and other numerical data. The proposed technique combines artificial neural networks with fuzzy logic yielding a structure that can be called a neuro-fuzzy controller or, broadly speaking, a fuzzy neural network. The paper presents a general structure of a neuro-fuzzy controller and two essential phases of its design, that is, a learning phase and a functioning phase. In turn, a numerical example which illustrates how the proposed controller works is presented. Finally, the paper describes an application of a neuro-fuzzy control to inverter drive systems for electric vehicles. The results of simulation and experimental investigations carried out on the laboratory model of an inverter drive system are also provided.     

Fuzzy fault tree analysis based on T–S model with application to INS/GPS navigation system

A novel fault tree analysis (FTA) technique based on the Takagi and Sugeno (T–S) model is proposed in this paper. In the proposed technique, referred to as the TS-FTA, the events in the conventional FTA can be expressed in terms of fuzzy possibilities, and the gates that represent the relations among the top event and the primary events are replaced by T–S fuzzy gates derived from the T–S model. The magnitudes of the faults in the system are expressed in term of fuzzy variables. Since the proposed TS-FTA is derived from fuzzy logic and the T–S model, it can readily handle fuzzy information and uncertainties in the relationships among events. Therefore, the TS-FTA is suitable for systems where exact information on the fault probabilities of the components and the failure mechanisms are not available. The proposed TS-FTA is applied to analyze the reliability of an Inertial Navigation System and Global Position System (INS/GPS) integrated navigation system.     

Compensatory neurofuzzy systems with fast learning algorithms

In this paper, a new adaptive fuzzy reasoning method using compensatory fuzzy operators is proposed to make a fuzzy logic system more adaptive and more effective. Such a compensatory fuzzy logic system is proved to be a universal approximator. The compensatory neural fuzzy networks built by both control-oriented fuzzy neurons and decision-oriented fuzzy neurons cannot only adaptively adjust fuzzy membership functions but also dynamically optimize the adaptive fuzzy reasoning by using a compensatory learning algorithm. The simulation results of a cart-pole balancing system and nonlinear system modeling have shown that: 1) the compensatory neurofuzzy system can effectively learn commonly used fuzzy IF-THEN rules from either well-defined initial data or ill-defined data; 2) the convergence speed of the compensatory learning algorithm is faster than that of the conventional backpropagation algorithm; and 3) the efficiency of the compensatory learning algorithm can be improved by choosing an appropriate compensatory degree.     

A self-learning fuzzy logic controller using genetic algorithmswith reinforcements

This paper presents a new method for learning a fuzzy logic controller automatically. A reinforcement learning technique is applied to a multilayer neural network model of a fuzzy logic controller. The proposed self-learning fuzzy logic control that uses the genetic algorithm through reinforcement learning architecture, called a genetic reinforcement fuzzy logic controller, can also learn fuzzy logic control rules even when only weak information such as a binary target of “success” or “failure” signal is available. In this paper, the adaptive heuristic critic algorithm of Barto et al. (1987) is extended to include a priori control knowledge of human operators. It is shown that the system can solve more concretely a fairly difficult control learning problem. Also demonstrated is the feasibility of the method when applied to a cart-pole balancing problem via digital simulations     

Tuning of a neuro-fuzzy controller by genetic algorithm

Due to their powerful optimization property, genetic algorithms (GAs) are currently being investigated for the development of adaptive or self-tuning fuzzy logic control systems. This paper presents a neuro-fuzzy logic controller (NFLC) where all of its parameters can be tuned simultaneously by GA. The structure of the controller is based on the radial basis function neural network (RBF) with Gaussian membership functions. The NFLC tuned by GA can somewhat eliminate laborious design steps such as manual tuning of the membership functions and selection of the fuzzy rules. The GA implementation incorporates dynamic crossover and mutation probabilistic rates for faster convergence. A flexible position coding strategy of the NFLC parameters is also implemented to obtain near optimal solutions. The performance of the proposed controller is compared with a conventional fuzzy controller and a PID controller tuned by GA. Simulation results show that the proposed controller offers encouraging advantages and has better performance.     

A neural fuzzy system with linguistic teaching signals

A neural fuzzy system learning with linguistic teaching signals is proposed. This system is able to process and learn numerical information as well as linguistic information. It can be used either as an adaptive fuzzy expert system or as an adaptive fuzzy controller. First, we propose a five-layered neural network for the connectionist realization of a fuzzy inference system. The connectionist structure can house fuzzy logic rules and membership functions for fuzzy inference. We use α-level sets of fuzzy numbers to represent linguistic information. The inputs, outputs, and weights of the proposed network can be fuzzy numbers of any shape. Furthermore, they can be hybrid of fuzzy numbers and numerical numbers through the use of fuzzy singletons. Based on interval arithmetics, two kinds of learning schemes are developed for the proposed system: fuzzy supervised learning and fuzzy reinforcement learning. Simulation results are presented to illustrate the performance and applicability of the proposed system     

Uncertainty analysis of rule-based expert systems with Dempster-Shafer mass assignments

This article extends Dempster-Shafer Theory (DST) mass probability assignments to Boolean algebra and considers how such probabilities can propagate through a system of Boolean equations, which form the basis for both rule-based expert systems and fault trees. the advantage of DST mass assignments over classical probability methods is the ability to accommodate when necessary uncommitted probability belief. This paper also examines rules in the context of a probabilistic logic, where a given rule itself may be true with some probability in the interval [0,1]. When expert system knowledge bases contain rules which may not always hold, or rules that occasionally must be operated upon with imprecise information, the DST mass assignment formalism is shown to be a suitable methodology for calculating probability assignments throughout the system.     

Application of type-2 fuzzy logic system for load frequency control using feedback error learning approaches

In this paper, the type-2 fuzzy logic system (T2FLS) controller using the feedback error learning (FEL) strategy has been proposed for load frequency control (LFC) in the restructure power system. The original FEL strategy consists of an intelligent feedforward controller (INFC) (i.e. artificial neural network (ANN)) and the conventional feedback controller (CFC). The CFC acting as a general feedback controller to guarantee the stability of the system plays a crucial role in the transient state. The INFC is adopted in forward path to take over the control problem in the steady state. In this work, to improve the performance of the FEL strategy, the T2FLS is adopted instead of ANN in the INFC part due to its ability to model uncertainties, which may exist in the rules and measured data of sensors more effectively. The proposed FEL controller has been compared with a type-1 fuzzy logic system (T1FLS) – based FEL controller and the proportional, integral and derivative (PID) controller to highlight the effectiveness of the proposed method.     

模糊组合神经网络智能选股模型的建立

针对传统的股票市场预测模型,为了准确地预测股票价格趋势、为广大投资者规避风险,应用模糊逻辑和组合神经网络,利用贝叶斯统计学与组合理论使二者有机结合,提出一种股票市场建模及预测方法。组合神经网络结合BP网络和径向基函数网络(RBF),神经元模糊系统有更强的学习和推理机制,能避免黑箱问题。实证研究结果表明,该方法有较高的预测精度和更好的稳定性。     

A Fuzzy Inference Network Model for Search Strategy Using Neural Logic Network

In this paper, a fuzzy inference network model for search strategy using neural logic network is presented. The model describes search strategy, and neural logic network is used to search. Fuzzy logic can bring about appropriate inference results by ignoring some information in the reasoning process. Neural logic networks are powerful tools for the reasoning process but not appropriate for the logical reasoning. To model human knowledge, besides the reasoning process capability, the logical reasoning capability is equally important. Another new neural network called neural logic network is able to do the logical reasoning. Because the fuzzy inference is a fuzzy logical reasoning, we construct a fuzzy inference network model based on the neural logic network, extending the existing rule inference network. And the traditional propagation rule is modified.     

A NEURO‐FUZZY SYSTEM DESIGN METHODOLOGY FOR VIBRATION CONTROL

Fuzzy system has been known to provide a framework for handling uncertainties and imprecision by taking linguistic information from human experts. However, difficulties arise in determining effectively the fuzzy system configuration, i.e., the number of rules, input and output membership functions. A neuro‐fuzzy system design methodology by combining neural network and fuzzy logic is developed in this paper to adaptively adjust the fuzzy membership functions and dynamically optimize the linguistic‐fuzzy rules. The structure of a five‐layer feedforward network is shown to determine systematically the correct fuzzy logic rules, tune optimally (in the sense of local region) the parameters of the membership functions, and perform accurately the fuzzy inference. It is shown both numerically and experimentally that engineering applications of the neuro‐fuzzy system to vibration control have been very successful.     

Mining fuzzy association rules for classification problems

The effective development of data mining techniques for the discovery of knowledge from training samples for classification problems in industrial engineering is necessary in applications, such as group technology. This paper proposes a learning algorithm, which can be viewed as a knowledge acquisition tool, to effectively discover fuzzy association rules for classification problems. The consequence part of each rule is one class label. The proposed learning algorithm consists of two phases: one to generate large fuzzy grids from training samples by fuzzy partitioning in each attribute, and the other to generate fuzzy association rules for classification problems by large fuzzy grids. The proposed learning algorithm is implemented by scanning training samples stored in a database only once and applying a sequence of Boolean operations to generate fuzzy grids and fuzzy rules; therefore, it can be easily extended to discover other types of fuzzy association rules. The simulation results from the iris data demonstrate that the proposed learning algorithm can effectively derive fuzzy association rules for classification problems.     

Representing rules as random sets,II: Iterated rules

I. R. Goodman, H. T. Nguyen, and others have proposed the theory of random sets as a unifying paradigm for knowledge-based systems. The more types of ambiguous evidence that can be brought under the random-set umbrella, the more potentially useful the theory becomes as a systematic methodology for comparing and fusing seemingly incongruent kinds of information. Conditional event algebra provides a general framework for dealing with rule-based evidence in a manner consistent with probability theory. This and a previous companion article bring conditional event logic—and through it, rules and iterated rules—under the random set umbrella. In this article, we derive a new conditional event algebra, denoted GNW₂, for rules which are contingent on other rules. We show that this logic is the only one which extends the GNW first-degree logic, admits a simple embedding into random sets, and has GNW-like behavior. We show that GNW events are contained in a Boolean subalgebra GNW* of GNW₂. Finally, we show how GNW₂ can be used to recursively define a GNW-like Boolean logic for iterated rules of any degree. © 1996 John Wiley & Sons, Inc.     

The construction of a fuzzy inference network by extension of the rule inference network

Fuzzy logic can bring about inappropriate inferences as a result of ignoring some information in the reasoning process. Neural networks are powerful tools for pattern processing, but are not appropriate for the logical reasoning needed to model human knowledge. The use of a neural logic network derived from a modified neural network, however, makes logical reasoning possible. In this paper, we construct a fuzzy inference network by extending the rule–inference network based on an existing neural logic network. The propagation rule used in the existing rule–inference network is modified and applied. In order to determine the belief value of a proposition pertaining to the execution part of the fuzzy rules in a fuzzy inference network, the nodes connected to the proposition to be inferenced should be searched for. The search costs are compared and evaluated through application of sequential and priority searches for all the connected nodes.     

An ART-based fuzzy adaptive learning control network

This paper addresses the structure and an associated online learning algorithm of a feedforward multilayer neural net for realizing the basic elements and functions of a fuzzy controller. The proposed fuzzy adaptive learning control network (FALCON) can be contrasted with traditional fuzzy control systems in network structure and learning ability. An online structure/parameter learning algorithm, FALCON-ART, is proposed for constructing FALCON dynamically. It combines backpropagation for parameter learning and fuzzy ART for structure learning. FALCON-ART partitions the input state space and output control space using irregular fuzzy hyperboxes according to the data distribution. In many existing fuzzy or neural fuzzy control systems, the input and output spaces are always partitioned into “grids”. As the number of variables increases, the number of partitioned grids grows combinatorially. To avoid this problem in some complex systems, FALCON-ART partitions the I/O spaces flexibly based on data distribution. It can create and train FALCON in a highly autonomous way. In its initial form, there is no membership function, fuzzy partition, and fuzzy logic rule. They are created and begin to grow as the first training pattern arrives. Thus, the users need not give it any a priori knowledge or initial information. FALCON-ART can online partition the I/O spaces, tune membership functions, find proper fuzzy logic rules, and annihilate redundant rules dynamically upon receiving online data     

特征扩展的随机向量函数链神经网络

基于宽度学习的动态模糊推理系统(broad-learning-based dynamic fuzzy inference system , BL-DFIS)能自动构建出精简的模糊规则并获得良好的分类性能. 然而, 当遇到大型复杂的数据集时, BL-DFIS因会使用较多模糊规则来试图达到令人满意的识别精度, 从而对其可解释性造成了不利影响. 对此, 提出一种兼顾分类性能和可解释性的模糊神经网络, 将其称为特征扩展的随机向量函数链神经网络(FA-RVFLNN). 在该网络中, 一个以原始数据为输入的RVFLNN被作为主体结构, BL-DFIS则用作性能补充, 这意味着FA-RVFLNN包含具有性能增强作用的直接链接. 由于主体结构的增强节点使用Sigmoid激活函数, 因此, 其推理过程可借助一种模糊逻辑算子(I-OR)来解释. 而且, 具有明确含义的原始输入数据也有助于解释主体结构的推理规则. 在直接链接的支撑下, FA-RVFLNN可利用增强节点、特征节点和模糊节点学到更丰富的有用信息. 实验表明: FA-RVFLNN既减缓了主体结构RVFLNN中过多增强节点带来的“规则爆炸”问题, 也提高了性能补充结构BL-DFIS的可解释性(平均模糊规则数降低了50%左右), 在泛化性能和网络规模上仍具有竞争力.