OWA trees and their role in security modeling using attack trees

We introduce the idea of an OWA node as an extension of the “and/or” node and use them to generalize “and/or” trees to OWA trees. We provide a semantics for an OWA node. Specifically while an “or” node requires only one of the children to be satisfied and an “and” node requires “all” the children be satisfied the OWA node allows us to model situations in which there is some probabilistic uncertainty in the number of children that need be satisfied. We then use these OWA nodes in the security related problem of constructing attack trees. Techniques for the evaluation of an OWA attack tree for the overall probability of success and cost of an attack are provided. A method is provided for describing different types of attackers.     

Creating Decision Trees from Rules using RBDT‐1

Most of the methods that generate decision trees for a specific problem use the examples of data instances in the decision tree–generation process. This article proposes a method called RBDT‐1—rule‐based decision tree—for learning a decision tree from a set of decision rules that cover the data instances rather than from the data instances themselves. The goal is to create on demand a short and accurate decision tree from a stable or dynamically changing set of rules. The rules could be generated by an expert, by an inductive rule learning program that induces decision rules from the examples of decision instances such as AQ‐type rule induction programs, or extracted from a tree generated by another method, such as the ID3 or C4.5. In terms of tree complexity (number of nodes and leaves in the decision tree), RBDT‐1 compares favorably with AQDT‐1 and AQDT‐2, which are methods that create decision trees from rules. RBDT‐1 also compares favorably with ID3 while it is as effective as C4.5 where both (ID3 and C4.5) are well‐known methods that generate decision trees from data examples. Experiments show that the classification accuracies of the decision trees produced by all methods under comparison are indistinguishable.     

Evolving Fuzzy Min–Max Neural Network Based Decision Trees for Data Stream Classification

Learning from data streams is a challenging task which demands a learning algorithm with several high quality features. In addition to space complexity and speed requirements needed for processing the huge volume of data which arrives at high speed, the learning algorithm must have a good balance between stability and plasticity. This paper presents a new approach to induce incremental decision trees on streaming data. In this approach, the internal nodes contain trainable split tests. In contrast with traditional decision trees in which a single attribute is selected as the split test, each internal node of the proposed approach contains a trainable function based on multiple attributes, which not only provides the flexibility needed in the stream context, but also improves stability. Based on this approach, we propose evolving fuzzy min–max decision tree (EFMMDT) learning algorithm in which each internal node of the decision tree contains an evolving fuzzy min–max neural network. EFMMDT splits the instance space non-linearly based on multiple attributes which results in much smaller and shallower decision trees. The extensive experiments reveal that the proposed algorithm achieves much better precision in comparison with the state-of-the-art decision tree learning algorithms on the benchmark data streams, especially in the presence of concept drift.     

Induction of multiple fuzzy decision trees based on rough set technique

The integration of fuzzy sets and rough sets can lead to a hybrid soft-computing technique which has been applied successfully to many fields such as machine learning, pattern recognition and image processing. The key to this soft-computing technique is how to set up and make use of the fuzzy attribute reduct in fuzzy rough set theory. Given a fuzzy information system, we may find many fuzzy attribute reducts and each of them can have different contributions to decision-making. If only one of the fuzzy attribute reducts, which may be the most important one, is selected to induce decision rules, some useful information hidden in the other reducts for the decision-making will be losing unavoidably. To sufficiently make use of the information provided by every individual fuzzy attribute reduct in a fuzzy information system, this paper presents a novel induction of multiple fuzzy decision trees based on rough set technique. The induction consists of three stages. First several fuzzy attribute reducts are found by a similarity based approach, and then a fuzzy decision tree for each fuzzy attribute reduct is generated according to the fuzzy ID3 algorithm. The fuzzy integral is finally considered as a fusion tool to integrate the generated decision trees, which combines together all outputs of the multiple fuzzy decision trees and forms the final decision result. An illustration is given to show the proposed fusion scheme. A numerical experiment on real data indicates that the proposed multiple tree induction is superior to the single tree induction based on the individual reduct or on the entire feature set for learning problems with many attributes.     

Geometric decision tree

In this paper, we present a new algorithm for learning oblique decision trees. Most of the current decision tree algorithms rely on impurity measures to assess the goodness of hyperplanes at each node while learning a decision tree in top-down fashion. These impurity measures do not properly capture the geometric structures in the data. Motivated by this, our algorithm uses a strategy for assessing the hyperplanes in such a way that the geometric structure in the data is taken into account. At each node of the decision tree, we find the clustering hyperplanes for both the classes and use their angle bisectors as the split rule at that node. We show through empirical studies that this idea leads to small decision trees and better performance. We also present some analysis to show that the angle bisectors of clustering hyperplanes that we use as the split rules at each node are solutions of an interesting optimization problem and hence argue that this is a principled method of learning a decision tree.     

多决策树的模糊积分融合在银行信贷管理系统中的应用

为了提高基于数据挖掘的商业银行信贷管理系统的信贷风险评估水平,将多决策树的Choquet模糊积分融合(MTCFF)模型应用到银行信贷管理系统中。基本思想是采用决策树在已知类型的客户数据上进行挖掘,按照决策树剪枝程度不同形成不同的决策树并产生规则,利用所生成的不同决策树的规则,对未知类型的客户数据进行分类,然后让Choquet模糊积分对多棵决策树的分类结果进行融合,形成最优判断。采用UCI数据库中German客户信用卡数据集进行验证,实验证明Choquet模糊积分的非线性融合效果优于单棵决策树的分类效果,也优于其他线性融合方法,并且Choquet模糊积分要优于Sugeno模糊积分。     

Machine learning approach to model sport training

The aim of this study was to use a machine learning approach combining fuzzy modeling with an immune algorithm to model sport training, in particular swimming. A proposed algorithm mines the available data and delivers the results in a form of a set of fuzzy rules “IF (fuzzy conditions) THEN (class)”. Fuzzy logic is a powerful method to cope with continuous data, to overcome problem of overlapping class definitions, and to improve the rule comprehensibility. Sport training is modeled at the level of microcycle and training unit by 12 independent attributes. The data was collected in two months (February-March 2008), among swimmers from swimming sections in Wroc?aw, Poland. The swimmers had minimum of 7 years of training and reached the II class level in swimming classification from 2005 to 2008. The goal of the performed experiments was to find the rules answering the question - how does the training unit influence swimmer’s feelings while being in water the next day? The fuzzy rules were inferred for two different scales of the class to be predicted. The effectiveness of the learned set of rules reached 68.66%. The performance, in terms of classification accuracy, of the proposed approach was compared with traditional classifier schemes. The accuracy of the result of compared methods is significantly lower than the accuracy of fuzzy rules obtained by a method presented in this study (paired t-test, P < 0.05).     

Type-2 fuzzy neural networks with fuzzy clustering and differential evolution optimization

In many real-world problems involving pattern recognition, system identification and modeling, control, decision making, and forecasting of time-series, available data are quite often of uncertain nature. An interesting alternative is to employ type-2 fuzzy sets, which augment fuzzy models with expressive power to develop models, which efficiently capture the factor of uncertainty. The three-dimensional membership functions of type-2 fuzzy sets offer additional degrees of freedom that make it possible to directly and more effectively account for model’s uncertainties. Type-2 fuzzy logic systems developed with the aid of evolutionary optimization forms a useful modeling tool subsequently resulting in a collection of efficient “If-Then” rules.The type-2 fuzzy neural networks take advantage of capabilities of fuzzy clustering by generating type-2 fuzzy rule base, resulting in a small number of rules and then optimizing membership functions of type-2 fuzzy sets present in the antecedent and consequent parts of the rules. The clustering itself is realized with the aid of differential evolution.Several examples, including a benchmark problem of identification of nonlinear system, are considered. The reported comparative analysis of experimental results is used to quantify the performance of the developed networks.     

Incorporation of Fuzziness in ID3 and Generation of Network Architerture

This article first describes a fuzzy version of ID3, called fuzzy ID3 by incorporating fuzziness at input, output and node levels. A fuzziness measure is computed at each node, in terms of class membership, to take care of the uncertainty arising from overlapping regions. The measure is such that in the crisp (non-overlapping) case, the algorithm boils down to the classical ID3. A confidence factor is estimated at the nodes for both making a decision and determining the rule base for network mapping. In the second part, we deal with a scheme of designing a fuzzy knowledge-based network by encoding an MLP with the rules generated using fuzzy ID3, whereby the network topology is automatically determined. The frequency of samples (representative of a rule) and the confidence factors of unresolved/ambiguous nodes are taken into consideration during mapping. The effectiveness of the system, in terms of recognition scores and speed of convergence, is demonstrated on two real life data sets.       

A computational intelligence technique for the effective diagnosis of diabetic patients using principal component analysis (PCA) and modified fuzzy SLIQ decision tree approach

Knowledge inference systems are built to identify hidden and logical patterns in huge data. Decision trees play a vital role in knowledge discovery but crisp decision tree algorithms have a problem with sharp decision boundaries which may not be implicated to all knowledge inference systems. A fuzzy decision tree algorithm overcomes this drawback. Fuzzy decision trees are implemented through fuzzification of the decision boundaries without disturbing the attribute values. Data reduction also plays a crucial role in many classification problems. In this research article, it presents an approach using principal component analysis and modified Gini index based fuzzy SLIQ decision tree algorithm. The PCA is used for dimensionality reduction, and modified Gini index fuzzy SLIQ decision tree algorithm to construct decision rules. Finally, through PID data set, the method is validated in the simulation experiment in MATLAB.     

规则简化与模糊决策树剪枝的比较

决策树归纳学习算法是机器学习领域中解决分类问题的最有效工具之一。由于决策树算法自身的缺陷了，因此需要进行相应的简化来提高预测精度。模糊决策树算法是对决策树算法的一种改进，它更加接近人的思维方式。文章通过实验分析了模糊决策树、规则简化与模糊规则简化；模糊决策树与模糊预剪枝算法的异同，对决策树的大小、算法的训练准确率与测试准确率进行比较，分析了模糊决策树的性能，为改进该算法提供了一些有益的线索。     

Two-stage binary classifier with fuzzy-valued loss function

In this paper we present the decision rules of a two-stage binary Bayesian classifier. The loss function in our case is fuzzy-valued and is dependent on the stage of the decision tree or on the node of the decision tree. The decision rules minimize the mean risk, i.e., the mean value of the fuzzy loss function. The model is first based on the notion of fuzzy random variable and secondly on the subjective ranking of fuzzy number defined by Campos and González. In this paper also, influence of choice of parameter λ in selected comparison fuzzy number method on classification results are presented. Finally, an example illustrating the study developed in the paper is considered.     

Building contextual classifiers by integrating fuzzy rule based classification technique and k-nn method for credit scoring

Credit-risk evaluation is a very challenging and important problem in the domain of financial analysis. Many classification methods have been proposed in the literature to tackle this problem. Statistical and neural network based approaches are among the most popular paradigms. However, most of these methods produce so-called “hard” classifiers, those generate decisions without any accompanying confidence measure. In contrast, “soft” classifiers, such as those designed using fuzzy set theoretic approach; produce a measure of support for the decision (and also alternative decisions) that provides the analyst with greater insight. In this paper, we propose a method of building credit-scoring models using fuzzy rule based classifiers. First, the rule base is learned from the training data using a SOM based method. Then the fuzzy k-nn rule is incorporated with it to design a contextual classifier that integrates the context information from the training set for more robust and qualitatively better classification. Further, a method of seamlessly integrating business constraints into the model is also demonstrated.     

Development of fuzzy process control charts and fuzzy unnatural pattern analyses

Many problems in scientific investigation generate nonprecise data incorporating nonstatistical uncertainty. A nonprecise observation of a quantitative variable can be described by a special type of membership function defined on the set of all real numbers called a fuzzy number or a fuzzy interval. A methodology for constructing control charts is proposed when the quality characteristics are vague, uncertain, incomplete or linguistically defined. Fuzzy set theory is an inevitable tool for fuzzy control charts as well as other applications subjected to uncertainty in any form. The vagueness can be handled by transforming incomplete or nonprecise quantities to their representative scalar values such as fuzzy mode, fuzzy midrange, fuzzy median, or fuzzy average. Then crisp methods may be applied to those representative values for control chart decisions as “in control” or “out of control”. Transforming the vague data by using one of the transformation methods may result in biased decisions since the information given by the vague data is lost by the transformation. Such data needs to be investigated as fuzzy sets without transformation, and the decisions based on the vague data should not be concluded with an exact decision. A “direct fuzzy approach (DFA)” to fuzzy control charts for attributes under vague data is proposed without using any transformation method. Then, the unnatural patterns for the proposed fuzzy control charts are defined using the probabilities of fuzzy events.     

A Bayesian approach to fuzzy hypotheses testing for the estimation of optimal age for vaccination against measles

Fuzzy Bayesian tests were performed to evaluate whether the mother’s seroprevalence and children’s seroconversion to measles vaccine could be considered as “high” or “low”. The results of the tests were aggregated into a fuzzy rule-based model structure, which would allow an expert to influence the model results. The linguistic model was developed considering four input variables. As the model output, we obtain the recommended age-specific vaccine coverage. The inputs of the fuzzy rules are fuzzy sets and the outputs are constant functions, performing the simplest Takagi–Sugeno–Kang model. This fuzzy approach is compared to a classical one, where the classical Bayes test was performed. Although the fuzzy and classical performances were similar, the fuzzy approach was more detailed and revealed important differences. In addition to taking into account subjective information in the form of fuzzy hypotheses it can be intuitively grasped by the decision maker.     

Discovering a cover set of ARsi with hierarchy from quantitative databases

This paper extends the work on discovering fuzzy association rules with degrees of support and implication (ARsi). The effort is twofold: one is to discover ARsi with hierarchy so as to express more semantics due to the fact that hierarchical relationships usually exist among fuzzy sets associated with the attribute concerned; the other is to generate a “core” set of rules, namely the rule cover set, that are of more interest in a sense that all other rules could be derived by the cover set. Corresponding algorithms for ARsi with hierarchy and the cover set are proposed along with pruning strategies incorporated to improve the computational efficiency. Some data experiments are conducted as well to show the effectiveness of the approach.     

A Further Comparison of Splitting Rules for Decision-Tree Induction

One approach to learning classification rules from examples is to build decision trees. A review and comparison paper by Mingers (Mingers, 1989) looked at the first stage of tree building, which uses a splitting rule to grow trees with a greedy recursive partitioning algorithm. That paper considered a number of different measures and experimentally examined their behavior on four domains. The main conclusion was that a random splitting rule does not significantly decrease classificational accuracy. This note suggests an alternative experimental method and presents additional results on further domains. Our results indicate that random splitting leads to increased error. These results are at variance with those presented by Mingers.     

Design and implementation of a fuzzy expert system for performance assessment of an integrated health, safety, environment (HSE) and ergonomics system: The case of a gas refinery

The objective of this study is to design a fuzzy expert system for performance assessment of health, safety, environment (HSE) and ergonomics system factors in a gas refinery. This will lead to a robust control system for continuous assessment and improvement of HSE and ergonomics performance. The importance of this study stems from the current lack of formal integrated methodologies for interpreting and evaluating performance data for HSE and ergonomics. Three important reasons to use fuzzy expert systems are (1) reduction of human error, (2) creation of expert knowledge and (3) interpretation of large amount of vague data. To achieve the objective of this study, standard indicators and technical tolerances for assessment of HSE and ergonomics factors are identified. Then, data is collected for all indicators and consequently, for each indicator four conditions are defined as “acceptance”, “low deviation”, “mid deviation” and “high deviation”. A membership function is defined for each fuzzy condition (set) because an indicator cannot be allocated to just one of the above conditions. The expert system uses fuzzy rules, which are structured with Data Engine. Previous studies have introduced HSE expert system whereas this study introduces an integrated HSE and ergonomics expert system through fuzzy logic.     

GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems

In this paper we propose GP-COACH, a Genetic Programming-based method for the learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems. GP-COACH learns disjunctive normal form rules (generated by means of a context-free grammar) coded as one rule per tree. The population constitutes the rule base, so it is a genetic cooperative-competitive learning approach. GP-COACH uses a token competition mechanism to maintain the diversity of the population and this obliges the rules to compete and cooperate among themselves and allows the obtaining of a compact set of fuzzy rules. The results obtained have been validated by the use of non-parametric statistical tests, showing a good performance in terms of accuracy and interpretability.     

Interpretable hierarchical clustering by constructing an unsupervised decision tree

We propose a method for hierarchical clustering based on the decision tree approach. As in the case of supervised decision tree, the unsupervised decision tree is interpretable in terms of rules, i.e., each leaf node represents a cluster, and the path from the root node to a leaf node represents a rule. The branching decision at each node of the tree is made based on the clustering tendency of the data available at the node. We present four different measures for selecting the most appropriate attribute to be used for splitting the data at every branching node (or decision node), and two different algorithms for splitting the data at each decision node. We provide a theoretical basis for the approach and demonstrate the capability of the unsupervised decision tree for segmenting various data sets. We also compare the performance of the unsupervised decision tree with that of the supervised one.