Improved Estimates for the Accuracy of Small Disjuncts

Learning systems often describe a target class as a disjunction of conjunctions of conditions. Recent work has noted that small disjuncts, i.e., those supported by few training examples, typically have poor predictive accuracy. One model of this accuracy is provided by the Bayes-Laplace formula based on the number of training examples covered by the disjunct and the number of them belonging to the target class. However, experiments show that small disjuncts associated with target classes of different relative frequencies tend to have different error rates. This note defines the context of a disjunct as the set of training examples that fail to satisfy at most one of its conditions. An empirical adaptation of the Bayes-Laplace formula is presented that also makes use of the relative frequency of the target class in this context. Trials are reported comparing the performance of the original formula and the adaptation in six learning tasks.     

二维矩形条带装箱问题的底部左齐择优匹配算法

针对二维矩形条带装箱问题提出了一种启发式布局算法,即底部左齐择优匹配算法(lowest-level left align best fit,简称LLABF). LLABF算法遵循最佳匹配优先原则,该原则综合考虑完全匹配优先、宽度匹配优先、高度匹配优先、组合宽度匹配优先及可装入优先等启发式规则.与BL(bottom-left),IBL(improved-bottom-left)与BLF(bottom-left-fill)等启发算法不同的是,LLABF能够在矩形装入过程中自动选择与可装区域匹配的下一个待装矩形     

一种基于混合算法的分类器设计

为了高效地从数据库中挖掘分类规则,提出一种将粒子群优化算法和遗传算法相结合的新算法。该算法的核心思想是对规则的前件进行固定长度编码,适应度函数的计算由分类规则的准确率、置信度、支持度和简洁度构成,从而实现基于两者混合算法的分类器设计。将该分类器与遗传算法分类器和粒子群算法分类器进行对比,实验结果表明,该分类器具有更高的分类准确率以及更快的收敛速度。     

Cascade ARTMAP: integrating neural computation and symbolic knowledge processing

This paper introduces a hybrid system termed cascade adaptive resonance theory mapping (ARTMAP) that incorporates symbolic knowledge into neural-network learning and recognition. Cascade ARTMAP, a generalization of fuzzy ARTMAP, represents intermediate attributes and rule cascades of rule-based knowledge explicitly and performs multistep inferencing. A rule insertion algorithm translates if-then symbolic rules into cascade ARTMAP architecture. Besides that initializing networks with prior knowledge can improve predictive accuracy and learning efficiency, the inserted symbolic knowledge can be refined and enhanced by the cascade ARTMAP learning algorithm. By preserving symbolic rule form during learning, the rules extracted from cascade ARTMAP can be compared directly with the originally inserted rules. Simulations on an animal identification problem indicate that a priori symbolic knowledge always improves system performance, especially with a small training set. Benchmark study on a DNA promoter recognition problem shows that with the added advantage of fast learning, cascade ARTMAP rule insertion and refinement algorithms produce performance superior to those of other machine learning systems and an alternative hybrid system known as knowledge-based artificial neural network (KBANN). Also, the rules extracted from cascade ARTMAP are more accurate and much cleaner than the NofM rules extracted from KBANN.     

Separable resolution method for checking the satisfiability of formulas in the languageL

Conclusion This paper offers an effective resolution method for checking the satisfiability of a collection of disjuncts in the languageL. The method permits one to substantially reduce the number of generated resolvents in comparison with the method ofR-resolution. One more factor ensuring the efficiency of the method is a significant reduction in the number of disjunct pairs checked for the possibility of resolving them. As for the number of generated disjuncts, its greatest reduction is obtained in the case of the use of the disjunct-set partition corresponding to the limiting system of predicate-symbol subsets given by symbol ordering. It is possible to interpret the result obtained for this case as proof of the completeness of the strategy combining an ordering of predicate symbols andR-resolution. It is necessary to note that to different orderings of predicate symbols correspond different partitions of the disjunct set giving, in turn, different numbers of generated disjuncts in the process ofSp-completion. Nevertheless, the methods described in Sec. 2, which use partition of the disjunct set into two classes, are of independent importance. As has already been said, completion of a disjunct set is used for solution of a number of problems during the design of a procedural automaton specification. For example, in the case of checking the consistency of two interacting automata [5] based on completion of a disjunct set, there exists a natural partition of the predicate symbols into input and output symbols, to which corresponds the partition of the disjunct set into subsets specifying the interacting automata. Translated from Kibernetika i Sistemnyi Analiz, No. 6, pp. 13–20, November–December, 1998.     

A hybrid coevolutionary algorithm for designing fuzzy classifiers

Rule learning is one of the most common tasks in knowledge discovery. In this paper, we investigate the induction of fuzzy classification rules for data mining purposes, and propose a hybrid genetic algorithm for learning approximate fuzzy rules. A novel niching method is employed to promote coevolution within the population, which enables the algorithm to discover multiple rules by means of a coevolutionary scheme in a single run. In order to improve the quality of the learned rules, a local search method was devised to perform fine-tuning on the offspring generated by genetic operators in each generation. After the GA terminates, a fuzzy classifier is built by extracting a rule set from the final population. The proposed algorithm was tested on datasets from the UCI repository, and the experimental results verify its validity in learning rule sets and comparative advantage over conventional methods.     

A hybrid model of fuzzy ARTMAP and genetic algorithm for data classification and rule extraction

A two-stage hybrid model for data classification and rule extraction is proposed. The first stage uses a Fuzzy ARTMAP (FAM) classifier with Q-learning (known as QFAM) for incremental learning of data samples, while the second stage uses a Genetic Algorithm (GA) for rule extraction from QFAM. Given a new data sample, the resulting hybrid model, known as QFAM-GA, is able to provide prediction pertaining to the target class of the data sample as well as to give a fuzzy if-then rule to explain the prediction. To reduce the network complexity, a pruning scheme using Q-values is applied to reduce the number of prototypes generated by QFAM. A ‘don't care’ technique is employed to minimize the number of input features using the GA. A number of benchmark problems are used to evaluate the effectiveness of QFAM-GA in terms of test accuracy, noise tolerance, model complexity (number of rules and total rule length). The results are comparable, if not better, than many other models reported in the literature. The main significance of this research is a usable and useful intelligent model (i.e., QFAM-GA) for data classification in noisy conditions with the capability of yielding a set of explanatory rules with minimum antecedents. In addition, QFAM-GA is able to maximize accuracy and minimize model complexity simultaneously. The empirical outcome positively demonstrate the potential impact of QFAM-GA in the practical environment, i.e., providing an accurate prediction with a concise justification pertaining to the prediction to the domain users, therefore allowing domain users to adopt QFAM-GA as a useful decision support tool in assisting their decision-making processes.     

Developing soft sensors using hybrid soft computing methodology: a neurofuzzy system based on rough set theory and genetic algorithms

This paper presents a hybrid soft computing modeling approach, a neurofuzzy system based on rough set theory and genetic algorithms (GA). To solve the curse of dimensionality problem of neurofuzzy system, rough set is used to obtain the reductive fuzzy rule set. Both the number of condition attributes and rules are reduced. Genetic algorithm is used to obtain the optimal discretization of continuous attributes. The fuzzy system is then represented via an equivalent artificial neural network (ANN). Because the initial parameter of the ANN is reasonable, the convergence of the ANN training is fast. After the rules are reduced, the structure size of the ANN becomes small, and the ANN is not fully weight-connected. The neurofuzzy approach based on RST and GA has been applied to practical application of building a soft sensor model for estimating the freezing point of the light diesel fuel in fluid catalytic cracking unit.     

Reverse Engineering the Neural Networks for Rule Extraction in Classification Problems

Artificial neural networks often achieve high classification accuracy rates, but they are considered as black boxes due to their lack of explanation capability. This paper proposes the new rule extraction algorithm RxREN to overcome this drawback. In pedagogical approach the proposed algorithm extracts the rules from trained neural networks for datasets with mixed mode attributes. The algorithm relies on reverse engineering technique to prune the insignificant input neurons and to discover the technological principles of each significant input neuron of neural network in classification. The novelty of this algorithm lies in the simplicity of the extracted rules and conditions in rule are involving both discrete and continuous mode of attributes. Experimentation using six different real datasets namely iris, wbc, hepatitis, pid, ionosphere and creditg show that the proposed algorithm is quite efficient in extracting smallest set of rules with high classification accuracy than those generated by other neural network rule extraction methods.     

基于IQPSO-GA优化ANFIS模型的服务器故障预警方法

针对服务器底层部分业务类硬件故障对系统稳定运行的影响,提出一种改进的量子行为粒子群优化(IQPSO)与遗传算法(GA)相结合的混合元启发式优化算法对自适应神经模糊推理系统(ANFIS)参数进行训练,以获得更准确的ANFIS规则进行硬件故障预警的方法。首先,通过分析服务器业务与硬件相关参数之间的映射关系,通过采集的数据集对ANFIS模型进行训练构造预测模型;其次,考虑ANFIS在梯度计算过程中存在容易陷入局部最优值的问题,设计了一种IQPSO算法结合GA中的交叉和变异算子操作混合元启发算法全局搜索ANFIS规则参数;最后,通过一组后处理样本数据集对所提方法有效性和稳定性进行了检验。实验结果表明,该方法可有效预警服务器硬件故障,基于所提混合元启发优化算法获得的ANFIS模型具备更快的收敛速度和更高的全局搜索精度,与传统ANFIS模型相比泛化精度提高了47%以上。     

A technical note on the paper “hGA: Hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems”

This paper provides a corrected formulation to the mixed integer programming model proposed by Aydogan et al. (2012) [1]. They proposed a genetic algorithm to learn fuzzy rules for a fuzzy rule-based classification system and developed a Mixed Integer Programming model (MIP) to prune the generated rules by selecting the best set of rules to maximize predictive accuracy. However, their proposed MIP formulation contains errors, which are described in this technical note. We develop corrections and improvements to the original formulation and test it with non-parametric statistical tests on the same data sets used to evaluate the original model. The statistical analysis shows that the results of the correction formulation are significantly different from the original model.     

Mining the optimal class association rule set

We define an optimal class association rule set to be the minimum rule set with the same predictive power of the complete class association rule set. Using this rule set instead of the complete class association rule set we can avoid redundant computation that would otherwise be required for mining predictive association rules and hence improve the efficiency of the mining process significantly. We present an efficient algorithm for mining the optimal class association rule set using an upward closure property of pruning weak rules before they are actually generated. We have implemented the algorithm and our experimental results show that our algorithm generates the optimal class association rule set, whose size is smaller than 1/17 of the complete class association rule set on average, in significantly less rime than generating the complete class association rule set. Our proposed criterion has been shown very effective for pruning weak rules in dense databases.     

Generating fuzzy rules for target tracking using a steady-stategenetic algorithm

Radar target tracking involves predicting the future trajectory of a target based on its past positions. This problem has been dealt with using trackers developed under various assumptions about statistical models of process and measurement noise and about target dynamics. Due to these assumptions, existing trackers are not very effective when executed in a stressful environment in which a target may maneuver, accelerate, or decelerate and its positions be inaccurately detected or missing completely from successive scans. To deal with target tracking in such an environment, recent efforts have developed fuzzy logic-based trackers. These have been shown to perform better as compared to traditional trackers. Unfortunately, however, their design may not be easier. For these trackers to perform effectively, a set of carefully chosen fuzzy rules are required. These rules are currently obtained from human experts through a time-consuming knowledge acquisition process of iterative interviewing, verifying, validating, and revalidating. To facilitate the knowledge acquisition process and ensure that the best possible set of rules be found, we propose to use an automatic rule generator that was developed based on the use of a genetic algorithm (GA). This genetic algorithm adopts a steady-state reproductive scheme and is referred to as the steady-state genetic algorithm (SSGA) in this paper. To generate fuzzy rules, we encode different rule sets in different chromosomes. Chromosome fitness is then determined according to a fitness function defined in terms of the number of track losses and the prediction accuracy when the set of rules it encodes is tested against training data. The rules encoded in the fittest chromosome at the end of the evolutionary process are taken to be the best possible set of fuzzy rules     

A hybrid genetic algorithm–adaptive network-based fuzzy inference system in prediction of wave parameters

An important issue in application of fuzzy inference systems (FISs) to a class of system identification problems such as prediction of wave parameters is to extract the structure and type of fuzzy if–then rules from an available input–output data set. In this paper, a hybrid genetic algorithm–adaptive network-based FIS (GA–ANFIS) model has been developed in which both clustering and rule base parameters are simultaneously optimized using GAs and artificial neural nets (ANNs). The parameters of a subtractive clustering method, by which the number and structure of fuzzy rules are controlled, are optimized by GAs within which ANFIS is called for tuning the parameters of rule base generated by GAs. The model has been applied in the prediction of wave parameters, i.e. wave significant height and peak spectral period, in a duration-limited condition in Lake Michigan. The data set of year 2001 has been used as training set and that of year 2004 as testing data. The results obtained by the proposed model are presented and analyzed. Results indicate that GA–ANFIS model is superior to ANFIS and Shore Protection Manual (SPM) methods in terms of their prediction accuracy.     

一种挖掘数值属性的二维优化关联规则方法

优化关联规则允许在规则中包含未初始化的属性.优化过程就是确定对这些属性进行初始化,使得某些度量最大化.最大化兴趣度因子用来发现更加有趣的规则;另一方面,允许优化规则在前提和结果中各包含一个未初始化的数值属性.对那些处理一个数值属性的算法进行直接的扩展,可以得到一个发现这种优化规则的简单算法.然而这种方法的性能很差,因此,为了改善性能,提出一种启发式方法,它发现的是近似最优的规则.在人造数据集上的实验结果表明,当优化规则包含两个数值属性时,优化兴趣度因子得到的规则比优化可信度得到的规则更有趣.在真实数据集上的实验结果表明,该算法具有近似线性的可扩展性和较好的精度.     

核属性蚁群算法的规则获取

蚁群算法是一种新型的模拟进化算法,研究已经表明该算法具有许多优良的性质,并且在优化计算中已得到了很多应用.粗糙集理论作为一种智能数据分析和数据挖掘的新的数学工具,其主要优点在于它不需要任何关于被处理数据的先验或额外知识.本文从规则获取和优化两方面研究基于粗糙集理论和蚁群算法的分类规则挖掘方法.通过研究决策表和决策规则系数,建立基于粗糙集表示和度量的知识理论,将粗糙集理论与蚁群算法融合,采用粗糙集理论进行属性约简,利用蚁群算法获取最优分类规则,优势互补.实验结果比较表明,算法获取的分类规则,具有良好的预测能力和更为简洁的表示形式.     

Multi-objective genetic algorithm based approaches for mining optimized fuzzy association rules

Association rules form one of the most widely used techniques to discover correlations among attribute in a database. So far, some efficient methods have been proposed to obtain these rules with respect to an optimal goal, such as: to maximize the number of large itemsets and interesting rules or the values of support and confidence for the discovered rules. This paper first introduces optimized fuzzy association rule mining in terms of three important criteria; strongness, interestingness and comprehensibility. Then, it proposes multi-objective Genetic Algorithm (GA) based approaches for discovering these optimized rules. Optimization technique according to given criterion may be one of two different forms; The first tries to determine the appropriate fuzzy sets of quantitative attributes in a prespecified rule, which is also called as certain rule. The second deals with finding both uncertain rules and their appropriate fuzzy sets. Experimental results conducted on a real data set show the effectiveness and applicability of the proposed approach.     

hGA: Hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems

The aim of this work is to propose a hybrid heuristic approach (called hGA) based on genetic algorithm (GA) and integer-programming formulation (IPF) to solve high dimensional classification problems in linguistic fuzzy rule-based classification systems. In this algorithm, each chromosome represents a rule for specified class, GA is used for producing several rules for each class, and finally IPF is used for selection of rules from a pool of rules, which are obtained by GA. The proposed algorithm is experimentally evaluated by the use of non-parametric statistical tests on seventeen classification benchmark data sets. Results of the comparative study show that hGA is able to discover accurate and concise classification rules.     

An intelligent hybrid trading system for discovering trading rules for the futures market using rough sets and genetic algorithms

Discovering intelligent technical trading rules from nonlinear and complex stock market data, and then developing decision support trading systems, is an important challenge. The objective of this study is to develop an intelligent hybrid trading system for discovering technical trading rules using rough set analysis and a genetic algorithm (GA). In order to obtain better trading decisions, a novel rule discovery mechanism using a GA approach is proposed for solving optimization problems (i.e., data discretization and reducts) of rough set analysis when discovering technical trading rules for the futures market. Experiments are designed to test the proposed model against comparable approaches (i.e., random, correlation, and GA approaches). In addition, these comprehensive experiments cover most of the current trading system topics, including the use of a sliding window method (with or without validation dataset), the number of trading rules, and the size of training period. To evaluate an intelligent hybrid trading system, experiments were carried out on the historical data of the Korea Composite Stock Price Index 200 (KOSPI 200) futures market. In particular, trading performance is analyzed according to the number of sets of decision rules and the size of the training period for discovering trading rules for the testing period. The results show that the proposed model significantly outperforms the benchmark model in terms of the average return and as a risk-adjusted measure.     

Hybrid particle swarm optimization for rule discovery in the diagnosis of coronary artery disease

Coronary artery disease (CAD) is one of the major causes of mortality worldwide. Knowledge about risk factors that increase the probability of developing CAD can help to understand the disease better and assist in its treatment. Recently, modern computer‐aided approaches have been used for the prediction and diagnosis of diseases. Swarm intelligence algorithms like particle swarm optimization (PSO) have demonstrated great performance in solving different optimization problems. As rule discovery can be modelled as an optimization problem, it can be mapped to an optimization problem and solved by means of an evolutionary algorithm like PSO. An approach for discovering classification rules of CAD is proposed. The work is based on the real‐world CAD data set and aims at the detection of this disease by producing the accurate and effective rules. The proposed algorithm is a hybrid binary‐real PSO, which includes the combination of categorical and numerical encoding of a particle and a different approach for calculating the velocity of particles. The rules were developed from randomly generated particles, which take random values in the range of each attribute in the rule. Two different feature selection methods based on multi‐objective evolutionary search and PSO were applied on the data set, and the most relevant features were selected by the algorithms. The accuracy of two different rule sets were evaluated. The rule set with 11 features obtained more accurate results than the rule set with 13 features. Our results show that the proposed approach has the ability to produce effective rules with highest accuracy for the detection of CAD.