Competition-Based Induction of Decision Models from Examples

Symbolic induction is a promising approach to constructing decision models by extracting regularities from a data set of examples. The predominant type of model is a classification rule (or set of rules) that maps a set of relevant environmental features into specific categories or values. Classifying loan risk based on borrower profiles, consumer choice from purchase data, or supply levels based on operating conditions are all examples of this type of model-building task. Although current inductive approaches, such as ID3 and CN2, perform well on certain problems, their potential is limited by the incremental nature of their search. Genetic algorithms (GA) have shown great promise on complex search domains, and hence suggest a means for overcoming these limitations. However, effective use of genetic search in this context requires a framework that promotes the fundamental model-building objectives of predictive accuracy and model simplicity. In this article we describe COGIN, a GA-based inductive system that exploits the conventions of induction from examples to provide this framework. The novelty of COGIN lies in its use of training set coverage to simultaneously promote competition in various classification niches within the model and constrain overall model complexity. Experimental comparisons with NewID and CN2 provide evidence of the effectiveness of the COGIN framework and the viability of the GA approach.     

Unifying Instance-Based and Rule-Based Induction

Several well-developed approaches to inductive learning low exist, but each has specific limitations that are hard to overcome. Multi-strategy learning attempts to tackle this problem combining multiple methods in one algorithm. This article describes a unification of two widely-used empirical approaches: rule induction and instance-based learning. In the new algorithm, instances are treated as maximally specific rules, and classification is oerformed using a best-match strategy. Rules are learned by gradually generalizing instances until no improvement in apparent accuracy is obtained. Theoretical analysis shows this approach to be efficient. It is implemented in the RISE 3.1 system. In an extensive empirical study, RISE consistently achieves higher accuracies than state-of-the-art representatives of both its parent approaches (PEBLS and CN2), as well as a decision tree learner (C4.5). Lesion studies show that eachoof RISE's components is essential to this performance. Most significantly, in 14 of the 30 domains studied, RISE is more accurate than the best of PEBLS and CN2, showing that a significant synergy can be obtained by combining multiple empirical methods.     

CN2 model to calculate the micrometeorological influences on the refractive index structure parameter

This paper presents the CN2 model for calculating the micrometeorological influences on the refractive index structure parameter. The CN2 model is a semi-empirical algorithm developed by the U.S. Army Research Laboratory to provide realistic values for the refractive index structure parameter over land given two vertical levels of conventional wind speed, temperature, and humidity data as input. The CN2 model is based on the structure function formulations of Tatarski. Equations for the real index of refraction are expressed in terms of temperature, pressure, and moisture (preferably the conserved elements potential temperature and specific humidity). Calculations of micrometeorological profile structure are carried out via Monin–Obukhov theory for the surface layer. Model results can be derived for unstable, stable, and near-neutral atmospheric conditions. The CN2 model algorithm is validated in comparison to optical scintillometer data collected at 2 m above ground level over a 450 m path. The paper reported here contains technical and user’s guide information on the CN2 model.     

Data mining with an ant colony optimization algorithm

The paper proposes an algorithm for data mining called Ant-Miner (ant-colony-based data miner). The goal of Ant-Miner is to extract classification rules from data. The algorithm is inspired by both research on the behavior of real ant colonies and some data mining concepts as well as principles. We compare the performance of Ant-Miner with CN2, a well-known data mining algorithm for classification, in six public domain data sets. The results provide evidence that: 1) Ant-Miner is competitive with CN2 with respect to predictive accuracy, and 2) the rule lists discovered by Ant-Miner are considerably simpler (smaller) than those discovered by CN2     

Learning classification rules from data

We present ELEM2, a machine learning system that induces classification rules from a set of data based on a heuristic search over a hypothesis space. ELEM2 is distinguished from other rule induction systems in three aspects. First, it uses a new heuristtic function to guide the heuristic search. The function reflects the degree of relevance of an attribute-value pair to a target concept and leads to selection of the most relevant pairs for formulating rules. Second, ELEM2 handles inconsistent training examples by defining an unlearnable region of a concept based on the probability distribution of that concept in the training data. The unlearnable region is used as a stopping criterion for the concept learning process, which resolves conflicts without removing inconsistent examples. Third, ELEM2 employs a new rule quality measure in its post-pruning process to prevent rules from overfitting the data. The rule quality formula measures the extent to which a rule can discriminate between the positive and negative examples of a class. We describe features of ELEM2, its rule induction algorithm and its classification procedure. We report experimental results that compare ELEM2 with C4.5 and CN2 on a number of datasets.     

Multiple Translational Containment Part II: Exact Algorithms

We present exact algorithms for finding a solution to the two-dimensional translational containment problem: find translations for k polygons which place them inside a polygonal container without overlapping. The term kCN denotes the version in which the polygons are convex and the container is nonconvex, and the term kNN denotes the version in which the polygons and the container are nonconvex. The notation (r,k)CN, (r,k)NN, and so forth refers to the problem of finding all subsets of size k out of r objects that can be placed in a container. The polygons have up to m vertices, and the container has n vertices, where n is usually much larger than m. We present exact algorithms for the following: 2CN in time, (r,2)CN in time (for ), 3CN in time, kCN in or time, and kNN in time, where LP(a,b) is the time to solve a linear program with a variables and b constraints. All these results are improvements on previously known running times except for the last. The algorithm for kNN is slower asymptotically than the naive algorithm, but is expected to be much faster in practice. The algorithm for 2CN is based on the use of separating line orientations as a means of characterizing the solution. The solution to 3CN also uses a separating line orientation characterization leading to a simple and robust ``carrousel' algorithm. The kCN algorithm uses the idea of disassembling the layout to the left. Finally, the kNN algorithm uses the concept of subdivision trees and linear programming. Received July 11, 1994; revised August 22, 1995, and February 26, 1996.     

一个增量式判定树学习算法INDUCE

ＩＮＤＵＣＥ算法采用自顶向下判定树归纳的学习方法,不仅具有健壮性好,效率高和正确率高等优点,还具有增量学习能力,可以动态修正概念描述的不足,该算法还运用了构造性归纳的思想,在学习过程中生成新的描述子,使概念描述空间搜索的效率得到提高。运行实例表明,ＩＮＤＵＣＥ具有很好的应用前景。     

Graphs,closability, and causality of linear time-invariant discrete-time systems

In this paper a linear, time-invariant, discrete-time system is considered by regarding it as an (unbounded) operator on 2(N), 2(Z), CN or CZ. The graphs of closed systems are characterized and properties such as causality and closability of systems are studied. It is shown that the closure of a causal, closable system on 2(N), CN and CZ is again causal, whereas this is in general not the case for systems on 2(Z). Necessary and sufficient conditions which guarantee that this property hold for systems on 2(Z) are given. Many of the results extend without change to the continuous-time case.