Integrating statistical and inductive learning methods for knowledge acquisition

Inductive learning is a method for automated knowledge acquisition. It converts a set of training data into a knowledge structure. In the process of knowledge induction, statistical techniques can play a major role in improving performance. In this paper, we investigate the competition and integration between the traditional statistical and the inductive learning methods. First, the competition between these two approaches is examined. Then, a general framework for integrating these two approaches is presented. This framework suggests three possible integrations: (1) statistical methods as preprocessors for inductive learning, (2) inductive learning methods as preprocessors for statistical classification, and (3) the combination of the two methods to develop new algorithms. Finally, empirical evidence concerning these three possible integrations are discussed. The general conclusion is that algorithms integrating statistical and inductive learning concepts are likely to make the most improvement in performance.     

Cohesion methods in inductive learning

According to Webster, cohesion is "the act or process of sticking together tightly." Here the term represents the underlying forces that drive the formation of classes during inductive learning. This paper considers several numerical and conceptual induction algorithms, and compares their methods of cohesion. While these algorithms represent several different methods, they also exhibit some significant commonalities.     

Knowledge acquisition from batch semiconductor manufacturing data

Semiconductor manufacturing data consist of the processes and the machines involved in the production of batches of semiconductor circuit wafers. Wafer quality depends on the manufacturing line status and it is measured at the end of the line. We have developed a knowledge discovery system that is intended to help the yield analysis expert by learning the tentative causes of low quality wafers from an exhaustive amount of manufacturing data. The yield analysis expert, by using the knowledge discovered, will decide on which corrective actions to perform on the manufacturing process. This paper discusses the transformations carried out within the data from raw data to discovered knowledge, and also the two main tasks performed by the system. The features of the inductive algorithm performing those tasks are also described. Yield analysis experts at Lucent Technologies, Bell Labs Innovations in Spain are currently using this knowledge discovery application.     

Knowledge acquisition by incremental learning from problem-solution pairs

This paper describes LEW (learning by watching), an implementation of a novel learning technique, and discusses its application to the learning of plans. LEW is a domain-independent learning system with user-limited autonomy that is designed to provide robust performance in realistic knowledge acquisition tasks in a variety of domains. It partly automates the knowledge acquisition process for different knowledge types, such as concepts, rules, and plans. The inputs to the system, which we call cues , consist of an environmental component and of pairs containing a problem and its solution. Unlike traditional forms of "learning from examples", in which the system uses the teacher's answer to improve the result of a prior generalization of an example, LEW treats the problem-solution or question-answer instances, i. e., the cues themselves, as the basic units for generalization.     

Improving inductive logic programming by using simulated annealing

In Inductive Logic Programming (ILP), algorithms that are purely of the bottom-up or top-down type encounter several problems in practice. Since a majority of them are greedy ones, these algorithms stop when finding clauses in local optima, according to the “quality” measure used for evaluating the results. Moreover, when learning clauses one by one, the induced clauses become less and less interesting as the algorithm is progressing to cover few remaining examples. In this paper, we propose a simulated annealing framework to overcome these problems. Using a refinement operator, we define neighborhood relations on clauses and on hypotheses (i.e. sets of clauses). With these relations and appropriate quality measures, we show how to induce clauses (in a coverage approach), or to induce hypotheses directly by using simulated annealing algorithms. We discuss the necessary conditions on the refinement operators and the evaluation measures to increase the effectiveness of the algorithm. Implementations (included a parallelized version of the algorithm) are described and experimentation results in terms of convergence of the method and in terms of accuracy are presented.     

A new algorithm for automatic knowledge acquisition in inductive learning

The objective of this study is to present a new algorithm, REX-1, developed for automatic knowledge acquisition in Inductive Learning. It aims at eliminating the pitfalls and disadvantages of the techniques and algorithms currently in use. The proposed algorithm makes use of the direct rule extraction approach, rather than the decision tree. For this purpose, it uses a set of examples to induce general rules. Using some widely used set of examples such as IRIS, Balance and Balloons, Monk, Splice, Promoter, Lenses, Zoo, and Vote, our algorithm is compared with other well-known algorithms such as ID3, C4.5, ILA, and Rules Family.     

Knowledge acquisition using hypertext

This paper addresses the issue of supporting knowledge acquisition using hypertext. We propose a way of tightly integrating hypertext and structured object representation, using Artificial Intelligence (AI) frames for the basic representation of hypertext nodes. Epistemologically, a dual view of the resulting space is of interest. One view is that of hypertext which emphasizes nodes containg g text, including formal knowledge representation. The other view focuses on objects with certain relationships, which define a semantic network. Both in hypertext and in semantic networks the relations between chunks of knowledge are explicitly represented by links. However, in today's hypertext systems a node typically contains just informal text and references to other nodes. Our approach additionally facilitates the explicit representation of structure “inside” hypertext nodes using partitions. We show the usefulness of such a tight integration for knowledge acquisition, providing several features useful for supporting it based on a level of basic hypertext functionality. In particular, we sketch a method for doing knowledge acquisition in such an environment. Hypertext is used as a mediating “semiformal” representation, which allows experts to directly represent knowledge without the immediate support of knowledge engineers. These help then to make this knowledge operational, supported by the system's facility to provide templates as well as their links to the semiformal representation. As an example of our results of using this method of knowledge acquisition, we illustrate the strategic knowledge in our application domain. More generally, our approach supports important aspects of (software) engineering knowledge-based systems and their maintenance. Also their user interface can be improved this way.     

Class-dependent discretization for inductive learning fromcontinuous and mixed-mode data

Inductive learning systems can be effectively used to acquire classification knowledge from examples. Many existing symbolic learning algorithms can be applied in domains with continuous attributes when integrated with a discretization algorithm to transform the continuous attributes into ordered discrete ones. In this paper, a new information theoretic discretization method optimized for supervised learning is proposed and described. This approach seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application. The method has been tested in a number of inductive learning examples to show that the class-dependent discretizer can significantly improve the classification performance of many existing learning algorithms in domains containing numeric attributes     

Intelligent condition monitoring using fuzzy inductive learning

Extensive research has been performed for developing knowledge based intelligent monitoring systems for improving the reliability of manufacturing processes. Due to the high expense of obtaining knowledge from human experts, it is expected to develop new techniques to obtain the knowledge automatically from the collected data using data mining techniques. Inductive learning has become one of the widely used data mining methods for generating decision rules from data. In order to deal with the noise or uncertainties existing in the data collected in industrial processes and systems, this paper presents a new method using fuzzy logic techniques to improve the performance of the classical inductive learning approach. The proposed approach, in contrast to classical inductive learning method using hard cut point to discretize the continuous-valued attributes, uses soft discretization to enable the systems have less sensitivity to the uncertainties and noise. The effectiveness of the proposed approach has been illustrated in an application of monitoring the machining conditions in uncertain environment. Experimental results show that this new fuzzy inductive learning method gives improved accuracy compared with using classical inductive learning techniques.     

Comparative study of model updating methods using simulated experimental data

The inverse eigensensitivity method and the response function method of analytical model updating have become relatively more popular among other methods and have been successfully applied to the practice of analytical model improvement. This paper gives a detailed comparison of these two approaches of model updating on the basis of computer simulated experimental data with the objective of studying the convergence of the two methods and the accuracy with which they predict the corrections required in a finite element model. The effect of the amount of experimental data used in the process of model improvement on the quality of an updated model is also studied. The test cases of complete, incomplete and noisy experimental data are considered. The updated models are compared on the basis of some error indices constructed to quantify error in the predicted natural frequencies, mode shapes and response functions.     

Physics-guided machine learning from simulated data with different physical parameters

Knowledge and Information Systems - Physics-based models are widely used to study dynamical systems in a variety of scientific and engineering problems. However, these models are necessarily...     

Prototyping structural description using an inductive learning program

Character recognition systems can contribute tremendously to the advancement of the automation process and can improve the interaction between man and machine in many applications, including office automation, cheque verification and a large variety of banking, business and data entry applications. The main theme of this paper is the automatic recognition of hand‐printed Arabic characters using machine learning. Conventional methods have relied on hand‐constructed dictionaries which are tedious to construct and difficult to make tolerant to variation in writing styles. The advantages of machine learning are that it can generalize over the large degree of variation between writing styles and recognition rules can be constructed by example. The system was tested on a sample of handwritten characters from several individuals whose writing ranged from acceptable to poor in quality and the correct average recognition rate obtained using cross‐validation was 89.65%. © 2000 John Wiley & Sons, Inc.     

Autonomous generation of reflexion-based robot controller using inductive learning

The paper proposes a novel architecture for autonomously generating and managing a robot control system, aiming for the application to planetary rovers which will move in a partially unknown, unstructured environment. The proposed architecture is similar to the well known subsumption architecture in that the movements are governed by a network of various reflexion patterns. The major departures are that firstly it utilizes inductive learning to automatically generate and modify a control architecture, which is, if human is to do, quite a difficult and time consuming task, secondly it employs the concept of “goal sensor” to deal with the system goal more explicitly, and thirdly it compiles the planning results into a reflexion network and decision trees to maintain the strong features of reflexion based planner such as real-timeness, robustness and extensibility. The architecture has been applied to movement control of a certain rover in computer simulations and simple experiments, in which its effectiveness and characteristics have been cleared.     

Combining models from neural networks and inductive learning algorithms

The knowledge-based artificial neural network (KBANN) is composed of phases involving the expression of domain knowledge, the abstraction of domain knowledge at neural networks, the training of neural networks, and finally, the extraction of rules from trained neural networks. The KBANN attempts to open up the neural network black box and generates symbolic rules with (approximately) the same predictive power as the neural network itself. An advantage of using KBANN is that the neural network considers the contribution of the inputs towards classification as a group, while rule-based algorithms like C5.0 measure the individual contribution of the inputs one at a time as the tree is grown. The knowledge consolidation model (KCM) combines the rules extracted using KBANN (NeuroRule), frequency matrix (which is similar to the Naïve Bayesian technique), and C5.0 algorithm. The KCM can effectively integrate multiple rule sets into one centralized knowledge base. The cumulative rules from other single models can improve overall performance as it can reduce error-term and increase R-square. The key idea in the KCM is to combine a number of classifiers such that the resulting combined system achieves higher classification accuracy and efficiency than the original single classifiers. The aim of KCM is to design a composite system that outperforms any individual classifier by pooling together the decisions of all classifiers. Another advantage of KCM is that it does not need the memory space to store the dataset as only extracted knowledge is necessary in build this integrated model. It can also reduce the costs from storage allocation, memory, and time schedule. In order to verify the feasibility and effectiveness of KCM, personal credit rating dataset provided by a local bank in Seoul, Republic of Korea is used in this study. The results from the tests show that the performance of KCM is superior to that of the other single models such as multiple discriminant analysis, logistic regression, frequency matrix, neural networks, decision trees, and NeuroRule. Moreover, our model is superior to a previous algorithm for the extraction of rules from general neural networks.     

Knowledge discovery from process operational data using PCA and fuzzy clustering

An industrial case study is presented which uses principal component analysis and fuzzy c-means clustering to identify operational spaces and develop operational strategies for manufacturing desired products. Analysis of 303 data cases collected from a refinery fluid catalytic cracking process revealed that the data can be projected to four operational zones in the reduced two-dimensional plane. Three zones were found to correspond to three different product grades and the fourth is a zone corresponding to product changeover. Variable contribution analysis was also carried out to identify the most important variables that are responsible for the observed operational spaces and consequently strategies were developed for monitoring and operating the process in order to be able to move the operation from producing one product grade to another, with minimum time delays.     

Monitoring urban sprawl using simulated PROBA-V data

Urbanization proceeds currently at a rapid pace and the impact on natural ecosystems cannot be neglected. Consequently, it is important to be able to monitor the expansion of urban areas. Yet the process of extracting them from satellite imagery is not trivial. Urban is a non-uniform class with spectral proximity to barren land. In this article, a method for extracting urban areas from medium-resolution Earth observation data is presented. The information source is simulated data of the PROBA-V sensor. Visual and near-infrared bands are classified by the adaptive neuro-fuzzy inference system (ANFIS) neuro-fuzzy classifier into urban and non-urban classes. The method can overcome the main difficulty in similar efforts, i.e. the extensive commission errors of barren to the class urban. The main novelty relies on exploiting annual spectral variability of each land-use class at the pixel level. The basic assumption is that urban and barren areas may have similar spectral values but they have different phenological cycles. The overall accuracy obtained by the classification is 91.57% with a Cohen’s kappa coefficient (khat) of 0.84. Sufficient correlation at the city level is also achieved. Change detection is also possible in terms of hot-spot identification, however marginally suitable for medium-sized cities.     

Knowledge acquisition of conjunctive rules using multilayered neural networks

A major bottleneck in developing knowledge-based systems is the acquisition of knowledge. Machine learning is an area concerned with the automation of this process of knowledge acquisition. Neural networks generally represent their knowledge at the lower level, while knowledge-based systems use higher-level knowledge representations. the method we propose here provides a technique that automatically allows us to extract conjunctive rules from the lower-level representation used by neural networks, the strength of neural networks in dealing with noise has enabled us to produce correct rules in a noisy domain. Thus we propose a method that uses neural networks as the basis for the automation of knowledge acquisition and can be applied to noisy, realworld domains. © 1993 John Wiley & Sons, Inc.     

Transductive multi-label learning from missing data using smoothed rank function

Pattern Analysis and Applications - In this paper, we propose two new algorithms for transductive multi-label learning from missing data. In transductive matrix completion (MC), the challenge is...