Integrating statistical and inductive learning methods for knowledge acquisition

Inductive learning is a method for automated knowledge acquisition. It converts a set of training data into a knowledge structure. In the process of knowledge induction, statistical techniques can play a major role in improving performance. In this paper, we investigate the competition and integration between the traditional statistical and the inductive learning methods. First, the competition between these two approaches is examined. Then, a general framework for integrating these two approaches is presented. This framework suggests three possible integrations: (1) statistical methods as preprocessors for inductive learning, (2) inductive learning methods as preprocessors for statistical classification, and (3) the combination of the two methods to develop new algorithms. Finally, empirical evidence concerning these three possible integrations are discussed. The general conclusion is that algorithms integrating statistical and inductive learning concepts are likely to make the most improvement in performance.     

Cohesion methods in inductive learning

According to Webster, cohesion is "the act or process of sticking together tightly." Here the term represents the underlying forces that drive the formation of classes during inductive learning. This paper considers several numerical and conceptual induction algorithms, and compares their methods of cohesion. While these algorithms represent several different methods, they also exhibit some significant commonalities.     

Knowledge acquisition by incremental learning from problem-solution pairs

This paper describes LEW (learning by watching), an implementation of a novel learning technique, and discusses its application to the learning of plans. LEW is a domain-independent learning system with user-limited autonomy that is designed to provide robust performance in realistic knowledge acquisition tasks in a variety of domains. It partly automates the knowledge acquisition process for different knowledge types, such as concepts, rules, and plans. The inputs to the system, which we call cues , consist of an environmental component and of pairs containing a problem and its solution. Unlike traditional forms of "learning from examples", in which the system uses the teacher's answer to improve the result of a prior generalization of an example, LEW treats the problem-solution or question-answer instances, i. e., the cues themselves, as the basic units for generalization.     

Improving inductive logic programming by using simulated annealing

In Inductive Logic Programming (ILP), algorithms that are purely of the bottom-up or top-down type encounter several problems in practice. Since a majority of them are greedy ones, these algorithms stop when finding clauses in local optima, according to the “quality” measure used for evaluating the results. Moreover, when learning clauses one by one, the induced clauses become less and less interesting as the algorithm is progressing to cover few remaining examples. In this paper, we propose a simulated annealing framework to overcome these problems. Using a refinement operator, we define neighborhood relations on clauses and on hypotheses (i.e. sets of clauses). With these relations and appropriate quality measures, we show how to induce clauses (in a coverage approach), or to induce hypotheses directly by using simulated annealing algorithms. We discuss the necessary conditions on the refinement operators and the evaluation measures to increase the effectiveness of the algorithm. Implementations (included a parallelized version of the algorithm) are described and experimentation results in terms of convergence of the method and in terms of accuracy are presented.     

Class-dependent discretization for inductive learning fromcontinuous and mixed-mode data

Inductive learning systems can be effectively used to acquire classification knowledge from examples. Many existing symbolic learning algorithms can be applied in domains with continuous attributes when integrated with a discretization algorithm to transform the continuous attributes into ordered discrete ones. In this paper, a new information theoretic discretization method optimized for supervised learning is proposed and described. This approach seeks to maximize the mutual dependence as measured by the interdependence redundancy between the discrete intervals and the class labels, and can automatically determine the most preferred number of intervals for an inductive learning application. The method has been tested in a number of inductive learning examples to show that the class-dependent discretizer can significantly improve the classification performance of many existing learning algorithms in domains containing numeric attributes     

Knowledge acquisition using hypertext

This paper addresses the issue of supporting knowledge acquisition using hypertext. We propose a way of tightly integrating hypertext and structured object representation, using Artificial Intelligence (AI) frames for the basic representation of hypertext nodes. Epistemologically, a dual view of the resulting space is of interest. One view is that of hypertext which emphasizes nodes containg g text, including formal knowledge representation. The other view focuses on objects with certain relationships, which define a semantic network. Both in hypertext and in semantic networks the relations between chunks of knowledge are explicitly represented by links. However, in today's hypertext systems a node typically contains just informal text and references to other nodes. Our approach additionally facilitates the explicit representation of structure “inside” hypertext nodes using partitions. We show the usefulness of such a tight integration for knowledge acquisition, providing several features useful for supporting it based on a level of basic hypertext functionality. In particular, we sketch a method for doing knowledge acquisition in such an environment. Hypertext is used as a mediating “semiformal” representation, which allows experts to directly represent knowledge without the immediate support of knowledge engineers. These help then to make this knowledge operational, supported by the system's facility to provide templates as well as their links to the semiformal representation. As an example of our results of using this method of knowledge acquisition, we illustrate the strategic knowledge in our application domain. More generally, our approach supports important aspects of (software) engineering knowledge-based systems and their maintenance. Also their user interface can be improved this way.     

Intelligent condition monitoring using fuzzy inductive learning

Extensive research has been performed for developing knowledge based intelligent monitoring systems for improving the reliability of manufacturing processes. Due to the high expense of obtaining knowledge from human experts, it is expected to develop new techniques to obtain the knowledge automatically from the collected data using data mining techniques. Inductive learning has become one of the widely used data mining methods for generating decision rules from data. In order to deal with the noise or uncertainties existing in the data collected in industrial processes and systems, this paper presents a new method using fuzzy logic techniques to improve the performance of the classical inductive learning approach. The proposed approach, in contrast to classical inductive learning method using hard cut point to discretize the continuous-valued attributes, uses soft discretization to enable the systems have less sensitivity to the uncertainties and noise. The effectiveness of the proposed approach has been illustrated in an application of monitoring the machining conditions in uncertain environment. Experimental results show that this new fuzzy inductive learning method gives improved accuracy compared with using classical inductive learning techniques.     

Autonomous generation of reflexion-based robot controller using inductive learning

The paper proposes a novel architecture for autonomously generating and managing a robot control system, aiming for the application to planetary rovers which will move in a partially unknown, unstructured environment. The proposed architecture is similar to the well known subsumption architecture in that the movements are governed by a network of various reflexion patterns. The major departures are that firstly it utilizes inductive learning to automatically generate and modify a control architecture, which is, if human is to do, quite a difficult and time consuming task, secondly it employs the concept of “goal sensor” to deal with the system goal more explicitly, and thirdly it compiles the planning results into a reflexion network and decision trees to maintain the strong features of reflexion based planner such as real-timeness, robustness and extensibility. The architecture has been applied to movement control of a certain rover in computer simulations and simple experiments, in which its effectiveness and characteristics have been cleared.     

Combining models from neural networks and inductive learning algorithms

The knowledge-based artificial neural network (KBANN) is composed of phases involving the expression of domain knowledge, the abstraction of domain knowledge at neural networks, the training of neural networks, and finally, the extraction of rules from trained neural networks. The KBANN attempts to open up the neural network black box and generates symbolic rules with (approximately) the same predictive power as the neural network itself. An advantage of using KBANN is that the neural network considers the contribution of the inputs towards classification as a group, while rule-based algorithms like C5.0 measure the individual contribution of the inputs one at a time as the tree is grown. The knowledge consolidation model (KCM) combines the rules extracted using KBANN (NeuroRule), frequency matrix (which is similar to the Naïve Bayesian technique), and C5.0 algorithm. The KCM can effectively integrate multiple rule sets into one centralized knowledge base. The cumulative rules from other single models can improve overall performance as it can reduce error-term and increase R-square. The key idea in the KCM is to combine a number of classifiers such that the resulting combined system achieves higher classification accuracy and efficiency than the original single classifiers. The aim of KCM is to design a composite system that outperforms any individual classifier by pooling together the decisions of all classifiers. Another advantage of KCM is that it does not need the memory space to store the dataset as only extracted knowledge is necessary in build this integrated model. It can also reduce the costs from storage allocation, memory, and time schedule. In order to verify the feasibility and effectiveness of KCM, personal credit rating dataset provided by a local bank in Seoul, Republic of Korea is used in this study. The results from the tests show that the performance of KCM is superior to that of the other single models such as multiple discriminant analysis, logistic regression, frequency matrix, neural networks, decision trees, and NeuroRule. Moreover, our model is superior to a previous algorithm for the extraction of rules from general neural networks.     

Transductive multi-label learning from missing data using smoothed rank function

Pattern Analysis and Applications - In this paper, we propose two new algorithms for transductive multi-label learning from missing data. In transductive matrix completion (MC), the challenge is...     

Knowledge acquisition of conjunctive rules using multilayered neural networks

A major bottleneck in developing knowledge-based systems is the acquisition of knowledge. Machine learning is an area concerned with the automation of this process of knowledge acquisition. Neural networks generally represent their knowledge at the lower level, while knowledge-based systems use higher-level knowledge representations. the method we propose here provides a technique that automatically allows us to extract conjunctive rules from the lower-level representation used by neural networks, the strength of neural networks in dealing with noise has enabled us to produce correct rules in a noisy domain. Thus we propose a method that uses neural networks as the basis for the automation of knowledge acquisition and can be applied to noisy, realworld domains. © 1993 John Wiley & Sons, Inc.     

Bayesian function learning using MCMC methods

The paper deals with the problem of reconstructing a continuous 1D function from discrete noisy samples. The measurements may also be indirect in the sense that the samples may be the output of a linear operator applied to the function. Bayesian estimation provides a unified treatment of this class of problems. We show that a rigorous Bayesian solution can be efficiently implemented by resorting to a Markov chain Monte Carlo (MCMC) simulation scheme. In particular, we discuss how the structure of the problem can be exploited in order to improve the computational and convergence performances. The effectiveness of the proposed scheme is demonstrated on two classical benchmark problems as well as on the analysis of IVGTT (IntraVenous glucose tolerance test) data, a complex identification-deconvolution problem concerning the estimation of the insulin secretion rate following the administration of an intravenous glucose injection     

Mining fuzzy association rules from questionnaire data

Association rule mining is one of most popular data analysis methods that can discover associations within data. Association rule mining algorithms have been applied to various datasets, due to their practical usefulness. Little attention has been paid, however, on how to apply the association mining techniques to analyze questionnaire data. Therefore, this paper first identifies the various data types that may appear in a questionnaire. Then, we introduce the questionnaire data mining problem and define the rule patterns that can be mined from questionnaire data. A unified approach is developed based on fuzzy techniques so that all different data types can be handled in a uniform manner. After that, an algorithm is developed to discover fuzzy association rules from the questionnaire dataset. Finally, we evaluate the performance of the proposed algorithm, and the results indicate that our method is capable of finding interesting association rules that would have never been found by previous mining algorithms.     

Investigating pilot performance using mixed-modality simulated data link

OBJECTIVE: General aviation (GA) pilot performance utilizing a mixed-modality simulated data link was objectively evaluated based on the time required in accessing, understanding, and executing data link commands. Additional subjective data were gathered on workload, situation awareness (SA), and preference. BACKGROUND: Research exploring mixed-modality data link integration to the single-pilot GA cockpit is lacking, especially with respect to potential effects on safety. METHODS: Sixteen visual flight rules (VFR)-rated pilots participated in an experiment using a flight simulator equipped with a mixed-modality data link. Data link modalities were text display, synthesized speech, digitized speech, and synthesized speech/text combination. Flight conditions included VFR (unlimited ceiling and visibility) or marginal VFR flight conditions (clouds 2,800 ft above ground level, 3-mile visibility). RESULTS: Statistically significant differences were found in pilot performance, mental workload, and SA across the data link modalities. Textual data link resulted in increased time and workload as compared with the three speech-type data link conditions, which did not differ. SA measures indicated higher performance with textual and digitized speech data link conditions. CONCLUSION: Textual data link can be significantly enhanced for single-pilot GA operations by the addition of a speech component. APPLICATION: Potential applications include operational safety in future GA systems that incorporate data link for use by a single pilot and guidance in the development of flight performance objectives for these systems.     

基于仿真数据的钻机动力头设计知识获取方法*

研究了如何从工程仿真获得的中间数据中提取设计知识的方法。对同一个设计问题,依据设计变量的不同取值构建正交实验表,对正交实验表中每一种实验方案的变量取值采用Pro/E建立设计问题的实体模型,将该实体模型导入ADAMS内并添加约束及驱动,从仿真后处理中导出设计问题的不同特征数据,并以该特征数据以及该问题的相关计算数据构成新的二维表,采用粗糙集理论对二维表数据进行知识抽取。以锚杆钻机动力头设计的仿真数据抽取知识加以验证,验证结果得出动力头设计的体积取中间值并且轴向推力取较大值时最终的冲击钻进能力才最强等实用性知     

面向大型数据库的审计数据采集方法

计算机辅助审计是目前审计领域研究的一个热点,审计数据采集是面向数据的计算机辅助审计的关键步骤。分析了常用的审计数据采集方法,比较了各自的优缺点。在此基础上,针对我国实施计算机辅助审计的现状以及面向大型数据库的审计数据采集的特点,分析了适合大型数据库的审计数据采集方法,并以Oracle数据库为例,分析了该方法的应用。     

Knowledge for courseware engineering: A framework for inductive knowledge acquisition based upon evaluation of adaptable courseware products

Authors or adaptors of courseware products preferably should receive support in the process of development and adaptation of courseware products. A predictive agent is defined as a system that is able to predict the expected effectiveness of various composable products from current product attributes. The described research addresses the questions of how to acquire the necessary knowledge for a predictive agent, how to organize this knowledge, and how to link it with methods and tools for courseware authoring and adaptation. We propose to use a methodology, derived from the field of machine learning, and present a framework for applying inductive knowledge acquisition based upon the empirical evaluation of adaptable courseware products.     

Modelling of diesel engine performance using advanced machine learning methods under scarce and exponential data set

Traditional methods on creating diesel engine models include the analytical methods like multi-zone models and the intelligent based models like artificial neural network (ANN) based models. However, those analytical models require excessive assumptions while those ANN models have many drawbacks such as the tendency to overfitting and the difficulties to determine the optimal network structure. In this paper, several emerging advanced machine learning techniques, including least squares support vector machine (LS-SVM), relevance vector machine (RVM), basic extreme learning machine (ELM) and kernel based ELM, are newly applied to the modelling of diesel engine performance. Experiments were carried out to collect sample data for model training and verification. Limited by the experiment conditions, only 24 sample data sets were acquired, resulting in data scarcity. Six-fold cross-validation is therefore adopted to address this issue. Some of the sample data are also found to suffer from the problem of data exponentiality, where the engine performance output grows up exponentially along the engine speed and engine torque. This seriously deteriorates the prediction accuracy. Thus, logarithmic transformation of dependent variables is utilized to pre-process the data. Besides, a hybrid of leave-one-out cross-validation and Bayesian inference is, for the first time, proposed for the selection of hyperparameters of kernel based ELM. A comparison among the advanced machine learning techniques, along with two traditional types of ANN models, namely back propagation neural network (BPNN) and radial basis function neural network (RBFNN), is conducted. The model evaluation is made based on the time complexity, space complexity, and prediction accuracy. The evaluation results show that kernel based ELM with the logarithmic transformation and hybrid inference is far better than basic ELM, LS-SVM, RVM, BPNN and RBFNN, in terms of prediction accuracy and training time.