Discrete probability estimation for classification usingcertainty-factor-based neural networks

Traditional probability estimation often demands a large amount of data for a problem of industrial scale. Neural networks have been used as an effective alternative for estimating input-output probabilities. In this paper, the certainty-factor-based neural network (CFNet) is explored for probability estimation in discrete domains. A new analysis presented here shows that the basis functions learned by the CFNet can bear precise semantics for dependencies. In the simulation study, the CFNet outperforms both the backpropagation network and the system based on the Rademacher-Walsh expansion. In the real-data experiments on splice junction and breast cancer data sets, the CFNet outperforms other neural networks and symbolic systems.     

The application of certainty factors to neural computing for rulediscovery

Discovery of domain principles has been a major long-term goal for scientists. The paper presents a system called DOMRUL for learning such principles in the form of rules. A distinctive feature of the system is the integration of the certainty factor (CF) model and a neural network. These two elements complement each other. The CF model offers the neural network better semantics and generalization advantage, and the neural network overcomes possible limitations such as inaccuracies and overcounting of evidence associated with certainty factors. It is a major contribution of the paper to show mathematically the quantizability nature of the CFNet since previously the quantizability of the CF model was demonstrated only empirically. The rule discovery system can be applied to any domain without restriction on both the rule number and rule size. In a hypothetical domain, DOMRUL discovered complex domain rules at a considerably higher accuracy than a commonly used rule-learning program C4.5 in both normal and noisy conditions. The scalability in a large domain is also shown. On a real data set concerning promoters prediction in molecular biology, DOMRUL learned rules with more complete semantics than C4.5.     

Learning Coordination Strategies for Cooperative Multiagent Systems

A central issue in the design of cooperative multiagent systems is how to coordinate the behavior of the agents to meet the goals of the designer. Traditionally, this had been accomplished by hand-coding the coordination strategies. However, this task is complex due to the interactions that can take place among agents. Recent work in the area has focused on how strategies can be learned. Yet, many of these systems suffer from convergence, complexity and performance problems. This paper presents a new approach for learning multiagent coordination strategies that addresses these issues. The effectiveness of the technique is demonstrated using a synthetic domain and the predator and prey pursuit problem.     

Reinforcement learning of iterative behaviour with multiple sensors

Reinforcement learning allows an agent to be both reactive and adaptive, but it requires a simple yet consistent representation of the task environment. In robotics this representation is the product of perception. Perception is a powerful simplifying mechanism because it ignores much of the complexity of the world by mapping multiple world states to each of a few representational states. The constraint of consistency conflicts with simplicity, however. A consistent representation distinguishes world states that have distinct utilities, but perception systems with sufficient acuity to do this tend to also make many unnecessary distinctions.In this paper we discuss reinforcement learning and the problem of appropriate perception. We then investigate a method for dealing with the problem, called theLion algorithm [1], and show that it can be used to reduce complexity by decomposing perception. The Lion algorithm does not allow iterative rules to be learned, and we describe modifications that overcome this limitation. We present experimental results that demonstrate their effectiveness in further reducing complexity. Finally, we mention some related research, and conclude with suggestions for further work.     

Inductive rule generation and interpretation

Abstract

The problem of knowledge acquisition has been recognized as the major bottleneck in the development of knowledge-based systems. An encouraging approach to alleviate this problem is inductive learning. Inductive learning systems accept, as input, a set of data that represent instances of the problem domain and produce, as output, the rules of the knowledge base. Each data item is described by a set of attribute values and is assigned to a unique decision class. A common characteristic of the existing inductive learning systems, is that they are empirical in nature and do not take into account the implications of the inductive rule generation process on the performance of the resulting set of rules. That performance is assessed when the rules are used to classify new unlabelled data. This paper demonstrates that the performance of a rule set is a function of the rule generation and rule interpretation processes. These two processes are interrelated and should not be considered separately. The interrelation of rule generation and rule interpretation is analysed and suggestions to improve the performance of existing inductive learning systems, are forwarded.     

LEARNING TO IMPROVE PLAN QUALITY

Adaptive automated planning systems that can, over time, improve the quality of plans they produce are a promising prospect. The first part of the article discusses the issues involved in designing quality improving learning for planning systems and reviews recent work on learning to improve plan quality. The second part describes our work on the Performance Improving Planning (PIP) System. The heart of PIP is an analytic technique that compares two planning episodes for solving a planning problem that led to two different quality solutions—a higher-quality solution and a lower quality solution—and identifies the critical differences that were responsible for the resulting differences in the quality of the completed plans. We compare the effectiveness of two different ways of storing and applying the knowledge learned from this analysis—as search-control rules and as rewrite rules. The results show that the search-control rules are more effective in improving plan quality. Further analysis of PIP-search-control—the version of PIP that stores the learned knowledge as search-control rules—shows that it is an effective technique for improving plan quality in a variety of situations.     

Probably approximately correct learning in fuzzy classificationsystems

An efficient method for learning (trapezoidal) membership functions for fuzzy predicates is presented. Positive and negative examples of one class are given together with a system of classification rules. The learned membership functions can be used for the fuzzy predicates occurring in the given rules to classify further examples. We show that the obtained classification is approximately correct with high probability. This justifies the obtained fuzzy sets within one particular classification problem, instead of relying on a subjective meaning of fuzzy predicates as normally done by a domain expert     

Learning from natural instructions

Machine learning is traditionally formalized and investigated as the study of learning concepts and decision functions from labeled examples, requiring a representation that encodes information about the domain of the decision function to be learned. We are interested in providing a way for a human teacher to interact with an automated learner using natural instructions, thus allowing the teacher to communicate the relevant domain expertise to the learner without necessarily knowing anything about the internal representations used in the learning process. In this paper we suggest to view the process of learning a decision function as a natural language lesson interpretation problem, as opposed to learning from labeled examples. This view of machine learning is motivated by human learning processes, in which the learner is given a lesson describing the target concept directly and a few instances exemplifying it. We introduce a learning algorithm for the lesson interpretation problem that receives feedback from its performance on the final task, while learning jointly (1) how to interpret the lesson and (2) how to use this interpretation to do well on the final task. traditional machine learning by focusing on supplying the learner only with information that can be provided by a task expert. We evaluate our approach by applying it to the rules of the solitaire card game. We show that our learning approach can eventually use natural language instructions to learn the target concept and play the game legally. Furthermore, we show that the learned semantic interpreter also generalizes to previously unseen instructions.     

APACS: a system for the automatic analysis and classification of conceptual patterns

Many existing inductive learning systems have been developed under the assumption that the learning tasks are performed in a noise-free environment. To cope with most real-world problems, it is important that a learning system be equipped with the capability to handle uncertainty. In this paper, we first identify the various sources of uncertainty that may be encountered in a noisy problem domain. Next, we present a method for the efficient acquisition of classification rules from training instances which may contain inconsistent, incorrect, or missing information. This algorithm consists of three phases: ( i ) the detection of inherent patterns in a set of noisy training data; ( ii ) the construction of classification rules based on these patterns; and ( iii ) the use of these rules to predict the class membership of an object. The method has been implemented in a system known as APACS (automatic pattern analysis and classification system). This system has been tested using both real-life and simulated data, and its performance is found to be superior to many existing systems in terms of efficiency and classification accuracy. Being able to handle uncertainty in the learning process, the proposed algorithm can be employed for applications in real-world problem domains involving noisy data.     

Layered approach to learning client behaviors in the robocup soccer server

In the past few years, multiagent systems (MAS) have emerged as an active subfield of artificial intelligence (AI). Because of the inherent complexity of MAS, there is much interest in using machine learning (ML) techniques to help build multiagent systems. Robotic soccer is a particularly good domain for studying MAS and multiagent learning. Our approach to using ML as a tool for building Soccer Server clients involves layering increasingly complex learned behaviors. In this article, we describe two levels of learned behaviors. First, the clients learn a low-level individual skill that allows them to control the ball effectively. Then, using this learned skill, they learn a higher level skill that involves multiple players. For both skills, we describe the learning method in detail and report on our extensive empirical testing. We also verify empirically that the learned skills are applicable to game situations.     

Coupled principal component analysis

A framework for a class of coupled principal component learning rules is presented. In coupled rules, eigenvectors and eigenvalues of a covariance matrix are simultaneously estimated in coupled equations. Coupled rules can mitigate the stability-speed problem affecting noncoupled learning rules, since the convergence speed in all eigendirections of the Jacobian becomes widely independent of the eigenvalues of the covariance matrix. A number of coupled learning rule systems for principal component analysis, two of them new, is derived by applying Newton's method to an information criterion. The relations to other systems of this class, the adaptive learning algorithm (ALA), the robust recursive least squares algorithm (RRLSA), and a rule with explicit renormalization of the weight vector length, are established.     

一类不确定运动系统的空间迭代学习控制

本文讨论了一类在有限空间区间内重复运行的不确定运动系统的跟踪控制问题.通过引入空间状态微分算子和空间复合能量函数,提出了一种空间周期的自适应迭代学习控制算法.首先利用空间状态微分算子,将系统从时间域转化到空间域形式.然后基于空间复合能量函数设计了控制器,利用含限幅作用的参数自适应律逼近系统中的不确定性,同时引入鲁棒项共同抑制非参数不确定性的影响.通过严格的数学分析,证明了在标准初始条件和随机有界初始误差两种情况下的跟踪误差收敛性.最后通过列车仿真进一步验证了该算法的有效性.     

AGFS: Adaptive Genetic Fuzzy System for medical data classification

A Genetic Fuzzy System (GFS) is basically a fuzzy system augmented by a learning process based on a genetic algorithm (GA). Fuzzy systems have demonstrated their ability to solve different kinds of problems in various application domains. Currently, there is an increasing interest to augment fuzzy systems with learning and adaptation capabilities. Two of the most successful approaches to hybridize fuzzy systems with learning and adaptation methods have been made in the realm of soft computing. The GA can be merged with Fuzzy system for different purposes like rule selection, membership function optimization, rule generation, co-efficient optimization, for data classification. Here we propose an Adaptive Genetic Fuzzy System (AGFS) for optimizing rules and membership functions for medical data classification process. The primary intension of the research is 1) Generating rules from data as well as for the optimized rules selection, adapting of genetic algorithm is done and to explain the exploration problem in genetic algorithm, introduction of new operator, called systematic addition is done, 2) Proposing a simple technique for scheming of membership function and Discretization, and 3) Designing a fitness function by allowing the frequency of occurrence of the rules in the training data. Finally, to establish the efficiency of the proposed classifier the presentation of the anticipated genetic-fuzzy classifier is evaluated with quantitative, qualitative and comparative analysis. From the outcome, AGFS obtained better accuracy when compared to the existing systems.     

Learning fuzzy rules for similarity assessment in case-based reasoning

Fundamental to case-based reasoning is the assumption that similar problems have similar solutions. The meaning of the concept of “similarity” can vary in different situations and remains an issue. This paper proposes a novel similarity model consisting of fuzzy rules to represent the semantics and evaluation criteria for similarity. We believe that fuzzy if-then rules present a more powerful and flexible means to capture domain knowledge for utility oriented similarity modeling than traditional similarity measures based on feature weighting. Fuzzy rule-based reasoning is utilized as a case matching mechanism to determine whether and to which extent a known case in the case library is similar to a given problem in query. Further, we explain that such fuzzy rules for similarity assessment can be learned from the case library using genetic algorithms. The key to this is pair-wise comparisons of cases with known solutions in the case library such that sufficient training samples can be derived for genetic-based fuzzy rule learning. The evaluations conducted have shown the superiority of the proposed method in similarity modeling over traditional schemes as well as the feasibility of learning fuzzy similarity rules from a rather small case base while still yielding competent system performance.     

A low-complexity fuzzy activation function for artificial neural networks

A novel fuzzy-based activation function for artificial neural networks is proposed. This approach provides easy hardware implementation and straightforward interpretability in the basis of IF-THEN rules. Backpropagation learning with the new activation function also has low computational complexity. Several application examples ( XOR gate, chaotic time-series prediction, channel equalization, and independent component analysis) support the potential of the proposed scheme.     

Hebbian and error-correction learning for complex-valued neurons

In this paper, we observe some important aspects of Hebbian and error-correction learning rules for complex-valued neurons. These learning rules, which were previously considered for the multi-valued neuron (MVN) whose inputs and output are located on the unit circle, are generalized for a complex-valued neuron whose inputs and output are arbitrary complex numbers. The Hebbian learning rule is also considered for the MVN with a periodic activation function. It is experimentally shown that Hebbian weights, even if they still cannot implement an input/output mapping to be learned, are better starting weights for the error-correction learning, which converges faster starting from the Hebbian weights rather than from the random ones.     

非线性函数的自适应分区多神经网络学习及仿真实验

根据BP神经网络学习非线性函数的精度与所学函数的区间大小及变化率等有关，提出了一种非线性函数的自适应分区多神经网络学习方法，这种方法根据学习精度的要求，自适应地把所学函数分成若干区间，分别用一个BP神经网络去学习，从而使学习精度大大提高，最后，给出了学习一维函数和多维函数的仿真实例，其结果表明分区学习的精度可提高10倍以上。