Symbolic and Neural Learning Algorithms: An Experimental Comparison

Despite the fact that many symbolic and neural network (connectionist) learning algorithms address the same problem of learning from classified examples, very little is known regarding their comparative strengths and weaknesses. Experiments comparing the ID3 symbolic learning algorithm with the perception and backpropagation neural learning algorithms have been performed using five large, real-world data sets. Overall, backpropagation performs slightly better than the other two algorithms in terms of classification accuracy on new examples, but takes much longer to train. Experimental results suggest that backpropagation can work significantly better on data sets containing numerical data. Also analyzed empirically are the effects of (1) the amount of training data, (2) imperfect training examples, and (3) the encoding of the desired outputs. Backpropagation occasionally outperforms the other two systems when given relatively small amounts of training data. It is slightly more accurate than ID3 when examples are noisy or incompletely specified. Finally, backpropagation more effectively utilizes a distributed output encoding.     

Combining Symbolic Constraint Solvers on Algebraic Domains

In the context of constraint logic programming and theorem proving, the development of constraint solvers on algebraic domains and their combination is of prime interest. As an example, a constraint solver in finite algebras is presented for a constraint language including for instance equations, disequations and inequations. By extending techniques used for the combination of unification in disjoint equational theories, we show how to combine constraint solvers on different algebraic domains that may share some constant symbols. We illustrate this technique by combining the constraint solver in finite algebras with other unification algorithms, and with another constraint solver on a different finite algebra.     

Combining Symmetry Reduction and Under-Approximation for Symbolic Model Checking

This work presents a collection of methods that integrate symmetry reduction and under-approximation with symbolic model checking in order to reduce space and time. The main objective of these methods is falsification. However, under certain conditions, they can provide verification as well.We first present algorithms that use symmetry reduction to perform on-the-fly model checking for temporal safety properties. These algorithms avoid building the orbit relation and choose representatives on-the-fly while computing the reachable states. We then extend these algorithms to check liveness properties as well. In addition, we introduce an iterative on-the-fly algorithm that builds subsets of the orbit relation rather than the full relation.Our methods are fully automatic once the user supplies some basic information about the symmetry in the verified system. Moreover, the methods are robust and work correctly even if the information supplied by the user is incorrect. Furthermore, the methods return correct results even when the computation of the symmetry reduction has not been completed due to memory or time explosion.We implemented our methods within the IBM model checker Rule-Base and compared their performance to that of RuleBase. In most cases, our algorithms outperformed RuleBase in both time and space.     

Combining Committee-Based Semi-Supervised Learning and Active Learning

Many data mining applications have a large amount of data but labeling data is usually difficult, expensive, or time consuming, as it requires human experts for annotation. Semi-supervised learning addresses this problem by using unlabeled data together with labeled data in the training process. Co-Training is a popular semi-supervised learning algorithm that has the assumptions that each example is represented by multiple sets of features (views) and these views are sufficient for learning and independent given the class. However, these assumptions are strong and are not satisfied in many real-world domains. In this paper, a single-view variant of Co-Training, called Co-Training by Committee (CoBC) is proposed, in which an ensemble of diverse classifiers is used instead of redundant and independent views. We introduce a new labeling confidence measure for unlabeled examples based on estimating the local accuracy of the committee members on its neighborhood. Then we introduce two new learning algorithms, QBC-then-CoBC and QBC-with-CoBC, which combine the merits of committee-based semi-supervised learning and active learning. The random subspace method is applied on both C4.5 decision trees and 1-nearest neighbor classifiers to construct the diverse ensembles used for semi-supervised learning and active learning. Experiments show that these two combinations can outperform other non committee-based ones.     

人工神经网络的符号解释

提出了人工神经网络符号解释的基本过程,并详细阐述了该过程中的三个重要步骤：网络的构建、规则的提取以及规则的评估。     

Analytica – An Experiment in Combining Theorem Proving and Symbolic Computation

Analytica is an automatic theorem prover for theorems in elementary analysis. The prover is written in the Mathematica language and runs in the Mathematica environment. The goal of the project is to use a powerful symbolic computation system to prove theorems that are beyond the scope of previous automatic theorem provers. The theorem prover is also able to deduce the correctness of certain simplification steps that would otherwise not be performed. We describe the structure of Analytica and explain the main techniques that it uses to construct proofs. Analytica has been able to prove several nontrivial theorems. In this paper, we show how it can prove a series of lemmas that lead to the Bernstein approximation theorem.     

进化算法与符号执行结合的程序复杂度分析方法

程序的最坏执行路径是计算程序复杂度的一项重要指标,有助于发现系统可能存在的复杂性漏洞.近年来将符号执行应用于程序复杂度分析的研究取得了不小的进展,但现有方法存在通用性较差、分析时间较长的问题.文中提出一种面向最坏路径探测的进化算法——EvoWca,其核心思想是利用程序在较小输入规模下的已知最坏路径特征指导较大输入规模下初始路径集合的构建,然后模拟进化算法,对路径进行组合、突变和选择迭代,使得在搜索范围内探测到的最坏路径逼近于最坏时间复杂度对应的路径.基于该算法实现了一个用于程序复杂度分析的原型工具EvoWca2j,使用该工具和已有技术对一组Java程序进行最坏路径探索和执行效率评估,实验结果表明,相比现有方法,EvoWca2j的通用性和探索效率都有明显提高.     

Octopus: Combining Learning and Parallel Search

This paper presents Octopus, an automated theorem-proving system that combines learning and parallel search. The learning technique involves proving a simpler version of a given theorem and then using what it has learned to prove the given theorem. As of January 2004 Octopus had successfully proved 43 of the 1.0-rated theorems of the TPTP Problem Library.     

基于符号神经网络的数据溶合系统的研究

本文介绍一种把多个神经网络和符号推理结合在一起用于移动式机器人多传感器数据溶合的新技术。用该技术实现的系统不仅能够准确定位，而且可以增强机器人的应变能力。     

结合卷积神经网络与OpenCV的人脸表情识别

针对实时人脸表情识别模型训练慢、识别速度慢的问题,提出一种OpenCV和卷积神经网络结合进行实时表情识别的方法.人脸表情是多个局部区域特征的集合,而卷积神经网络提取出的特征能更多地关注局部,因此采取卷积神经网络的方式进行模型的训练.所提网络在全连接层中加入了Dropout,能有效预防过拟合现象的发生,并且提升模型泛化能...     

混合多元回归学习模型的改进

研究统计方法分析问题,针对在实际应用外特性模型的输入普遍为混合变量,既包括连续随机变量,也包括离散随机变量.目前已有混合多元回归学习模型大多只处理连续随机变量,且有着多重共线性的缺陷.针对上述问题,研究了基于贝叶斯网络的回归树学习模型.基于贝叶斯网络的回归树学习模型的研究方法建立在朴素贝叶斯网络模型基础上,采用分而治之的原则构造决策树,以朴素贝叶斯取代叶节点.在2个UCI机器学习数据集上的仿真实验结果表明模型性能良好.基于贝叶斯网络的回归树学习模型可以有效减小预测误差.     

融合柯西变异和反向学习的改进麻雀算法

针对基本麻雀搜索算法在迭代后期种群多样性减小,容易陷入局部极值的问题,提出一种融合柯西变异和反向学习的改进麻雀算法(ISSA).首先,采用一种映射折叠次数无限的Sin混沌初始化种群,为全局寻优奠定基础;其次,在发现者位置更新方式中引入上一代全局最优解,提高全局搜索的充分性,同时加入自适应权重,协调局部挖掘和全局探索的能...     

工学结合在“动态Web技术(JSP)”课程中的深度融合

本文介绍了工学结合模式在"动态Web技术(JSP)"课程中如何进行深度融合,根据在教学中的探索实践,提出了具体的实施方法,将工学结合的教学理念与高职学科教学特点紧密结合,将工学结合的内涵与外延具体化、学科化。     

结合特征选择与集成学习的密码体制识别方案

在密文识别过程中,加密算法是进一步分析密文的必要前提。然而现有密文识别方案存在形式单一的问题,并且在识别多种密码体制时难以应对不同密码体制间存在的差异。分析密文特征对识别效果的影响机制,结合Relief特征选择算法和异质集成学习算法,提出一种可适应多种密码体制识别情景的动态特征识别方案。在36种加密算法产生的密文数据集上进行实验,结果表明,与基于随机森林的密码体制分层识别方案相比,该方案在3类不同密码体制识别情景下的识别准确率分别提高了6.41%、10.03%和11.40%。     

Symbolic Regression Based Extreme Learning Machine Models for System Identification

Neural Processing Letters - Reproducible machine learning models with less number of parameters and fast optimization are preferred in embedded system design for the applications of artificial...     

A Weighted Nearest Neighbor Algorithm for Learning with Symbolic Features

In the past, nearest neighbor algorithms for learning from examples have worked best in domains in which all features had numeric values. In such domains, the examples can be treated as points and distance metrics can use standard definitions. In symbolic domains, a more sophisticated treatment of the feature space is required. We introduce a nearest neighbor algorithm for learning in domains with symbolic features. Our algorithm calculates distance tables that allow it to produce real-valued distances between instances, and attaches weights to the instances to further modify the structure of feature space. We show that this technique produces excellent classification accuracy on three problems that have been studied by machine learning researchers: predicting protein secondary structure, identifying DNA promoter sequences, and pronouncing English text. Direct experimental comparisons with the other learning algorithms show that our nearest neighbor algorithm is comparable or superior in all three domains. In addition, our algorithm has advantages in training speed, simplicity, and perspicuity. We conclude that experimental evidence favors the use and continued development of nearest neighbor algorithms for domains such as the ones studied here.     

Supervised Learning Probabilistic Neural Networks

This study proposed supervised learning probabilistic neural networks (SLPNN) which have three kinds of network parameters: variable weights representing the importance of input variables, the reciprocal of kernel radius representing the effective range of data, and data weights representing the data reliability. These three kinds of parameters can be adjusted through training. We tested three artificial functions as well as 15 benchmark problems, and compared it with multi-layered perceptron (MLP) and probabilistic neural networks (PNN). The results showed that SLPNN is slightly more accurate than MLP, and much more accurate than PNN. Besides, the data weights can find the noise data in data set, and the variable weights can measure the importance of input variables and have the greatest contribution to accuracy of model among the three kinds of network parameters.     

Neural Learning from Unbalanced Data

This paper describes the result of our study on neural learning to solve the classification problems in which data is unbalanced and noisy. We conducted the study on three different neural network architectures, multi-layered Back Propagation, Radial Basis Function, and Fuzzy ARTMAP using three different training methods, duplicating minority class examples, Snowball technique and multidimensional Gaussian modeling of data noise. Three major issues are addressed: neural learning from unbalanced data examples, neural learning from noisy data, and making intentional biased decisions. We argue that by properly generated extra training data examples around the noise densities, we can train a neural network that has a stronger capability of generalization and better control of the classification error of the trained neural network. In particular, we focus on problems that require a neural network to make favorable classification to a particular class such as classifying normal(pass)/abnormal(fail) vehicles in an assembly plant. In addition, we present three methods that quantitatively measure the noise level of a given data set. All experiments were conducted using data examples downloaded directly from test sites of an automobile assembly plant. The experimental results showed that the proposed multidimensional Gaussian noise modeling algorithm was very effective in generating extra data examples that can be used to train a neural network to make favorable decisions for the minority class and to have increased generalization capability.     

Privacy-Preserving Backpropagation Neural Network Learning

With the development of distributed computing environment , many learning problems now have to deal with distributed input data. To enhance cooperations in learning, it is important to address the privacy concern of each data holder by extending the privacy preservation notion to original learning algorithms. In this paper, we focus on preserving the privacy in an important learning model, multilayer neural networks. We present a privacy-preserving two-party distributed algorithm of backpropagation which allows a neural network to be trained without requiring either party to reveal her data to the other. We provide complete correctness and security analysis of our algorithms. The effectiveness of our algorithms is verified by experiments on various real world data sets.