Integrating Neural and Symbolic Approaches: A Symbolic Learning Scheme for a Connectionist Associative Memory

This paper deals with the integration of neural and symbolic approaches. It focuses on associative memories where a connectionist architecture tries to provide a storage and retrieval component for the symbolic level. In this light, the classic model for associative memory, the Hopfield network is briefly reviewed. Then, a new model for associative memory, the hybrid Hopfield-clique network is presented in detail. Its application to a typically symbolic task, the post -processing of the output of an optical character recognizer, is also described. In the author's view, the hybrid Hopfield -clique network constitutes an example of a successful integration of the two approaches. It uses a symbolic learning scheme to train a connectionist network, and through this integration, it can provide perfect storage and recall. As a conclusion, an analysis of what can be learned from this specific architecture is attempted. In the case of this model, a guarantee for perfect storage and recall can only be given because it was possible to analyze the problem using the well-defined symbolic formalism of graph theory. In general, we think that finding an adequate formalism for a given problem is an important step towards solving it.     

基于神经网络的加工过程模型辨识

介绍了神经网络动态建模方法,以车削加工过程为例,用一个带单隐层的反向传播网络对非线性的加工过程进行了辨识研究,并将神经网络模型的跟踪响应与参数模型的跟踪响应作了对比分析。仿真结果表明,神经网络是建立非线性加工过程模型的一种有效方法。     

Integration of Neural Heuristics into Knowledge-based Inference

The rule base and the inference engine of a knowledge-based system are transformed into a kind of neural network called a conceptualization network. An approach is presented that generalizes the backpropagation teaming rule of the neural-network approach such that it can effectively deal with errors in conceptualization networks, which are often multilayered and involve logic conjunction. The idea is to use hill-climbing search where the backpropagation rule falls short because the transfer function is not differentiable. When the generalized backpropagation rule is applied to a conceptualization network which has been constrained by initial correct knowledge, incorrect rules can be recognized. Experiments in a practical domain have demonstrated that the approach can satisfactorily conduct credit and blame assignment for rules which may involve intermediate concepts and logic conjunction. Effective removal of incorrect rules with significant improvement of the system performance has been observed.     

Connectionist learning: A comparison of neural networks and an optical thin-film multilayer model

Current work on connectionist models has been focused largely on artificial neural networks that are inspired by the networks of biological neurons in the human brain. However, there are also other connectionistarchitectures that differ significantly from this biological exemplar. We proposed a novel connectionist learning architecture inspired by the physics associated with optical coatings of multiple layers of thin-films in a previous paper (Li and Purvis 1999, Annals of Mathematics and Artificial Intelligence, 26: 1-4). The proposed model differs significantly from the widely used neuron-inspired models. With thin-film layer thicknesses serving as adjustable parameters (as compared with connection weights in a neural network) for the learning system, the optical thin-film multilayer model (OTFM) is capable of approximating virtually any kind of highly nonlinear mappings. The OTFM is not a physical implementation using optical devices. Instead, it is proposed as a new connectionist learning architecture with its distinct optical properties as compared with neural networks. In this paper we focus on a detailed comparison of neural networks and the OTFM (Li 2001, Proceedings ofINNS-IEEE International Joint Conference on Neural Networks, Washington, DC, pp. 1727-1732). We describe the architecture of the OTFM and show how it can be viewed as a connectionist learning model. We then present experimental results on solving a classification problem and a time series prediction problem that are typical of conventional connectionist architectures to demonstrate the OTFM's learning capability.     

A Connectionist Production System with Partial Match and its Use for Approximate Reasoning

The paper discusses a connectionist implementation of knowledge engineering concepts and concepts related to production systems in particular. Production systems are one of the most used artificial intelligence techniques as well as a widely explored model of cognition. The use of neural networks for building connectionist production systems opens the door for developing production systems with partial match and approximate reasoning. An architecture of a neural production system (NPS) and its third realization—NPS3, designed to facilitate approximate reasoning—are presented in the paper. NPS3 facilitates partial match between facts and rules, variable binding, different conflict resolution strategies and chain inference. Facts are represented in a working memory by so-called certainty degrees. Different inference control parameters are attached to every production rule. Some of them are known neuronal parameters, receiving an engineering meaning here. Others, which have their context in knowledge engineering, have been implemented in a connectionist way. The partial match implemented in NPS3 is demonstrated on the same test production system as used by other authors. The ability of NPS3 for approximate reasoning is illustrated by reasoning over a set of simple diagnostic productions and a set of decision support fuzzy rules.     

The foundations of connectionist computation

This paper presents a formal syntax and semantics for computation in neural networks. The main motivation for this is to provide a foundation for rigorous mathematical analysis of the capabilities of neural networks in relation to other types of computational system. A secondary benefit is that it helps to clarify obscurities and controversial issues in the notion of connectionist computation as it is informally understood. The paper reviews the various informal and formal definitions of connectionism in the literature and attempts to identify common principles and areas of disagreement. Central to connectionism is the idea of a system of simple nodes working together to solve a task, where each node acts in a purely local way on its neighbours: the vague words 'simple' and 'local' are clarified and defined by my formal system in a precise way, free from arbitrary restrictions. The system also defines the semantics of node growth, node pruning and connectivity change-operations that are used in an increasing number of recent connectionist algorithms but are not taken into account by previous definitions of connectionism.     

Adaptation of Learning Rule Parameters Using a Meta Neural Network

This paper proposes an application-independent method of automating learning rule parameter selection using a form of supervisor neural network (NN), known as a meta neural network (MNN), to alter the value of a learning rule parameter during training. The MNN is trained using data generated by observing the training of a NN and recording the effects of the selection of various parameter values. The MNN is then combined with a normal learning rule to augment its performance. Experiments are undertaken to see how this method performs by using it to adapt a global parameter of the resilient backpropagation and quickpropagation learning rules.     

Combining Prior Symbolic Knowledge and Constructive Neural Network Learning

The concepts of knowledge-based systems and machine learning are combined by integrating an expert system and a constructive neural networks learning algorithm. Two approaches are explored: embedding the expert system directly and converting the expert system rule base into a neural network. This initial system is then extended by constructively learning additional hidden units in a problem-specific manner. Experiments performed indicate that generalization of a combined system surpasses that of each system individually.     

A Recurrent Neural Network that Learns to Count

Parallel distributed processing (PDP) architectures demonstrate a potentially radical alternative to the traditional theories of language processing that are based on serial computational models. However, learning complex structural relationships in temporal data presents a serious challenge to PDP systems. For example, automata theory dictates that processing strings from a context-free language (CFL) requires a stack or counter memory device. While some PDP models have been hand-crafted to emulate such a device, it is not clear how a neural network might develop such a device when learning a CFL. This research employs standard backpropagation training techniques for a recurrent neural network (RNN) in the task of learning to predict the next character in a simple deterministic CFL (DCFL). We show that an RNN can learn to recognize the structure of a simple DCFL. We use dynamical systems theory to identify how network states reflect that structure by building counters in phase space. The work is an empirical investigation which is complementary to theoretical analyses of network capabilities, yet original in its specific configuration of dynamics involved. The application of dynamical systems theory helps us relate the simulation results to theoretical results, and the learning task enables us to highlight some issues for understanding dynamical systems that process language with counters.     

基于动态权重的自适应PSO-BP神经网络焊接缺陷成因诊断

焊接缺陷产生原因复杂,影响因素众多,基于人工智能的缺陷成因诊断算法成为焊接智能化的发展方向.?将PSO-BP神经网络应用于焊接缺陷成因诊断,利用神经网络的连接学习机制代替传统专家系统的规则推理机制,并对PSO算法进行自适应调整,引入动态权重因子,搭建自适应PSO-BP神经网络模型.?结果表明,与传统PSO-BP神经网络...     

几种模数转换的结构特点及其发展趋势

模数转换器是许多信息处理系统的关键组成部分, 围绕着精度和速度两个方向快速发展,并出现了多种结构,且各具特色.文章简要介绍了当前模数转换主流技术的原理、结构、工作方式以及特点,并指出了各自所存在的优缺点.最后,根据对各种模数转换技术的分析,展示了模数转换技术的发展趋势.     

High-level Inferencing in a Connectionist Network

Connectionist models have had problems representing and applying general knowledge rules that specifically require variables. This variable binding problem has barred them from performing the high-level inferencing necessary for planning, reasoning, and natural language understanding. This paper describes ROBIN, a structured neural network model capable of high-level inferencing requiring variable bindings and rule application. Variable bindings are handled by signatures—activation patterns which uniquely identify the concept bound to a role. Signatures allow multiple role-bindings to be propagated across the network in parallel for rule application and dynamic inference path instantiation. Signatures are integrated within a connectionist semantic network structure whose constraint-relaxation process selects between those newly-instantiated inferences. This allows ROBIN to handle an area of high-level inferencing difficult even for symbolic models, that of resolving multiple constraints from context to select the best interpretation from among several alternative and possibly ambiguous inference paths.     

用人工神经网络构建碳钢、低合金钢大气腐蚀模型

采用人工神经网络技术建立了碳钢、低合金钢大气腐蚀预测模型,神经网络拓朴结构为13-19-1,神经网络模型预测结果和实验数据紧密相符,而且通过单一因素敏感性分析方法,研究了合金元素和环境因素对于大气腐蚀速率的影响,表明该方法的有效性.     

神经网络在线自学习跟踪控制及其在伺服系统中的应用

针对传统自适应和自校正控制中存在的问题，提出一种基于神经网络的在线自学习控制方法，既做到了对象模型的在线辨识和控制器的在线设计，又避免了神经网络控制方法通常存在的实时控制的困难，使复杂系统的在线学习控制成为可能。仿真表明该方法具有良好的鲁棒性和控制精度。     

Semantic Transparency,Brain Monitoring and Evaluation of Hybrid Cognitive Architectures

In constructing hybrid systems, there is a need for a principled basis to determine the relative roles or functions of artificial neural network and symbolic approaches. The primary objective of the work to be reported is the construction of a conceptual and methodological framework that permits an iterative sequence in which a hybrid model predicts the basis of cognitive performance and an objective analysis of performance provides empirical data, evaluating (and thus constraining) the structure and processes of the model. In seeking a linkage between a hybrid model of cognition and human performance the concept of “semantic transparency” has been adopted, since it can be used in analyzing and describing both the chracteristics of a model of cognition and the processes underlying human performance. An overview of a specific, ”strong” hybrid architecture is presented. The characteristics of the virtual machines which compose it and the nature of their interaction are illustrated. An approach to the questions of evaluation is described based on empirical data obtained by brain monitoring of subjects during cognitive performance.     

基于RS_RBFNN的钛合金焊接接头疲劳寿命预测

建立了基于RS与RBF神经网络集成的钛合金焊接接头疲劳寿命预测模型(RS_RBFNN),该模型首先基于熵的连续属性离散化算法离散化疲劳数据并应用遗传算法约简疲劳寿命评价指标;基于最小约简指标提取焊接结构疲劳寿命分类判别规则以及对RBF神经网络进行训练;最后使用粗糙集理论判别与规则库匹配的检验样本疲劳寿命等级,使用RBF神经网络判别不与规则库任何规则匹配的检验样本疲劳寿命等级.基于钛合金疲劳试验数据的实证分析结果表明,RS_RBFNN模型容错性较好、精度较高,对钛合金焊接结构疲劳寿命预测具有一定的实际指导意义.     

面向认证的二维表面形貌规范合理性自动检验

针对二维表面形貌规范不合理导致认证方案关键信息不完整或矛盾,使认证无法完成的问题,提出规范与认证过程自动检验语义本体模型。采用以描述逻辑为数学基础的网络本体语言和语义网规则语言表示面向认证过程的二维表面形貌规范合理性判断方法中的概念语义,给出描述合理性检验过程信息描述方法。在实例中结合描述逻辑的Tableau算法验证自动检验语义本体模型的有效性。     

Drilling wear detection and classification using vibration signals and artificial neural network

In automated flexible manufacturing systems the detection of tool wear during the cutting process is one of the most important considerations. This study presents a comparison between several architectures of the multi-layer feed-forward neural network with a back propagation training algorithm for tool condition monitoring (TCM) of twist drill wear. The algorithm utilizes vibration signature analysis as the main and only source of information from the machining process. The objective of the proposed study is to produce a TCM system that will lead to a more efficient and economical drilling tool usage. Five different drill wear conditions were artificially introduced to the neural network for prediction and classification. The experimental procedure for acquiring vibration data and extracting features in both the time and frequency domains to train and test the neural network models is detailed. It was found that the frequency domain features, such as the averaged harmonic wavelet coefficients and the maximum entropy spectrum peaks, are more efficient in training the neural network than the time domain statistical moments. The results demonstrate the effectiveness and robustness of using the vibration signals in a supervised neural network for drill wear detection and classification.     

Emergent latent symbol systems in recurrent neural networks

Fodor and Pylyshyn [(1988). Connectionism and cognitive architecture: A critical analysis. Cognition, 28(1–2), 3–71] famously argued that neural networks cannot behave systematically short of implementing a combinatorial symbol system. A recent response from Frank et al. [(2009). Connectionist semantic systematicity. Cognition, 110(3), 358–379] claimed to have trained a neural network to behave systematically without implementing a symbol system and without any in-built predisposition towards combinatorial representations. We believe systems like theirs may in fact implement a symbol system on a deeper and more interesting level: one where the symbols are latent – not visible at the level of network structure. In order to illustrate this possibility, we demonstrate our own recurrent neural network that learns to understand sentence-level language in terms of a scene. We demonstrate our model's learned understanding by testing it on novel sentences and scenes. By paring down our model into an architecturally minimal version, we demonstrate how it supports combinatorial computation over distributed representations by using the associative memory operations of Vector Symbolic Architectures. Knowledge of the model's memory scheme gives us tools to explain its errors and construct superior future models. We show how the model designs and manipulates a latent symbol system in which the combinatorial symbols are patterns of activation distributed across the layers of a neural network, instantiating a hybrid of classical symbolic and connectionist representations that combines advantages of both.