Superpositional connectionism: A reply to Marinov

Marinov's critique I argue, is vitiated by its failure to recognize the distinctive role of superposition within the distributed connectionist paradigm. The use of so-called subsymbolic distributed encodings alone is not, I agree, enough to justify treating distributed connectionism as a distinctive approach. It has always been clear that microfeatural decomposition is both possible and actual within the confines of recognizably classical approaches. When such approaches also involve statistically-driven learning algorithms — as in the case of ID3 — the fundamental differences become even harder to spot. To see them, it is necessary to consider not just the nature of an acquired input-output function but the nature of the representational scheme underlying it. Differences between such schemes make themselves best felt outside the domain of immediate problem solving. It is in the more extended contexts of performance DURING learning and cognitive change as a result of SUBSEQUENT training on new tasks (or simultaneous training on several tasks) that the effects of superpositional storage techniques come to the fore. I conclude that subsymbols, distribution and statistically driven learning alone are indeed not of the essence. But connectionism is not just about subsymbols and distribution. It is about the generation of whole subsymbol SYSTEMS in which multiple distributed representations are created and superposed.Some of the material in sections 2 and 3 is drawn from Clark, A., ASSOCIATIVE ENGINES: CONNECTIONISM, CONCEPTS AND REPRESENTATIONAL CHANGE (Cambridge, MA MIT/Bradford Books, forthcoming 1993). Thanks to the publishers for permission to reproduce that material here. Thanks also to Chris Thornton for an illuminating discussion of ID3.     

A brief history of connectionism and its psychological implications

Critics of the computational connectionism of the last decade suggest that it shares undesirable features with earlier empiricist or associationist approaches, and with behaviourist theories of learning. To assess the accuracy of this charge the works of earlier writers are examined for the presence of such features, and brief accounts of those found are given for Herbert Spencer, William James and the learning theorists Thorndike, Pavlov and Hull. The idea that cognition depends on associative connections among large networks of neurons is indeed one with precedents, although the implications of this for psychological issues have been interpreted variously — not all versions of connectionism are alike.     

Fodor and Pylyshyn on connectionism

Fodor and Pylyshyn (1988) have argued that the cognitive architecture is not Connectionist. Their argument takes the following form: (1) the cognitive architecture is Classical; (2) Classicalism and Connectionism are incompatible; (3) therefore the cognitive architecture is not Connectionist. In this essay I argue that Fodor and Pylyshyn's defenses of (1) and (2) are inadequate. Their argument for (1), based on their claim that Classicalism best explains the systematicity of cognitive capacities, is an invalid instance of inference to the best explanation. And their argument for (2) turns out to be question-begging. The upshot is that, while Fodor and Pylyshyn have presented Connectionists with the important empirical challenge of explaining systematicity, they have failed to provide sufficient reason for inferring that the cognitive architecture is Classical and not Connectionist.     

Pragmatism,connectionism and the internet: A mind’s perfect storm

This paper explores that natural relationships between Pragmatic theory of knowing, the dynamic structuring of the mind and thinking suggested by connectionist theory, and the way information is distributed and organized through the world wide web (www). We suggest that these three “innovations” can be brought together to offer a better understanding of the way the human mind works. The internet and the information revolution may finally offer the opportunity to use and develop inductive learning practices and information based social inquiry in ways Pragmatic philosophers envisioned a hundred years ago, while the recent rise of connectionist and cognitive architecture works provides a concrete context for such developments. This confluence of process represents the type of synergy that only history can offer. The information revolution – exemplified by both the rise of connectionism and the internet – is the apotheosis of the Pragmatic revolution – bringing together radical empiricism and democratization of information in community practice. We offer three important realizations in our understanding of how information is organized and thinking progresses made possible by burgeoning virtual communities on the internet – open source thinking, scale-free networks, and interrelationships in the development of blogs to illustrate our thesis.     

基于多层卷积神经网络特征和双向长短时记忆单元的行为识别

鲁棒的视频行为识别由于其复杂性成为了一项极具挑战的任务. 如何有效提取鲁棒的时空特征成为解决问题的关键. 在本文中, 提出使用双向长短时记忆单元(Bi--LSTM)作为主要框架去捕获视频序列的双向时空特征. 首先, 为了增强特征表达, 使用多层的卷积神经网络特征代替传统的手工特征. 多层卷积特征融合了低层形状信息和高层语义信息, 能够捕获丰富的空间信息. 然后, 将提取到的卷积特征输入Bi--LSTM, Bi--LSTM包含两个不同方向的LSTM层. 前向层从前向后捕获视频演变, 后向层反方向建模视频演变. 最后两个方向的演变表达融合到Softmax中, 得到最后的分类结果. 在UCF101和HMDB51数据集上的实验结果显示本文的方法在行为识别上可以取得较好的性能.     

Representational trajectories in connectionist learning

The paper considers the problems involved in getting neural networks to learn about highly structured task domains. A central problem concerns the tendency of networks to learn only a set of shallow (non-generalizable) representations for the task, i.e., to miss the deep organizing features of the domain. Various solutions are examined, including task specific network configuration and incremental learning. The latter strategy is the more attractive, since it holds out the promise of a task-independent solution to the problem. Once we see exactly how the solution works, however, it becomes clear that it is limited to a special class of cases in which (1) statistically driven undersampling is (luckily) equivalent to task decomposition, and (2) the dangers of unlearning are somehow being minimized. The technique is suggestive nonetheless, for a variety of developmental factors may yield the functional equivalent of both statistical AND informed undersampling in early learning.     

联接主义智能控制综述

综述了近年来联接主义智能控制的理论和应用上的研究进展,覆盖了神经网络的逼近和泛化能力、神经网络与混沌、监督学习算法等基本性质,以及神经网络建模、预测、优化和控制等联接主义智能控制系统的各个部分,并对今后的研究发展提出了展望.     

Action,connectionism and enaction: A developmental perspective

This article compares the potential of classical and connectionist computational concepts for explanations of innate infant knowledge and of its development. It focuses on issues relating to: the perceptual process; the control and form(s) of perceptual-behavioural coordination; the role of environmental structure in the organization of action; and the construction of novel forms of activity. There is significant compatibility between connectionist and classical views of computation, though classical concepts are, at present, better able to provide a comprehensive computational view of the infant. However, Varela's enaction perspective poses a significant challenge for both approaches.An earlier version of this article was presented at the interdisciplinary seminar La Cognition, organized by the International College of Philosophy, Paris, and the Technological University of Compiègne, Chantilly, France, 1989.     

Implementations of artificial neural networks on the Connection Machine

A review of five distinct artificial neural network implementations on the Connection Machine is presented along with a brief discussion of the more general issues surrounding the implementation of artificial neural network models in parallel. The implementation which proves to be fastest on the Connection Machine is parallel in the training patterns and runs at more than 1300 million interconnects per second.     

Active symbols and internal models: Towards a cognitive connectionism

In the first section of the article, we examine some recent criticisms of the connectionist enterprise: first, that connectionist models are fundamentally behaviorist in nature (and, therefore, non-cognitive), and second that connectionist models are fundamentally associationist in nature (and, therefore, cognitively weak). We argue that, for a limited class of connectionist models (feed-forward, pattern-associator models), the first criticism is unavoidable. With respect to the second criticism, we propose that connectionist modelsare fundamentally associationist but that this is appropriate for building models of human cognition. However, we do accept the point that there are cognitive capacities for which any purely associative model cannot provide a satisfactory account. The implication that we draw from is this is not that associationist models and mechanisms should be scrapped, but rather that they should be enhanced.In the next section of the article, we identify a set of connectionist approaches which are characterized by active symbols — recurrent circuits which are the basis of knowledge representation. We claim that such approaches avoid criticisms of behaviorism and are, in principle, capable of supporting full cognition. In the final section of the article, we speculate at some length about what we believe would be the characteristics of a fully realized active symbol system. This includes both potential problems and possible solutions (for example, mechanisms needed to control activity in a complex recurrent network) as well as the promise of such systems (in particular, the emergence of knowledge structures which would constitute genuine internal models).     

Embedding Connectionist Autonomous Agents in Time: The ‘Road Sign Problem’

This paper identifies a problem of significance for approaches to adaptive autonomous agent research seeking to go beyond reactive behaviour without resorting to hybrid solutions. The feasibility of recurrent neural network solutions are discussed and compared in the light of experiments designed to test ability to handle long-term temporal dependencies, in a more situated context than hitherto. It is concluded that a general-purpose recurrent network with some processing enhancements can begin to fulfil the requirements of this non-trivial problem.     

Convergence of Delayed Dynamical Systems

In this paper, we point out that the conditions given in [1] are sufficient but unnecessary for the global asymptotically stable equilibrium of a class of delay differential equations. Instead, we prove that under weaker conditions, it is still global asymptotically stable.     

Connectionist approaches to the control of manipulation robots at the executive hierarchical level: An overview

One of the most interesting and important properties of connectionist systems is their ability to control sophisticated manipulation robots, i.e. to produce a large number of efficient control commands in real-time. This paper represents an attempt to give a comprehensive report of the basic principles and concepts of connectionism in robotics, with an outline of a number of recent algorithms used in learning control of a manipulation robot. A major concern in this paper is the application of neural networks for off-line and on-line learning of kinematic and dynamic relations used in robot control at the executive hierarchical level.     

Variable Hidden Layer Sizing in Elman Recurrent Neuro-Evolution

The relationship between the size of the hidden layer in a neural network and performance in a particular domain is currently an open research issue. Often, the number of neurons in the hidden layer is chosen empirically and subsequently fixed for the training of the network. Fixing the size of the hidden layer limits an inherent strength of neural networks—the ability to generalize experiences from one situation to another, to adapt to new situations, and to overcome the brittleness often associated with traditional artificial intelligence techniques. This paper proposes an evolutionary algorithm to search for network sizes along with weights and connections between neurons.This research builds upon the neuro-evolution tool SANE, developed by David Moriarty. SANE evolves neurons and networks simultaneously, and is modified in this work in several ways, including varying the hidden layer size, and evolving Elman recurrent neural networks for non-Markovian tasks. These modifications allow the evolution of better performing and more consistent networks, and do so more efficiently and faster.SANE, modified with variable network sizing, learns to play modified casino blackjack and develops a successful card counting strategy. The contributions of this research are up to 8.3% performance increases over fixed hidden layer size models while reducing hidden layer processing time by almost 10%, and a faster, more autonomous approach to the scaling of neuro-evolutionary techniques to solving larger and more difficult problems.     

基于人工神经元网络的控制系统模型简化的专家系统

本文研究并实现了一个基于人工神经元网络的控制系统模型简化的专家系统(简称为ESOMRT)。该系统适用于专家和非专家用户,能够针对更体的连续和离散时间的高阶控制系统模型和简化要求选择合适的简化方法,并可对简化质量从时域和频域方面进行评估。在构造这个系统的过程中,作者提出了智能数据库的概念,使用了过程型和人工神经元网络方法相结合的知识表达方式,并利用神经元网络的再学习机制实现了斗自动知识获取,该系统具有三种工作模式和友好的人机界面,使系统的智能水平比较高并有实用价值,现已在IBM-PC/XT和386机上运行。     

语音编码中的递归神经网络技术研究

众所周知,线谱对(LSP-LinearSpectrumPair)系数是一种线性预测系数,它表征的是语音谱包络。在时域中它的谱插值性能良好,但是它的插值间隔一般都限制在20～30毫秒之间。为了解决这个问题,本文介绍一种使用递归神经神经网络(RNN-RecurrentNeuralNetworks)来对线谱对系数进行插值的算法。实验结果表明,使用递归神经网络可以使插值的间隔增加到100毫秒而不明显降低合成语音的质量。     

Fault Diagnosis Based on the Fuzzy‐Recurrent Neural Network

A fuzzy‐recurrent neural network (FRNN) has been constructed by adding some feedback connections to a feedforward fuzzy neural network (FNN). The FRNN expands the modeling ability of a FNN in order to deal with temporal problems. A basic concept of the FRNN is first to use process or expert knowledge, including appropriate fuzzy logic rules and membership functions, to construct an initial structure and to then use parameter‐learning algorithms to fine‐tune the membership functions and other parameters. Its recurrent property makes it suitable for dealing with temporal problems, such as on‐line fault diagnosis. In addition, it also provides human‐understandable meaning to the normal feedforward multilayer neural network, in which the internal units are always opaque to users. In a word, the trained FRNN has good interpreting ability and one‐step‐ahead predicting ability. To demonstrate the performance of the FRNN in diagnosis, a comparison is made with a conventional feedforward network. The efficiency of the FRNN is verified by the results.     

多项式函数型回归神经网络模型及应用

文中利用回归神经网络既有前馈通路又有反馈通路的特点，将网络隐层中神经元的激活函数设置为可调多项式函数序列，提出了多项式函数型回归神经网络新模型，它不但具有传统回归神经网络的特点，而且具有较强的函数逼近能力，针对递归计算问题，提出了多项式函数型回归神经网络学习算法，并将该网络模型应用于多元多项式近似因式分解，其学习算法在多元多项式近似分解中体现了较强的优越性，通过算例分析表明，该算法十分有效，收敛速度快，计算精度高，可适用于递归计算问题领域，该文所提出的多项式函数型回归神经网络模型及学习算法对于代数符号近似计算有重要的指导意义。     

证券投资的神经网络专家系统

提出证券投资的神经网络专家系统结构，设计了知识获取、推理机、知识库和人机界面４个器件，分析了神经网络专家系统的预前技术。证券投资的实验设计和仿真实验结果表明，这种新的专家系统设计合理可行。     

A class of discrete time recurrent neural networks with multivalued neurons

This paper discusses a class of discrete time recurrent neural networks with multivalued neurons (MVN), which have complex-valued weights and an activation function defined as a function of the argument of a weighted sum. Complementing state-of-the-art of such networks, our research focuses on the convergence analysis of the networks in synchronous update mode. Two related theorems are presented and simulation results are used to illustrate the theory.