Schemas,logics, and neural assemblies

To implement schemas and logics in connectionist models, some form of basic-level organization is needed. This paper proposes such an organization, which is termed a discrete neural assembly. Each discrete neural assembly is in turn made up of discrete neurons (nodes), that is, a node that processes inputs based on a discrete mapping instead of a continuous function. A group of discrete neurons (nodes) closely interconnected form an assembly and carry out a basic functionality. Some substructures and superstructures of such assemblies are developed to enable complex symbolic schemas to be represented and processed in connectionist networks. The paper shows that logical inference can be performed precisely, when necessary, in these networks and with certain genaralization, more flexible inference (fuzzy inference) can also be performed. The development of various connectionist constructs demonstrates the possibility of implementing symbolic schemas, in their full complexity, in connectionist networks.     

A Hybrid Architecture for Situated Learning of Reactive Sequential Decision Making

In developing autonomous agents, one usually emphasizes only (situated) procedural knowledge, ignoring more explicit declarative knowledge. On the other hand, in developing symbolic reasoning models, one usually emphasizes only declarative knowledge, ignoring procedural knowledge. In contrast, we have developed a learning model CLARION, which is a hybrid connectionist model consisting of both localist and distributed representations, based on the two-level approach proposed in [40]. CLARION learns and utilizes both procedural and declarative knowledge, tapping into the synergy of the two types of processes, and enables an agent to learn in situated contexts and generalize resulting knowledge to different scenarios. It unifies connectionist, reinforcement, and symbolic learning in a synergistic way, to perform on-line, bottom-up learning. This summary paper presents one version of the architecture and some results of the experiments.     

Neural networks for learning in the real world: representation, reinforcement and dynamics

It is argued that the backpropagation learning algorithm is unsuited to tackling real world problems such as sensory-motor coordination learning or the encoding of large amounts of background knowledge in neural networks. One difficulty in the real world - the unavailability of ‘teachers’ who already know the solution to problems, may be overcome by the use of reinforcement learning algorithms in place of backpropagation. It is suggested that the complexity of search space in real world neural network learning problems may be reduced if learning is divided into two components. One component is concerned with abstracting structure from the environment and hence with developing representations of stimuli. The other component involves associating and refining these representations on the basis of feedback from the environment. Time-dependent learning problems are also considered in this hybrid framework. Finally, an ‘open systems’ approach in which subsets of a network may adapt independently on the basis of spatio-temporal patterns is briefly discussed.     

Learning invariant features using inertial priors

We address the technical challenges involved in combining key features from several theories of the visual cortex in a single coherent model. The resulting model is a hierarchical Bayesian network factored into modular component networks embedding variable-order Markov models. Each component network has an associated receptive field corresponding to components residing in the level directly below it in the hierarchy. The variable-order Markov models account for features that are invariant to naturally occurring transformations in their inputs. These invariant features give rise to increasingly stable, persistent representations as we ascend the hierarchy. The receptive fields of proximate components on the same level overlap to restore selectivity that might otherwise be lost to invariance.      

Integrative connectionist learning systems inspired by nature: current models, future trends and challenges

The so far developed and widely utilized connectionist systems (artificial neural networks) are mainly based on a single brain-like connectionist principle of information processing, where learning and information exchange occur in the connections. This paper extends this paradigm of connectionist systems to a new trend—integrative connectionist learning systems (ICOS) that integrate in their structure and learning algorithms principles from different hierarchical levels of information processing in the brain, including neuronal-, genetic-, quantum. Spiking neural networks (SNN) are used as a basic connectionist learning model which is further extended with other information learning principles to create different ICOS. For example, evolving SNN for multitask learning are presented and illustrated on a case study of person authentification based on multimodal auditory and visual information. Integrative gene-SNN are presented, where gene interactions are included in the functioning of a spiking neuron. They are applied on a case study of computational neurogenetic modeling. Integrative quantum-SNN are introduced with a quantum Hebbian learning, where input features as well as information spikes are represented by quantum bits that result in exponentially faster feature selection and model learning. ICOS can be used to solve more efficiently challenging biological and engineering problems when fast adaptive learning systems are needed to incrementally learn in a large dimensional space. They can also help to better understand complex information processes in the brain especially how information processes at different information levels interact. Open questions, challenges and directions for further research are presented.     

On connectionism, rule extraction, and brain-like learning

There is a growing body of work that shows that both fuzzy and symbolic rule systems can be implemented using neural networks. This body of work also shows that these fuzzy and symbolic rules can be retrieved from these networks, once they have been learned by procedures that generally fall under the category of rule extraction. The paper argues that the idea of rule extraction from a neural network involves certain procedures, specifically the reading of parameters from a network, that are not allowed by the connectionist framework that these neural networks are based on. It argues that such rule extraction procedures imply a greater freedom and latitude about the internal mechanisms of the brain than is permitted by connectionism, but that such latitude is permitted by the recently proposed control theoretic paradigm for the brain. The control theoretic paradigm basically suggests that there are parts of the brain that control other parts and has far less restrictions on the kind of procedures that can be called “brain like”. The paper shows that this control theoretic paradigm is supported by new evidence from neuroscience about the role of neuromodulators and neurotransmitters in the brain. In addition, it shows that the control theoretic paradigm is also used in connectionist algorithms, although never acknowledged explicitly. The paper suggests that far better learning and rule extraction algorithms can be developed using these control theoretic notions and they would be consistent with the more recent understanding of how the brain works and learns     

Artificial neural networks for coordination and control: The portability of experiential representations

It is time to locate connectionist representation theory in the new wave of robotics research. The utility of representations developed in artificial neural networks (ANNs) during learning has been demonstrated in cognitive science research since the 1980s. The research reported here puts learned representations to work in a decentered control task, the disembodied arm problem, in which a mobile robot operates an arm fixed to a table to pick up objects. There is no physical linkage between the arm and the robot and so the robot's point of view must be decentered. This is done by developing a modular Artificial Neural Net system in three stages: (i) a classifier net is trained with laser scan data to output transformationally invariant position classes; (ii) an arm net is trained for picking up objects; (iii) an inter net is trained to communicate and coordinate the sensing and acting. The completed system is shown to create new nonsymbolic transformationally invariant representations in order to perform the effective generalization of decentered viewpoints.     

MOVIH-IDS: A mobile-visualization hybrid intrusion detection system

A novel hybrid artificial intelligent system for intrusion detection, called MObile-VIsualization Hybrid IDS (MOVIH-IDS), is presented in this study. A hybrid model built by means of a multiagent system that incorporates an unsupervised connectionist intrusion detection system (IDS) has been defined to guaranty an efficient computer network security architecture. This hybrid IDS facilitates the intrusion detection in dynamic networks, in a more flexible and adaptable manner. The proposed improvement of the system in this paper includes deliberative agents characterized by the use of an unsupervised connectionist model to identify intrusions in computer networks. This hybrid IDS has been probed through several real anomalous situations related to the simple network management protocol as it is potentially dangerous. Experimental results probed the successful detection of such attacks through MOVIH-IDS.     

Representing diverse mathematical problems using neural networks in hybrid intelligent systems

In recent years, artificial neural networks have attracted considerable attention as candidates for novel computational systems. Computer scientists and engineers are developing neural networks as representational and computational models for problem solving: neural networks are expected to produce new solutions or alternatives to existing models. This paper demonstrates the flexibility of neural networks for modeling and solving diverse mathematical problems including Taylor series expansion, Weierstrass's first approximation theorem, linear programming with single and multiple objectives, and fuzzy mathematical programming. Neural network representations of such mathematical problems may make it possible to overcome existing limitations, to find new solutions or alternatives to existing models, and to achieve synergistic effects through hybridization.     

Heterogeneous knowledge representation: integrating connectionist and symbolic computation

Heterogeneous knowledge representation allows combination of several knowledge representation techniques. For instance, connectionist and symbolic systems are two different computational paradigms and knowledge representations. Unfortunately, the integration of different paradigms and knowledge representations is not easy and very often is informal. In this paper, we propose a formal approach to integrate these two paradigms where as a symbolic system we consider a (logic) rule-based system. The integration is operated at language level between neural networks and rule languages. The formal model that allows the integration is based on constraint logic programming and provides an integrated framework to represent and process heterogeneous knowledge. In order to achieve this we define a new language that allows expression and modelling in a natural and intuitive way the above issues together with the operational semantics.     

Bayesian network modelling through qualitative patterns

In designing a Bayesian network for an actual problem, developers need to bridge the gap between the mathematical abstractions offered by the Bayesian-network formalism and the features of the problem to be modelled. Qualitative probabilistic networks (QPNs) have been put forward as qualitative analogues to Bayesian networks, and allow modelling interactions in terms of qualitative signs. They thus have the advantage that developers can abstract from the numerical detail, and therefore the gap may not be as wide as for their quantitative counterparts. A notion that has been suggested in the literature to facilitate Bayesian-network development is causal independence. It allows exploiting compact representations of probabilistic interactions among variables in a network. In the paper, we deploy both causal independence and QPNs in developing and analysing a collection of qualitative, causal interaction patterns, called QC patterns. These are endowed with a fixed qualitative semantics, and are intended to offer developers a high-level starting point when developing Bayesian networks.     

Learning in artificial neural systems

This paper presents an overview and analysis of teaming in artificial neural systems (ANSs). It begins with a general introduction to neural networks and connectionist approaches to information processing. The basis for learning in ANSs is then described and compared with classical machine learning. While similar in some ways, ANS learning deviates from tradition in its dependence on the modification of individual weights to bring about changes in a knowledge representation distributed across connections in a network. This unique form of learning is analyzed from two aspects: the selection of an appropriate network architecture for representing the problem, and the choice of a suitable learning rule capable of reproducing the desired function within the given network. The various network architectures are classified, and then identified with explicit restrictions on the types of functions they are capable of representing. The learning rules, i.e., algorithms that specify how the network weights are modified, are similarly taxonomized and, where possible, the limitations inherent to specific classes of rules are outlined.     

深度计算、广度计算与信息网络

计算技术一直在向深度和广度进军,因此可以把计算技术的发展过程与趋势用“深度计算”和“广度计算”高度概括。从计算技术与通信技术结合的角度来看,网络一直在向“信息传输”与“信息处理”紧密结合的方向发展,即向信息网络方向发展。     

Fractal encoding of context-free grammars in connectionist networks

Connectionist network learning of context-free languages has so far been applied only to very simple cases and has often made use of an external stack. Learning complex context-free languages with a homogeneous neural mechanism looks like a much harder problem. The current paper takes a step toward solving this problem by analyzing context-free grammar computation (without addressing learning) in a class of analog computers called dynamical automata, which are naturally implemented in connectionist networks. The result is a widely applicable method of using fractal sets to organize infinite-state computations in a bounded state space. An appealing consequence is the development of parameter-space maps, which locate various complex computers in spatial relationships to one another. An example suggests that such a global perspective on the organization of the parameter space may be helpful for solving the hard problem of getting connectionist networks to learn complex grammars from examples.     

Distributed Representations,Simple Recurrent Networks,And Grammatical Structure

Machine Learning - In this paper three problems for a connectionist account of language are considered: 1. What is the nature of linguistic representations? 2. How can complex structural...     

过程神经元网络及其在时变信息处理中的应用

针对时变信息处理和动态系统建模等类问题,建立了输入输出均为时变函数的过程神经元网络和有理式过程神经元网络2种网络模型.在输入输出为时变函数的过程神经元网络中,过程神经元的时间累积算子取为对时间的积分或其他代数运算,它的时空聚合机制和激励能同时反映外部时变输入信号对输出结果的空间聚合作用和时间累积效应,可实现非线性系统输入、输出之间的复杂映射关系.在有理式过程神经元网络中,其基本信息处理单元为由2个成对偶出现的过程神经元组成,逻辑上分为分子和分母2部分,通过有理式整合后输出,可有效提高过程神经元网络对带有奇异值过程函数的柔韧逼近性和在奇异值点附近反应的灵敏性.分析了2种过程神经元网络模型的性质,给出了具体学习算法,并以油田开发过程模拟和旋转机械故障诊断问题为例,验证了这2种网络模型在时变信息处理中的有效性.     

Connectionist modelling of skill dynamics

This paper discusses a method for modelling skilled action for synthetic actors in a virtual environment. The method guides lower-level motor skills from a connectionist model of skill memory, implemented as collections of trained neural networks. The relationship between this work and that of other projects in task-level animation is discussed, the principles of connectionist learning are explained, and a series of experiments testing the concept of connectionist skill modelling are reviewed.     

The Causal and Explanatory Role of Information Stored in Connectionist Networks

In this paper I defend the propriety of explaining the behavior of distributed connectionist networks by appeal to selected data stored therein. In particular, I argue that if there is a problem with such explanations, it is a consequence of the fact that information storage in networks is superpositional, and not because it is distributed. I then develop a ``proto-account' of causation for networks, based on an account of Andy Clark's, that shows even superpositionality does not undermine information-based explanation. Finally, I argue that the resulting explanations are genuinely informative and not vacuous.     

Datacab: a geographical-information-system-based expert system for the design of cable networks

Abstract: Telecommunication networks have evolved over time as a result of technological advances, and network topologies and equipment have become increasingly complex. Expert systems are being successfully applied to the management of telecommunication networks. However, applying expert systems to network design is another especially beneficial yet still not very common approach. In this paper we propose a rule-based expert system called Datacab. Datacab was developed at Enditel Endesa in collaboration with the Electronic Technology Department of the University of Seville, for the automatic design of hybrid fibre coax (HFC) cable networks. Using data from a geographical information system as input, it automatically generates viable HFC network designs.