Interruptibility as a constraint on hybrid systems

It is widely mooted that a plausible computational cognitive model should involve both symbolic and connectionist components. However, sound principles for combining these components within a hybrid system are currently lacking; the design of such systems is oftenad hoc. In an attempt to ameliorate this we provide a framework of types of hybrid systems and constraints therein, within which to explore the issues. In particular, we suggest the use of system independent constraints, whose source lies in general considerations about cognitive systems, rather than in particular technological or task-based considerations. We illustrate this through a detailed examination of an interruptibility constraint: handling interruptions is a fundamental facet of cognition in a dynamic world. Aspects of interruptions are delineated, as are their precise expression in symbolic and connectionist systems. We illustrate the interaction of the various constraints from interruptibility in the different types of hybrid systems. The picture that emerges of the relationship between the connectionist and the symbolic within a hybrid system provides for sufficient flexibility and complexity to suggest interesting general implications for cognition, thus vindicating the utility of the framework.     

Syntactic systematicity in sentence processing with a recurrent self-organizing network

As potential candidates for explaining human cognition, connectionist models of sentence processing must demonstrate their ability to behave systematically, generalizing from a small training set. It has recently been shown that simple recurrent networks and, to a greater extent, echo-state networks possess some ability to generalize in artificial language learning tasks. We investigate this capacity for a recently introduced model that consists of separately trained modules: a recursive self-organizing module for learning temporal context representations and a feedforward two-layer perceptron module for next-word prediction. We show that the performance of this architecture is comparable with echo-state networks. Taken together, these results weaken the criticism of connectionist approaches, showing that various general recursive connectionist architectures share the potential of behaving systematically.     

Building brains for bodies

We describe a project to capitalize on newly available levels of computational resources in order to understand human cognition. We are building an integrated physical system including vision, sound input and output, and dextrous manipulation, all controlled by a continuously operating large scale parallel MIMD computer. The resulting system will learn to think by building on its bodily experiences to accomplish progressively more abstract tasks. Past experience suggests that in attempting to build such an integrated system we will have to fundamentally change the way artificial intelligence, cognitive science, linguistics, and philosophy think about the organization of intelligence. We expect to be able to better reconcile the theories that will be developed with current work in neuroscience.     

Connectionism and Novel Combinations of Skills: Implications for Cognitive Architecture

In the late 1980s, there were many who heralded the emergence of connectionism as a new paradigm – one which would eventually displace the classically symbolic methods then dominant in AI and Cognitive Science. At present, there remain influential connectionists who continue to defend connectionism as a more realistic paradigm for modeling cognition, at all levels of abstraction, than the classical methods of AI. Not infrequently, one encounters arguments along these lines: given what we know about neurophysiology, it is just not plausible to suppose that our brains are digital computers. Thus, they could not support a classical architecture. I argue here for a middle ground between connectionism and classicism. I assume, for argument's sake, that some form(s) of connectionism can provide reasonably approximate models – at least for lower-level cognitive processes. Given this assumption, I argue on theoretical and empirical grounds that most human mental skills must reside in separate connectionist modules or sub-networks. Ultimately, it is argued that the basic tenets of connectionism, in conjunction with the fact that humans often employ novel combinations of skill modules in rule following and problem solving, lead to the plausible conclusion that, in certain domains, high level cognition requires some form of classical architecture. During the course of argument, it emerges that only an architecture with classical structure could support the novel patterns of information flow and interaction that would exist among the relevant set of modules. Such a classical architecture might very well reside in the abstract levels of a hybrid system whose lower-level modules are purely connectionist.     

Strong Semantic Systematicity from Hebbian Connectionist Learning

Fodor's and Pylyshyn's stand on systematicity in thought and language has been debated and criticized. Van Gelder and Niklasson, among others, have argued that Fodor and Pylyshyn offer no precise definition of systematicity. However, our concern here is with a learning based formulation of that concept. In particular, Hadley has proposed that a network exhibits strong semantic systematicity when, as a result of training, it can assign appropriate meaning representations to novel sentences (both simple and embedded) which contain words in syntactic positions they did not occupy during training. The experience of researchers indicates that strong systematicity in any form is difficult to achieve in connectionist systems.Herein we describe a network which displays strong semantic systematicity in response to Hebbian, connectionist training. During training, two-thirds of all nouns are presented only in a single syntactic position (either as grammatical subject or object). Yet, during testing, the network correctly interprets thousands of sentences containing those nouns in novel positions. In addition, the network generalizes to novel levels of embedding. Successful training requires a, corpus of about 1000 sentences, and network training is quite rapid. The architecture and learning algorithms are purely connectionist, but classical insights are discernible in one respect, viz, that complex semantic representations spatially contain their semantic constituents. However, in other important respects, the architecture is distinctly non-classical.     

Link Travel Times II: Properties Derived from Traffic-Flow Models

We investigate the properties of travel times when the latter are derived from traffic-flow models. In particular we consider exit-flow models, which have been used to model time-varying flows on road networks, in dynamic traffic assignment (DTA). But we here define the class more widely to include, for example, models based on finite difference approximations to the LWR (Lighthill, Whitham and Richards) model of traffic flow, and large step versions of these. For the derived travel times we investigate the properties of existence, uniqueness, continuity, first-in-first-out (FIFO), causality and time-flow consistency (or intertemporal consistency). We assume a single traffic type and assume that time may be treated as continuous or as discrete, and for each case we obtain conditions under which the above properties are satisfied, and interrelations among the properties. For example, we find that FIFO is easily satisfied, but not strict causality, and find that if we redefine travel time to ensure strict causality then we lose time-flow consistency, and that neither of these conditions is strictly necessary or sufficient for FIFO. All of the models can be viewed as an approximation to a model that is continuous in time and space (the LWR model), and it seems that any loss of desirable properties is the price we pay for using such approximations. We also extend the exit-flow models and results to allow inhomogeneity over time (link capacity or other parameters changing over time), and show that FIFO is still ensured if the exit-flow function is defined appropriately.     

Color constancy from mutual reflection

Mutual reflection occurs when light reflected from one surface illuminates a second surface. In this situation, the color of one or both surfaces can be modified by a color-bleeding effect. In this article we examine how sensor values (e.g., RGB values) are modified in the mutual reflection region and show that a good approximation of the surface spectral reflectance function for each surface can be recovered by using the extra information from mutual reflection. Thus color constancy results from an examination of mutual reflection. Use is made of finite dimensional linear models for ambient illumination and for surface spectral reflectance. If m and n are the number of basis functions required to model illumination and surface spectral reflectance respectively, then we find that the number of different sensor classes p must satisfy the condition p(2 n+m)/3. If we use three basis functions to model illumination and three basis functions to model surface spectral reflectance, then only three classes of sensors are required to carry out the algorithm. Results are presented showing a small increase in error over the error inherent in the underlying finite dimension models.     

On the artificiality of artificial intelligence

In this article the question is raised whether artificial intelligence has any psychological relevance, i.e. contributes to our knowledge of how the mind/brain works. It is argued that the psychological relevance of artificial intelligence of the symbolic kind is questionable as yet, since there is no indication that the brain structurally resembles or operates like a digital computer. However, artificial intelligence of the connectionist kind may have psychological relevance, not because the brain is a neural network, but because connectionist networks exhibit operating characteristics which mimic operant behavior. Finally it is concluded that, since most of the work done so far in AI and Law is of the symbolic kind, it has as yet contributed little to our understanding of the legal mind.     

The “explicit-implicit” distinction

Much of traditional AI exemplifies the explicit representation paradigm, and during the late 1980's a heated debate arose between the classical and connectionist camps as to whether beliefs and rules receive an explicit or implicit representation in human cognition. In a recent paper, Kirsh (1990) questions the coherence of the fundamental distinction underlying this debate. He argues that our basic intuitions concerning explicit and implicit representations are not only confused but inconsistent. Ultimately, Kirsh proposes a new formulation of the distinction, based upon the criterion ofconstant time processing.The present paper examines Kirsh's claims. It is argued that Kirsh fails to demonstrate that our usage of explicit and implicit is seriously confused or inconsistent. Furthermore, it is argued that Kirsh's new formulation of the explicit-implicit distinction is excessively stringent, in that it banishes virtually all sentences of natural language from the realm of explicit representation. By contrast, the present paper proposes definitions for explicit and implicit which preserve most of our strong intuitions concerning straightforward uses of these terms. It is also argued that the distinction delineated here sustains the meaningfulness of the abovementioned debate between classicists and connectionists.     

Multiscale modeling of developmental processes

In contrast to most synthetic neural nets, biological neural networks have a strong component of genetic determination which acts before and during experiential learning. Three broad levels of phenomena are present: long-term evolution, involving crossover as well as point mutation; a developmental process mapping genetic information to a set of cells and their internal states of gene expression (genotype to phenotype); and the subsequent synaptogenesis. We describe a very simple mathematical idealization of these three levels which combines the crossover search method of genetic algorithms with the developmental models used in our previous work on genetic or recursively generated artificial neural nets [18] (and elaborated into a connectionist model of biological development [19]). Despite incorporating all three levels (evolution on genes; development of cells; synapse formation) the model may actually be far cheaper to compute with than a comparable search directly in synaptic weight space.Supported in part by grant 1-RO1-RR07801 from the National Institutes of Health of the US.Supported in part by the US Air Force Office of Scientific Research under grant US AFOSR 88-0240.     

Quantum Surface of Section Method: Decomposition of the Resolvent (E - Ĥ)-1

The paper presents the exact surface of section reduction of quantum mechanics. The main theoretical result is a decomposition of the energy-dependent propagator (E) = (E - )-1 in terms of the propagators which (also or exclusively) act in Hilbert space of complex-valued functions over the configurational surface of section, which has one dimension less than the original configuration space. These energy-dependent quantum propagators from and/or onto the configurational surface of section can be explicitly constructed as the solutions of the first order nonlinear Riccati-like initial value problems.     

Comments on Bechtel, “Levels of description and explanation in cognitive science”

I begin by tracing some of the confusions regarding levels and reduction to a failure to distinguish two different principles according to which theories can be viewed as hierarchically arranged — epistemic authority and ontological constitution. I then argue that the notion of levels relevant to the debate between symbolic and connectionist paradigms of mental activity answers to neither of these models, but is rather correlative to the hierarchy of functional decompositions of cognitive tasks characteristic of homuncular functionalism. Finally, I suggest that the incommensurability of the intentional and extensional vocabularies constitutes a strongprima facie reason to conclude that there is little likelihood of filling in the story of Bechtel's missing level in such a way as to bridge the gap between such homuncular functionalism and his own model of mechanistic explanation.     

Balancing the criticisms: Validating multi-agent models of social systems

Using multi-agent models to study social systems has attracted criticisms because of the challenges involved in their validation. Common criticisms that we have encountered are described, and for each one we attempt to give a balanced perspective of the criticism. A model of intra-state conflict is used to help demonstrate these points. We conclude that multi-agent models for social systems are most useful when (1) the connection between micro-behaviors and macro-behaviors are not well-understood and (2) when data collection from the real-world system is prohibitively expensive in terms of time or money or if it puts human lives at risk.     

Necessary and sufficient conditions for the universality of programming formalisms

Summary Over many familiar datatypes the notion of computable coincides with the notion of flowchartable. It is also known that flowcharts are not a universal programming formalism over arbitrary datatypes, in the sense that there are datatypes over which not all computable functions are flowchartable. In this paper we consider various extensions and restrictions of the basic formalism of flowcharts, and then for every such formalism, we characterize the datatypes over which the computable functions are exactly the functions programmable in this formalism. We say that a function is computable over a datatype if it is effective relative to the primitive operations and relations of the datatype.     

Assessing the Changeability of two Object-Oriented Design Alternatives--a Controlled Experiment

An important motivation for the object-oriented paradigm is to improve the changeability of the software, thereby reducing lifetime development costs. This paper describes the results of controlled experiments assessing the changeability of a given responsibility-driven (RD) design versus an alternative control-oriented mainframe (MF) design. According to Coad and Yourdon's OO design quality principles, the RD design represents a good design. The MF design represents a bad design. To investigate which of the designs have better changeability, we conducted two controlled experiments--a pilot experiment and a main experiment. In both experiments, the subjects were divided in two groups in which the individuals designed, coded and tested several identical changes on one of the two design alternatives.The results clearly indicate that the good RD design requires significantly more change effort for the given set of changes than the alternative bad MF design. This difference in change effort is primarily due to the difference in effort required to understand how to solve the change tasks. Consequently, reducing class-level coupling and increasing class cohesion may actually increase the cognitive complexity of a design. With regards to correctness and learning curve, we found no significant differences between the twodesigns. However, we found that structural attributes change less for the RD design than for the MF design. Thus, the RD design may be less prone to structural deterioration. A challenging issue raised in this paper is therefore the tradeoff between change effort and structural stability.     

A Nondeterministic Extension over Finitely Recursive Process Model

This paper extends the Finitely Recursive Process framework introduced by Inan and Varaiya for modelling Discrete Event Systems to encompass nondeterministic processes. Nondeterminism has been captured as a set of possible deterministic futures instead of using the standard failure model of Communicating Sequential Processes. In the beginning a general structure of finitely recursive process space is provided with some important modifications. Next, the nondeterministic process space has been introduced as a special case of the general algebraic process space. A collection of operators has been defined over this nondeterministic process space that enables its characterisation in a finitely recursive manner. Finally, the advantages and disadvantages of the proposed model vis-a-vis other nondeterministic models of discrete event systems are discussed.     

Grounding symbols into perceptions

This paper deals with symbol formation, from a cognitive point of view, through a connectionist model. To give an idea of our aim, let us consider the metaphor of learning to play tennis. Two knowledge forms are involved:

- implicit knowledge, e.g. sensori-motor associations; this knowledge is subsymbolic

- explicit knowledge, e.g. a teacher giving verbal advice, which makes use of symbols.

Learned knowledge consists of a combination of subsymbolic and symbolic items. More than a juxtaposition, this combination involves grounding symbols into a subsymbolic substratum. This leads us to connectionist modelling which is considered as the common framework for both kinds of knowledge.     

Fast Evolutionary Learning with Batch-Type Self-Organizing Maps

Although no distance function over the input data is definable, it is still possible to implement the self-organizing map (SOM) process using evolutionary-learning operations. The process can be made to converge more rapidly when the probabilistic trials of conventional evolutionary learning are replaced by averaging using the so-called Batch Map version of the self-organizing map. Although no other condition or metric than a fitness function between the input samples and the models is assumed, an order in the map that complies with the functional similarity of the models can be seen to emerge. There exist two modes of use of this new principle: representation of nonmetric input data distributions by models that may have variable structures, and fast generation of evolutionary cycles that resemble those defined by the genetic algorithms. The spatial order in the array of models can be utilized for finding more uniform variations, such as crossings between functionally similar models.