Automated labeling for unsupervised neural networks: a hierarchicalapproach

In this paper a hybrid system and a hierarchical neural-net approaches are proposed to solve the automatic labeling problem for unsupervised clustering. The first method involves the application of nonneural clustering algorithms directly to the output of a neural net; and the second one is based on a multilayer organization of neural units. Both methods are a substantial improvement with respect to the most important unsupervised neural algorithms existing in the literature. Experimental results are shown to illustrate clustering performance of the systems.     

Artificial biochemical networks: a different connectionist paradigm

Connectionist models are usually based on artificial neural networks. However, there is another route towards parallel distributed processing. This is by considering the origins of the intelligence displayed by the single celled organisms known as protoctists. Such intelligence arises by means of the biochemical interactions within the animal. An artificial model of this might therefore be termed an artificial biochemical network or ABN. This paper describes the attributes of such networks and illustrates their abilities in pattern recognition problems and in generating time-varying signals of a type which can be used in many control tasks. The flexibility of the system is explained using legged robots as an example. The networks are trained using back propagation and evolutionary algorithms such as genetic algorithms.     

Stability criteria for unsupervised temporal association networks

A biologically realizable, unsupervised learning rule is described for the online extraction of object features, suitable for solving a range of object recognition tasks. Alterations to the basic learning rule are proposed which allow the rule to better suit the parameters of a given input space. One negative consequence of such modifications is the potential for learning instability. The criteria for such instability are modeled using digital filtering techniques and predicted regions of stability and instability tested. The result is a family of learning rules which can be tailored to the specific environment, improving both convergence times and accuracy over the standard learning rule, while simultaneously insuring learning stability.     

Categorization in unsupervised neural networks: the Eidos model

Proulx and Begin (1995) recently explained the power of a learning rule that combines Hebbian and anti-Hebbian learning in unsupervised auto-associative neural networks. Combined with the brain-state-in-a-box transmission rule, this learning rule defines a new model of categorization: the Eidos model. To test this model, a simulated neural network, composed of 35 interconnected units, is subjected to an alphabetical characters recognition task. The results indicate the necessity of adding two parameters to the model: a restraining parameter and a forgetting parameter. The study shows the outstanding capacity of the model to categorize highly altered stimuli after a suitable learning process. Thus, the Eidos model seems to be an interesting option to achieve categorization in unsupervised neural networks.     

Fast connectionist learning: words and case

Some basic principles of connectionist research are explained along with an account of a number of the techniques necessary for constructing connectionist models. The objective is to introduce the area to people with limited mathematical and computational backgrounds by reducing the examples to simple arithmetic. In this way, a solid basis will be provided for one of the learning algorithms that have been fundamental to the development of network learning: the Hebbian learning rule. After outlining the technique in detail, two examples are provided to make the ideas concrete. These are learning to associate visual features with words and learning case representations.1. Of course, this is a very simple account of language representation, but it suffices our current purposes. We do not discuss more difficult problems such as prepositional attachment and recursion.Notes     

A taxonomy for wavelet neural networks applied to nonlinear modelling

This article presents a novel classification of wavelet neural networks based on the orthogonality/non-orthogonality of neurons and the type of nonlinearity employed. On the basis of this classification different network types are studied and their characteristics illustrated by means of simple one-dimensional nonlinear examples. For multidimensional problems, which are affected by the curse of dimensionality, the idea of spherical wavelet functions is considered. The behaviour of these networks is also studied for modelling of a low-dimension map.     

Consciousness: A connectionist manifesto

Connectionism and phenomenology can mutually inform and mutually constrain each other. In this manifesto I outline an approach to consciousness based on distinctions developed by connectionists. Two core identities are central to a connectionist theory of consciouness: conscious states of mind are identical to occurrent activation patterns of processing units; and the variable dispositional strengths on connections between units store latent and unconscious information. Within this broad framework, a connectionist model of consciousness succeeds according to the degree of correspondence between the content of human consciousness (the world as it is experienced) and the interpreted content of the network. Constitutive self-awareness and reflective self-awareness can be captured in a model through its ability to respond to self-reflexive information, identify self-referential categories, and process information in the absence of simultaneous input. The qualitative feel of sensation appears in a model as states of activation that are not fully discriminated by later processing. Connectionism also uniquely explains several specific features of experience. The most important of these is the superposition of information in consciousness — our ability to perceive more than meets the eye, and to apprehend complex categorical and temporal information in a single highly-cognized glance. This superposition in experience matches a superposition of representational content in distributed representations.     

Classifying spatiotemporal object trajectories using unsupervised learning in the coefficient feature space

This paper proposes a novel technique for clustering and classification of object trajectory-based video motion clips using spatiotemporal function approximations. Assuming the clusters of trajectory points are distributed normally in the coefficient feature space, we propose a Mahalanobis classifier for the detection of anomalous trajectories. Motion trajectories are considered as time series and modelled using orthogonal basis function representations. We have compared three different function approximations – least squares polynomials, Chebyshev polynomials and Fourier series obtained by Discrete Fourier Transform (DFT). Trajectory clustering is then carried out in the chosen coefficient feature space to discover patterns of similar object motions. The coefficients of the basis functions are used as input feature vectors to a Self- Organising Map which can learn similarities between object trajectories in an unsupervised manner. Encoding trajectories in this way leads to efficiency gains over existing approaches that use discrete point-based flow vectors to represent the whole trajectory. Our proposed techniques are validated on three different datasets – Australian sign language, hand-labelled object trajectories from video surveillance footage and real-time tracking data obtained in the laboratory. Applications to event detection and motion data mining for multimedia video surveillance systems are envisaged.     

A design and analysis of objective function-based unsupervised neural networks for fuzzy clustering

Fuzzy clustering has played an important role in solving many problems. In this paper, we design an unsupervised neural network model based on a fuzzy objective function, called OFUNN. The learning rule for the OFUNN model is a result of the formal derivation by the gradient descent method of a fuzzy objective function. The performance of the cluster analysis algorithm is often evaluated by counting the number of crisp clustering errors. However, the number of clustering errors alone is not a reliable and consistent measure for the performance of clustering, especially in the case of input data with fuzzy boundaries. We introduce two measures to evaluate the performance of the fuzzy clustering algorithm. The clustering results on three data sets, Iris data and two artificial data sets, are analyzed using the proposed measures. They show that OFUNN is very competitive in terms of speed and accuracy compared to the fuzzy c-means algorithm.     

Evolving fuzzy neural networks for supervised/unsupervised onlineknowledge-based learning

This paper introduces evolving fuzzy neural networks (EFuNNs) as a means for the implementation of the evolving connectionist systems (ECOS) paradigm that is aimed at building online, adaptive intelligent systems that have both their structure and functionality evolving in time. EFuNNs evolve their structure and parameter values through incremental, hybrid supervised/unsupervised, online learning. They can accommodate new input data, including new features, new classes, etc., through local element tuning. New connections and new neurons are created during the operation of the system. EFuNNs can learn spatial-temporal sequences in an adaptive way through one pass learning and automatically adapt their parameter values as they operate. Fuzzy or crisp rules can be inserted and extracted at any time of the EFuNN operation. The characteristics of EFuNNs are illustrated on several case study data sets for time series prediction and spoken word classification. Their performance is compared with traditional connectionist methods and systems. The applicability of EFuNNs as general purpose online learning machines, what concerns systems that learn from large databases, life-long learning systems, and online adaptive systems in different areas of engineering are discussed.     

Evaluation of unsupervised feature extraction neural networks for face recognition

In this paper, new appearances based on neural networks (NN) algorithms are presented for face recognition. Face recognition is subdivided into two main stages: feature extraction and classifier. The suggested NN algorithms are the unsupervised Sanger principal component neural network (Sanger PCNN) and the self-organizing feature map (SOFM), which will be applied for features extraction of the frontal view of a face image. It is of interest to compare the unsupervised network with the traditional Eigenfaces technique. This paper presents an experimental comparison of the statistical Eigenfaces method for feature extraction and the unsupervised neural networks in order to evaluate the classification accuracies as comparison criteria. The classifier is done by the multilayer perceptron (MLP) neural network. Overcoming of the problem of the finite number of training samples per person is discussed. Experimental results are implemented on the Olivetti Research Laboratory database that contains variability in expression, pose, and facial details. The results show that the proposed method SOFM/MLP neural network is more efficient and robust than the Sanger PCNN/MLP and the Eigenfaces/MLP, when used a few number of training samples per person. As a result, it would be more applicable to utilize the SOFM/MLP NN in order to accomplish a higher level of accuracy within a recognition system.     

Solving tangram puzzles: A connectionist approach

We present a connectionist approach for solving Tangram puzzles. Tangram is an ancient Chinese puzzle where the object is to decompose a given figure into seven basic geometric figures. One connectionist approach models Tangram pieces and their possible placements and orientations as connectionist neuron units which receive excitatory connections from input units defining the puzzle and lateral inhibitory connections from competing or conflicting units. the network of these connectionist units, operating as a Boltzmann Machine, relaxes into a configuration in which units defining the solution receive no inhibitory input from other units. We present results from an implementation of our model using the Rochester Connectionist Simulator. © 1993 John Wiley & Sons, Inc.     

A connectionist processing metaphor for diagnostic reasoning

This article describes LIBRA/Dx, a competition-based parallel activation model for diagnostic reasoning. Within a causal network, the model uses a neurally inspired processing paradigm to generate the most plausible explanation for a set of observed manifestations. the model was built using LIBRA: a domain-independent parallel activation network generator, that can be used to build network models with processing paradigms that are tailored to the specifics of an application domain. the underlying theory postulates that by simultaneously satisfying multiple constraints that may exist locally among domain concepts in a causal network (e.g., among disorders, syndromes, manifestations, etc.) it is possible to construct a plausible global explanation for a set of observed signs and symptoms. the proposed processing paradigm which uses an associative network of concepts to represent domain knowledge, lends itself to the kind of interactive processing that is necessary to capture the generative capacity of human diagnostic ability in novel situations. LIBRA/Dx offers a new approach to modeling a higher cognitive process: diagnostic reasoning, specifically in terms of the time-course of processing and the nature of knowledge representation. It further contributes to our current understanding of the phenomena of human cognition, which have eluded successful explication in conventional computational formalisms.     

Certification-based trust models in mobile ad hoc networks: A survey and taxonomy

A mobile ad hoc network is a wireless communication network which does not rely on a pre-existing infrastructure or any centralized management. Securing the exchanges in such network is compulsory to guarantee a widespread development of services for this kind of networks. The deployment of any security policy requires the definition of a trust model that defines who trusts who and how. There is a host of research efforts in trust models framework to securing mobile ad hoc networks. The majority of well-known approaches is based on public-key certificates, and gave birth to miscellaneous trust models ranging from centralized models to web-of-trust and distributed certificate authorities. In this paper, we survey and classify the existing trust models that are based on public-key certificates proposed for mobile ad hoc networks, and then we discuss and compare them with respect to some relevant criteria. Also, we have developed analysis and comparison among trust models using stochastic Petri nets in order to measure the performance of each one with what relates to the certification service availability.     

A connectionist inference model for pattern-directed knowledge representation

In this paper we propose a connectionist model for variable binding. The model is topology dependent on the graph it builds based on the predicates available. The irregular connections between perceptron-like assemblies facilitate forward and backward chaining. The model treats the symbolic data as a sequence and represents the training set as a partially connected network using basic set and graph theory to form the internal representation. Inference is achieved by opportunistic reasoning via the bidirectional connections. Consequently, such activity stabilizes to a multigraph. This multigraph is composed of isomorphic subgraphs which all represent solutions to the query made. Such a model has a number of advantages over other methods in that irrelevant connections are avoided by superimposing positionally dependent sub-structures that are identical, variable binding can be encoded and multiple solutions can be extracted simultaneously. The model also has the ability to adapt its existing architecture when presented with new clauses and therefore add new relationships/rules to the model explicitly; this is done by some partial retraining of the network due to the superimposition properties.     

The nature of unsupervised learning in deep neural networks: A new understanding and novel approach

Over the last decade, the deep neural networks are a hot topic in machine learning. It is breakthrough technology in processing images, video, speech, text and audio. Deep neural network permits us to overcome some limitations of a shallow neural network due to its deep architecture. In this paper we investigate the nature of unsupervised learning in restricted Boltzmann machine. We have proved that maximization of the log-likelihood input data distribution of restricted Boltzmann machine is equivalent to minimizing the cross-entropy and to special case of minimizing the mean squared error. Thus the nature of unsupervised learning is invariant to different training criteria. As a result we propose a new technique called “REBA” for the unsupervised training of deep neural networks. In contrast to Hinton’s conventional approach to the learning of restricted Boltzmann machine, which is based on linear nature of training rule, the proposed technique is founded on nonlinear training rule. We have shown that the classical equations for RBM learning are a special case of the proposed technique. As a result the proposed approach is more universal in contrast to the traditional energy-based model. We demonstrate the performance of the REBA technique using wellknown benchmark problem. The main contribution of this paper is a novel view and new understanding of an unsupervised learning in deep neural networks.     

Cyber security methods for aerial vehicle networks: taxonomy,challenges and solution

Aerial vehicle networks (AVNs) compose a large number of heterogeneous aerial nodes, such as unmanned aerial vehicles, aircrafts and helicopters. The main characteristics of these networks are the high mobility of aerial nodes and the dynamic network topology. AVNs represent attractive targets for attackers due to the fact that aerial nodes could be connected to an untrusted network and hence lead the attackers to launch lethal threats, e.g., aircraft crash. Therefore, the security of AVNs is mandatory. In this article, we examine the challenges of cyber detection methods to secure AVNs and review exiting security schemes proposed in the current literature. Furthermore, we propose a security framework to protect an aircraft (SFA) against malicious behaviors that target aircrafts communication systems. Numerical results show that SFA achieves a high accuracy detection and prediction rates as compared to the current intrusion detection for aircrafts communication system.     

A fuzzy connectionist expert system for visual pattern classification

In this paper, a connectionist model to integrate knowledge-based techniques into neural network approaches for visual pattern classification is presented. We propose a new structure of connectionist model which has rule-following capability as well as instance-based learning capability. Each node of the proposed network is doubly linked by two types of connections: positive connection and negative connection. Such connectionism provides a methodology to construct the classifier from the rule base and allows the expert knowledge to be utilized for the effective learning. For visual pattern classification, we present the techniques for knowledge representation and utilization using the concepts of fuzzy rules and fuzzy relations. We also discuss in this paper some advantageous characteristics of the model: result explanation capability and rule refinement capability. From the experimental results of the handwritten digit classification, the feasibility of the proposed model is evaluated.     

A connectionist computational model for epistemic and temporal reasoning

The importance of the efforts to bridge the gap between the connectionist and symbolic paradigms of artificial intelligence has been widely recognized. The merging of theory (background knowledge) and data learning (learning from examples) into neural-symbolic systems has indicated that such a learning system is more effective than purely symbolic or purely connectionist systems. Until recently, however, neural-symbolic systems were not able to fully represent, reason, and learn expressive languages other than classical propositional and fragments of first-order logic. In this article, we show that nonclassical logics, in particular propositional temporal logic and combinations of temporal and epistemic (modal) reasoning, can be effectively computed by artificial neural networks. We present the language of a connectionist temporal logic of knowledge (CTLK). We then present a temporal algorithm that translates CTLK theories into ensembles of neural networks and prove that the translation is correct. Finally, we apply CTLK to the muddy children puzzle, which has been widely used as a test-bed for distributed knowledge representation. We provide a complete solution to the puzzle with the use of simple neural networks, capable of reasoning about knowledge evolution in time and of knowledge acquisition through learning.