MIQR Active Learning on a Continuous Function and a Discontinuous Function

Active learning balances the cost of data acquisition against its usefulness for training. We select only those data points which are the most informative about the system being modelled. The MIQR (Maximum Inter-Quartile Range) criterion is defined by computing the inter-quartile range of the outputs of an ensemble of networks, and finding the input parameter values for which this is maximal. This method ensures data selection is not unduly influenced by ‘outliers’, but is principally dependent upon the ‘mainstream’ state of the ensemble. MIQR is more effective and efficient than contending methods¹ . The algorithm automatically regulates the training threshold and the network architecture as necessary. We compare active learning methods by applying them to a continuous function and a discontinuous function. Training is more difficult for a discontinuous function than a continuous function, and the volume of data for active learning is substantially less than for passive learning.     

A neural network based methodology for machining operations selection in Computer-Aided Process Planning for rotationally symmetrical parts

The relevant literature on machining operations selection in Computer-Aided Process Planning (CAPP) by decision trees, expert systems and neural networks has been reviewed, highlighting their contributions and shortcomings. This paper aims at contributing to the applicability of back-propagation neural network method for the selection of all possible operations for machining rotationally symmetrical components, by prestructuring the neural network with prior domain knowledge in the form of heuristic or thumb rules. It has been achieved by developing two forms of representation for the input data to the neural network. The external representation is used to enter the crisp values of the input decision variables (namely the feature type and its attributes such as diameter or width, tolerance and surface finish). The purpose of internal representation is to categorize the above crisp values into sets, which correspond to all the possible different ranges of the above input variables encountered in the antecedent ‘IF’ part of the thumb rules mentioned above. The input layer of the neural network has been designed in such a way that one neuronal node is allocated for each of the feature types and the sets of various feature attributes. In the output layer of the neural network, one neuronal node is allocated to each of the various feasible machining operation sequences found in the consequent ‘THEN’ part of the thumb rules. A systematic method for training of the neural network has been presented with the above thumb rules used to serve as guidelines for choosing the input patterns of the training examples. This method simplifies the process of training, reduces the time for preparation of training examples and hence the time to develop the overall process planning system. It can further help ensure that the entire problem domain is represented in a better manner and improve the quality of response of the neural network. The example of an industrially-relevant rotationally symmetrical workpiece has been analyzed using the proposed approach to demonstrate its potential for use in the real manufacturing environment. By this novel methodology, workpieces of complex shapes can be handled by investing a very limited amount of time, hence making it attractive and cost effective for industrial applications. Received: June 2005 / Accepted: January 2006     

Study on IAC networks models: learning and stability

This paper presents a methodology that uses evolutionary learning in training ‘A’ model networks, a topology based on Interactive Activation and Competition (IAC) neural networks. IAC networks show local knowledge and processing units clustered in pools. The connections among units may assume only 1, 0 or −1. On the other hand, ‘A’ model network uses values in interval [−1, 1]. This feature provides a wider range of applications for this network, including problems which do not show mutually exclusive concepts. However, there is no algorithm to adjust the network weights and still preserve the desired characteristics of the original network. Accordingly, we propose the use of genetic algorithms in a new methodology to obtain the correct weight set for this network. Two examples are used to illustrate the proposed method. Findings are considered consistent and generic enough to allow further applications on similar classes of problems suitable for ‘A’ model IAC Networks.     

Efficient distribution-free population learning of simple concepts

We consider a variant of the ‘population learning model’ proposed by Kearns and Seung [8], in which the learner is required to be ‘distribution-free’ as well as computationally efficient. A population learner receives as input hypotheses from a large population of agents and produces as output its final hypothesis. Each agent is assumed to independently obtain labeled sample for the target concept and output a hypothesis. A polynomial time population learner is said to PAC-learn a concept class, if its hypothesis is probably approximately correct whenever the population size exceeds a certain bound which is polynomial, even if the sample size for each agent is fixed at some constant. We exhibit some general population learning strategies, and some simple concept classes that can be learned by them. These strategies include the ‘supremum hypothesis finder’, the ‘minimum superset finder’ (a special case of the ‘supremum hypothesis finder’), and various voting schemes. When coupled with appropriate agent algorithms, these strategies can learn a variety of simple concept classes, such as the ‘high–low game’, conjunctions, axis-parallel rectangles and others. We give upper bounds on the required population size for each of these cases, and show that these systems can be used to obtain a speed up from the ordinary PAC-learning model [11], with appropriate choices of sample and population sizes. With the population learner restricted to be a voting scheme, what we have is effectively a model of ‘population prediction’, in which the learner is to predict the value of the target concept at an arbitrarily drawn point, as a threshold function of the predictions made by its agents on the same point. We show that the population learning model is strictly more powerful than the population prediction model. Finally, we consider a variant of this model with classification noise, and exhibit a population learner for the class of conjunctions in this model. This revised version was published online in June 2006 with corrections to the Cover Date.     

Identification of Residential Property Sub-Markets using Evolutionary and Neural Computing Techniques

This paper expands on previous work considering methods of stratifying property data in order to enhance its susceptibility to modelling for mortgage value estimation. Previous work [1] considered a clustering approach using a Kohonen Self-Organising Map (SOM) to stratify the training data prior to training a suite of MLPs. Although the results were encouraging, the approach suffers from its estimation of trainability post-clustering. The following method ameliorates the approach by replacing the static clustering step with a dynamic genetic algorithm implementation. The results show a healthy improvement in accuracy over the non-stratified approach, and a more consistent level of accuracy compared with the Kohonen SOM approach. The paper concludes by analysing the underlying content of the derived stratas, thus providing a ‘human readable’ element to the approach that enhances its potential for acceptance by valuation institutions for as a complementary technique to traditional valuation methods.     

On Optimal Bilinear Quadrilateral Meshes

The novelty of this work is in presenting interesting error properties of two types of asymptotically ‘optimal’ quadrilateral meshes for bilinear approximation. The first type of mesh has an error equidistributing property, where the maximum interpolation error is asymptotically the same over all elements. The second type has faster than expected ‘super-convergence’ property for certain saddle-shaped data functions. The ‘super-convergent’ mesh may be an order of magnitude more accurate than the error equidistributing mesh. Both types of mesh are generated by a coordinate transformation of a regular mesh of squares. The coordinate transformation is derived by interpreting the Hessian matrix of a data function as a metric tensor. The insights in this work may have application in mesh design near known corner or point singularities.     

Backpropagation of an Image Similarity Metric for Autoassociative Neural Networks

Autoassociative Neural Networks (AANNs) are most commonly used for image data compression. The goal of an AANN for image data is to have the network output be ‘similar’ to the input. Most of the research in this area use backpropagation training with Mean-Squared Error (MSE) as the optimisation criteria. This paper presents an alternative error function called the Visual Difference Predictor (VDP) based on concepts from the human-visual system. Using the VDP as the error function provides a criteria to train an AANN more efficiently, and results in faster convergence of the weights, while producing an output image perceived to be very similar by a human observer. Received: 02 December 1998, Received in revised form: 28 June 1999, Accepted: 05 August 1999     

Extraction and recognition of geographical features from paper maps

With the emergence of Geographical Information Systems (GIS), map acquisition and recognition have become hotly pursued topics, both by the industry and the academia. The paper presents a novel methodology for the extraction and recognition of symbol features from topographic maps. The method proceeds by separating the map into its constituent layers and then attempting to recognize the features in different layers on the basis of symbol-specific geometrical and morphological attributes. Text strings have also been separated. The output is obtained in the form of an ‘e-map’ that is vectorized and hence is suitable for GIS. To demonstrate the usefulness of the proposed system a simple database along with a query processing facility is constructed integrating the information obtained from the e-map and ‘some’ user inputs. The methodology has been observed to perform quite satisfactorily.     

Image Fusion for Enhanced Visualization: A Variational Approach

We present a variational model to perform the fusion of an arbitrary number of images while preserving the salient information and enhancing the contrast for visualization. We propose to use the structure tensor to simultaneously describe the geometry of all the inputs. The basic idea is that the fused image should have a structure tensor which approximates the structure tensor obtained from the multiple inputs. At the same time, the fused image should appear ‘natural’ and ‘sharp’ to a human interpreter. We therefore propose to combine the geometry merging of the inputs with perceptual enhancement and intensity correction. This is performed through a minimization functional approach which implicitly takes into account a set of human vision characteristics.     

Quantifying counts and costs via classification

Many business applications track changes over time, for example, measuring the monthly prevalence of influenza incidents. In situations where a classifier is needed to identify the relevant incidents, imperfect classification accuracy can cause substantial bias in estimating class prevalence. The paper defines two research challenges for machine learning. The ‘quantification’ task is to accurately estimate the number of positive cases (or class distribution) in a test set, using a training set that may have a substantially different distribution. The ‘cost quantification’ variant estimates the total cost associated with the positive class, where each case is tagged with a cost attribute, such as the expense to resolve the case. Quantification has a very different utility model from traditional classification research. For both forms of quantification, the paper describes a variety of methods and evaluates them with a suitable methodology, revealing which methods give reliable estimates when training data is scarce, the testing class distribution differs widely from training, and the positive class is rare, e.g., 1% positives. These strengths can make quantification practical for business use, even where classification accuracy is poor.     

Long future frame prediction using optical flow-informed deep neural networks for enhancement of robotic teleoperation in high latency environments

High latency in teleoperation has a significant negative impact on operator performance. While deep learning has revolutionized many domains recently, it has not previously been applied to teleoperation enhancement. We propose a novel approach to predict video frames deep into the future using neural networks informed by synthetically generated optical flow information. This can be employed in teleoperated robotic systems that rely on video feeds for operator situational awareness. We have used the image-to-image translation technique as a basis for the prediction of future frames. The Pix2Pix conditional generative adversarial network (cGAN) has been selected as a base network. Optical flow components reflecting real-time control inputs are added to the standard RGB channels of the input image. We have experimented with three data sets of 20,000 input images each that were generated using our custom-designed teleoperation simulator with a 500-ms delay added between the input and target frames. Structural Similarity Index Measures (SSIMs) of 0.60 and Multi-SSIMs of 0.68 were achieved when training the cGAN with three-channel RGB image data. With the five-channel input data (incorporating optical flow) these values improved to 0.67 and 0.74, respectively. Applying Fleiss' κ gave a score of 0.40 for three-channel RGB data, and 0.55 for five-channel optical flow-added data. We are confident the predicted synthetic frames are of sufficient quality and reliability to be presented to teleoperators as a video feed that will enhance teleoperation. To the best of our knowledge, we are the first to attempt to reduce the impacts of latency through future frame prediction using deep neural networks.     

A divide-and-conquer method for multi-net classifiers

Several researchers have shown that substantial improvements can be achieved in difficult pattern recognition problems by combining the outputs of multiple neural networks. In this work, we present and test a pattern classification multi-net system based on both supervised and unsupervised learning. Following the ‘divide-and-conquer’ framework, the input space is partitioned into overlapping subspaces and neural networks are subsequently used to solve the respective classification subtasks. Finally, the outputs of individual classifiers are appropriately combined to obtain the final classification decision. Two clustering methods have been applied for input space partitioning and two schemes have been considered for combining the outputs of the multiple classifiers. Experiments on well-known data sets indicate that the multi-net classification system exhibits promising performance compared with the case of single network training, both in terms of error rates and in terms of training speed (especially if the training of the classifiers is done in parallel). ID="A1"Correspondance and offprint requests to: D. Frosyniotis, National Technical University of Athens, Department of Electrical and Computer Engineering, Zographou 157 73, Athens, Greece. E-mail: andreas@cs.ntua.gr     

Designing for social interaction with mundane technologies: issues of security and trust

This paper documents some of the socio-technical issues involved in developing security measures for wireless mesh networks (WMNs) that are deployed as part of a community network. We are interested in discovering whether (and exactly how) everyday social interaction over the network is affected by security issues, and any consequent design implications. We adopt an interdisciplinary methodological approach to requirements, treating a community as an ‘organization’ and implementing an approach, OCTAVE, originally designed to uncover security elements for organizations. Using a focus group technique we chart some of the assets and security concerns of the community, concerns that need to be addressed in order for WMNs, or indeed any network, to become a truly ‘mundane technology’.     

Symmetries and discriminability in feedforward networkarchitectures

This paper investigates the effects of introducing symmetries into feedforward neural networks in what are termed symmetry networks. This technique allows more efficient training for problems in which we require the output of a network to be invariant under a set of transformations of the input. The particular problem of graph recognition is considered. In this case the network is designed to deliver the same output for isomorphic graphs. This leads to the question of which inputs can be distinguished by such architectures. A theorem characterizing when two inputs can be distinguished by a symmetry network is given. As a consequence, a particular network design is shown to be able to distinguish nonisomorphic graphs if and only if the graph reconstruction conjecture holds.     

What the <0.70, 1.17, 0.99, 1.07> is a Symbol?

The notion of a ‘symbol’ plays an important role in the disciplines of Philosophy, Psychology, Computer Science, and Cognitive Science. However, there is comparatively little agreement on how this notion is to be understood, either between disciplines, or even within particular disciplines. This paper does not attempt to defend some putatively ‘correct’ version of the concept of a ‘symbol.’ Rather, some terminological conventions are suggested, some constraints are proposed and a taxonomy of the kinds of issue that give rise to disagreement is articulated. The goal here is to provide something like a ‘geography’ of the various notions of ‘symbol’ that have appeared in the various literatures, so as to highlight the key issues and to permit the focusing of attention upon the important dimensions. In particular, the relationship between ‘tokens’ and ‘symbols’ is addressed. The issue of designation is discussed in some detail. The distinction between simple and complex symbols is clarified and an apparently necessary condition for a system to be potentially symbol, or token bearing, is introduced.     

Memory Consolidation and Forgetting During Sleep: A Neural Network Model

Experimental research with humans and animals suggests that sleep — particularly REM sleep — is, in some way, associated with learning. However, the nature of the association and the underlying mechanism remain unclear. A number of theoretical models have drawn inspiration from research into Artificial Neural Networks. Crick and Mitchinson's ‘unlearning’ and Robins and McCallum's ‘pseudo-rehearsal’ models suggest alternative mechanisms through which sleep could contribute to learning. In this paper we present simulations, suggesting a possible synthesis. Our simulations use a modified version of a Hopfield network to model the possible contribution of sleep to memory consolidation. Sleep is simulated by removing all sensory input to the network and by exposing it to a ‘noise’, intended as a highly abstract model of the signals generated by the Ponto-geniculate-occipital system during sleep. The results show that simulated sleep does indeed contribute to learning and that the relationship between the observed effect and the length of simulated sleep can be represented by a U-shaped curve. It is shown that while high-amplitude, low-frequency noise (reminiscent of NREM sleep) leads to a general reinforcement of memory, low-amplitude, high-frequency noise (as observed in REM sleep) leads to ‘forgetting’ of all but the strongest memory traces. This suggests that a combination of the two kinds of sleep might produce a stronger effect than either kind of sleep on its own and that effective consolidation of memory during sleep may depend not just on REM or NREM sleep but on the overall dynamics of the sleep cycle.     

Electrostatic field framework for supervised and semi-supervised learning from incomplete data

In this paper a classification framework for incomplete data, based on electrostatic field model is proposed. An original approach to exploiting incomplete training data with missing features, involving extensive use of electrostatic charge analogy, has been used. The framework supports a hybrid supervised and unsupervised training scenario, enabling learning simultaneously from both labelled and unlabelled data using the same set of rules and adaptation mechanisms. Classification of incomplete patterns has been facilitated by introducing a local dimensionality reduction technique, which aims at exploiting all available information using the data ‘as is’, rather than trying to estimate the missing values. The performance of all proposed methods has been extensively tested in a wide range of missing data scenarios, using a number of standard benchmark datasets in order to make the results comparable with those available in current and future literature. Several modifications to the original Electrostatic Field Classifier aiming at improving speed and robustness in higher dimensional spaces have also been introduced and discussed.     

Why Computers Can’t Feel Pain

The most cursory examination of the history of artificial intelligence highlights numerous egregious claims of its researchers, especially in relation to a populist form of ‘strong’ computationalism which holds that any suitably programmed computer instantiates genuine conscious mental states purely in virtue of carrying out a specific series of computations. The argument presented herein is a simple development of that originally presented in Putnam’s (Representation & Reality, Bradford Books, Cambridge in 1988) monograph, “Representation & Reality”, which if correct, has important implications for turing machine functionalism and the prospect of ‘conscious’ machines. In the paper, instead of seeking to develop Putnam’s claim that, “everything implements every finite state automata”, I will try to establish the weaker result that, “everything implements the specific machine Q on a particular input set (x)”. Then, equating Q (x) to any putative AI program, I will show that conceding the ‘strong AI’ thesis for Q (crediting it with mental states and consciousness) opens the door to a vicious form of panpsychism whereby all open systems, (e.g. grass, rocks etc.), must instantiate conscious experience and hence that disembodied minds lurk everywhere.     

Pest Control Expert System for Tomato (PCEST)

This paper presents a real-life pest control expert system for tomato. The system involves two main subtasks, namely: ‘diagnose’ and ‘treat’. The ‘diagnose’ subtask finds out the causes of the growers' complaints, while the ‘treat’ subtask finds out a treatment plan for these causes. CommonKADS methodology has been used to develop the system. Dependency network is used as one of our knowledge representation schemes in both subtasks. An expert system evaluation methodology has been suggested and applied to the developed system. Received May 1998 / Revised January 1999 / Accepted in revised form May 1999     

A Temporal Logic for Supporting Historical Databases

Tackling data with gap-interval time is an important issue faced by the temporal database community. While a number of interval logics have been developed, less work has been reported on gap-interval time. To represent and handle data with time, a clause ‘when’ is generally added into each conventional operator so as to incorporate time dimension in temporal databases, which clause ‘when’ is really a temporal logical sentence. Unfortunately, though several temporal database models have dealt with data with gap-interval time, they still put interval calculus methods on gap-intervals. Certainly, it is inadequate to tackle data with gap-interval time using interval calculus methods in historical databases. Consequently, what temporal expressions are valid in the clause ‘when’ for tackling data with gap-interval time? Further, what temporal operations and relations can be used in the clause ‘when’? To solve these problems, a formal tool for supporting data with gap-interval time must be explored. For this reason, a gap-interval-based logic for historical databases is established in this paper. In particular, we discuss how to determine the temporal relationships after an event explodes. This can be used to describe the temporal forms of tuples splitting in historical databases. Received 2 February 1999 / Revised 9 May 1999 / Accepted in revised form 20 November 1999