Unsupervised neural network learning procedures for feature extraction and classification

In this article, we review unsupervised neural network learning procedures which can be applied to the task of preprocessing raw data to extract useful features for subsequent classification. The learning algorithms reviewed here are grouped into three sections: information-preserving methods, density estimation methods, and feature extraction methods. Each of these major sections concludes with a discussion of successful applications of the methods to real-world problems.The first author is supported by research grants from the James S. McDonnell Foundation (grant #93–95) and the Natural Sciences and Engineering Research Council of Canada. For part of this work, the second author was supported by a Temporary Lectureship from the Academic Initiative of the University of London, and by a grant (GR/J38987) from the Science and Engineering Research Council (SERC) of the UK.     

Fast algorithms for computing the diameter of a finite planar set

Three algorithms for computing the diameter of a finite planar set are proposed. Although all three algorithms have (O(n ²) worst-case running time, an expected-complexity analysis shows that, under reasonable probabilistic assumptions, all three algorithms have linear expected running time. Experimental results indicate that two of these algorithms perform very well for some distributions, and are competitive with an existing method. Finally, we exhibit situations where these exact algorithms out-perform a published approximate algorithm.Research of the first author was supported by grant NSERC A 2422. Research of the second author was supported by grants NSERC A 9293, FCAC EQ-1678 and a Killam Senior Research Fellowship awarded by the Canada Council     

Block {ω}-circulant preconditioners¶for the systems of differential equations

The numerical solution of large and sparse nonsymmetric linear systems of algebraic equations is usually the most time consuming part of time-step integrators for differential equations based on implicit formulas. Preconditioned Krylov subspace methods using Strang block circulant preconditioners have been employed to solve such linear systems. However, it has been observed that these block circulant preconditioners can be very ill-conditioned or singular even when the underlying nonpreconditioned matrix is well-conditioned. In this paper we propose the more general class of the block { ω }-circulant preconditioners. For the underlying problems, ω can be chosen so that the condition number of these preconditioners is much smaller than that of the Strang block circulant preconditioner (which belongs to the same class with ω =1) and the related iterations can converge very quickly. Received: January 2002 / Accepted: December 2002 The research of the first author was supported in part by INdAM-GNCS and by a grant from the MURST project “Progetto Giovani Ricercatori anno 2000”. The research of the second author was supported in part by Hong Kong Research Grants Council Grants Nos. HKU 7130/02P and HKU 7132/00P and UK/HK Joint Research Scheme Grant No. 20009819.     

State feedback and estimation of well-posed systems

The class of well-posed systems includes many systems modeled by partial differential equations with boundary control and point sensing as well as many other systems with possibly unbounded control and observation. The closed-loop system created by applying state-feedback to any well-posed system is well-posed. A state-space realization of the closed loop is derived. A similar result holds for state estimation of a well-posed system. Also, the classical state-feedback/estimator structure extends to well-posed systems. In the final section state-space realizations for a doubly coprime factorization for well-posed systems are derived.This research was partially supported by the Fields Institute, which is funded by grants from the Ontario Ministry of Colleges and Universities and the Natural Sciences and Engineering Research Council of Canada, and by a grant from the Natural Sciences and Engineering Research Council of Canada.     

Testing Planarity of Geometric Automorphisms in Linear Time

It is a well-known result that testing a graph for planarity and, in the affirmative case, computing a planar embedding can be done in linear time. In this paper, we show that the same holds if additionally we require that the produced drawing be symmetric with respect to a given automorphism of the graph. This problem arises naturally in the area of automatic graph drawing, where symmetric and planar drawings are desired whenever possible. An extended abstract of this paper has been published in the proceedings of ISAAC 2002 (Lecture Notes in Computer Science, vol. 2518, pp. 563–574, 2002). The first author is supported by the Marie Curie Research Training Network ADONET 504438 funded by the EU. This paper was partially written when he was visiting the University of Sydney. The second author was supported by a grant from the Australian Research Council. National ICT Australia is funded by the Australian Government’s Backing Australia’s Ability initiative, in part through the Australian Research Council.     

Strongest invariant functions: Their use in the systematic analysis of while statements

Summary Strongest invariant functions, a useful tool in the analysis of while statements, are defined and discussed. Their relationships to loop invariants and to the function computed by the while statement are investigated.This research is partially supported by the National Science and Engineering Research Council of Canada under grant number A2509     

Higher order data types

We consider a generalization of the concept of abstract data type suitable for modeling situations in which there is more than one level of functionality. An instance of such a situation is the difference in level of functionality between the query and update functions in a data base. We introduce the concept of a higher order data type to model this idea. The underlying algebraic ideas are outlined, and sample applications of the concept are presented.This work was partially supported by a research grant from the National Research Council of Canada.This work was partially supported by the Conselho Nacional de Desenvolvimento Cientifico e Tecnológico (CNPq; Brazil).     

Analysis of parallel algorithms using pipeline architectures in computer vision applications

Recently a number of machine vision systems have been successfully implemented using pipeline architectures and various new algorithms have been proposed. In this paper we propose a method of analysis of both time complexity and space complexity for algorithms using conventional general purpose pipeline architectures. We illustrate our method by applying it to an algorithm schema for local window operations satisfying a property we define as decomposability. It is shown that the proposed algorithm schema and its analysis generalize previous published results. We further analyse algorithms implementing operators that are not decomposable. In particular the complexities of several median-type operations are compared and the implication on algorithm choice is discussed. We conclude with discussions on space-time trade-offs and implementation issues.This research was partially supported by a grant from the Natural Science and Engineering Research Council of Canada. Part of this work was done while the author was at the University of Guelph, Guelph, Ontario, Canada.     

A Fifth Order Flux Implicit WENO Method

The weighted essentially non-oscillatory (WENO) method is an excellent spatial discretization for hyperbolic partial differential equations with discontinuous solutions. However, the time-step restriction associated with explicit methods may pose severe limitations on their use in applications requiring large scale computations. An efficient implicit WENO method is necessary. In this paper, we propose a prototype flux-implicit WENO (iWENO) method. Numerical tests on classical scalar equations show that this is a viable and stable method, which requires appropriate time-stepping methods. Future study will include the examination of such methods as well as extension of iWENO to systems and higher dimensional problems.Sigal Gottlieb - The work of this author supported by NSF grant DMS-0106743.Steven J. Ruuth - The work of this author was partially supported by a grant from NSERC Canada.     

Computing extremal and approximate distances in graphs having unit cost edges

Summary Using binary search and a Strassen-like matrix multiplication algorithm we obtain efficient algorithms for computing the diameter, the radius, and other distance-related quantities associated with undirected and directed graphs having unit cost (unweighted) edges. Similar methods are used to find approximate values for the distances between all pairs of vertices, and if the graph satisfies certain regularity conditions to find the exact distances.A preliminary version of this research was presented at the Sixteenth Annual Allerton Conference on Communication, Control, and Computing and appears in the Proceedings for that conference. This work was supported in part by the Natural Sciences and Engineering Research Council of Canada under grant A4307 and by the Office of Naval Research under grant N00014-75-6-0752     

An efficiency preorder for processes

A simple efficiency preorder for CCS processes is introduced in whichpq means thatq is at least as fast asp, or more generally,p uses at least as much resources asq. It is shown to be preserved by all CCS contexts except summation and it is used to analyse a non-trivial example: two different implementations of a bounded buffer. Finally we give a sound and complete proof system for finite processes.Most of this work was done while the first author was at the University of Sussex and supported by SERC grant GR/D 97368 of the Science and Engineering Research Council of Great Britain.The second author would like to acknowledge the support of the ESPRIT II Basic Research Action Concur.     

Concurrency and Trie Hashing

Summary The Trie Hashing (TH), defined by Litwin, is one of the fastest access methods for dynamic and ordered files. The hashing function is defined in terms of a trie, which is basically a binary tree where a character string is associated implicitly with each node. This string is compared with a prefix of the given key in the search process, and depending on the result either the left or the right child is chosen as the next node to visit. The leaf nodes point to buckets which contain the records. The buckets are on a disk, whereas the trie itself is in the core memory. In this paper we consider concurrent execution of the TH operations. In addition to the usual search, insertion and deletion operations, we also include range queries among the concurrent operations. Our algorithm locks only leaf nodes and at most two nodes need to be locked simultaneously by any operation regardless of the number of buckets being accessed. The modification required in the basic data structure in order to accommodate concurrent operations is very minor.The work of this author was supported in part by grant 2545-2-87 from the Israeli National Council for Research and DevelopmentThis research was done while visiting I.N.R.I.A., and is supported in part by the Natural Sciences and Engineering Research Council of Canada, Individual Operating Grant A-3182     

The Impact of Non-Gaussian Distribution Traffic on Network Performance

Recent extensive measurements of real-life traffic demonstrate that the probability density function of the traffic in non-Gaussian.If a traffic model does not capture this characteristics,any analytical or simulation results will not be accurate.In this work,we study the impact of non-Gaussian traffic on network performance,and present an approach that can accurately model the marginal distribution of real-life traffic.Both the long-and short-range autocorrelations are also accounted.We show that the removal of non-Gaussian components of the process does not change its correlation structure,and we validate our promising procedure by simulations.     

Stability of discrete-time linear systems with Markovian jumping parameters

This paper deals with the robustness of a class of discrete-time linear systems with Markovian jumping parameters and unknown but bounded uncertainties. Assuming that the Markovian jump process (disturbance) has finite state space and that there is complete access to the system's state and its mode, we establish necessary and sufficient conditions for stochastic stability of the autonomous nominal model. We also establish sufficient conditions for robust stability for this class of uncertain systems under matching conditions and with bounded uncertainties.The research of the first author was supported by the Natural Sciences and Engineering Research Council of Canada under Grant OGP0036444.     

Inductive completion with retracts

Summary In this paper we give countably infinitely many extensions of Huet and Hullot's inductive completion procedure. Also we try and throw some light on the problem of functions which are only partially defined by some set of rewrite rules. We also give a procedure which attempts to show that two derived F-algebras are isomorphic when both of the algebras are realised as retracts.Supported by Science and Engineering Research Council under grant GR/D/81718. Whilst the paper was under revision the author was supported by S.E.R.C grant GR/D/57942     

Algebraic specification of data types: A synthetic approach

A mathematical interpretation is given to the notion of a data type, which allows procedural data types and circularly defined data types. This interpretation seems to provide a good model for what most computer scientists would call data types, data structures, types, modes, clusters or classes. The spirit of this paper is that of McCarthy [43] and Hoare [18]. The mathematical treatment is the conjunction of the ideas of Scott on the solution of domain equations [34], [35], and [36] and the initiality property noticed by the ADJ group (ADJ [2] and [3]). The present work adds operations to the data types proposed by Scott and proposes an alternative to the equational specifications proposed by Guttag [14], Guttag and Horning [15] and ADJ [2]. The advantages of such a mathematical interpretation are the following: throwing light on some ill-understood constructs in high-level programming languages, easing the task of writing correct programs and making possible proofs of correctness for programs or implementations.This research was conducted at the University of Warwick while both authors were supported by the Science Research Council grant B/RG 31948 to D. Park and M. Paterson. During the final redaction of the paper the first author was partially supported by the National Science Foundation grant MCS78-07461. EDITOR'S NOTE: This paper is one of several invited for submission to this journal to present different approaches to the subject of the semantics of programming languages.     

The complexity of induced minors and related problems

The computational complexity of a number of problems concerning induced structures in graphs is studied, and compared with the complexity of corresponding problems concerning non-induced structures. The effect on these problems of restricting the input to planar graphs is also considered. The principal results include: (1) Induced Maximum Matching and Induced Directed Path are NP-complete for planar graphs, (2) for every fixed graphH, InducedH-Minor Testing can be accomplished for planar graphs in time0(n), and (3) there are graphsH for which InducedH-Minor Testing is NP-complete for unrestricted input. Some useful structural theorems concerning induced minors are presented, including a bound on the treewidth of planar graphs that exclude a planar induced minor.The research of the first author was supported by the U.S. Office of Naval Research under Contract N00014-88-K-0456, by the U.S. National Science Foundation under Grant MIP-8603879, and by the National Science and Engineering Research Council of Canada. The second author acknowledges the support of the U.S. Office of Naval Research when visiting the University of Idaho in spring 1990. Some results were also obtained during a visit to the University of Cologne in fall 1990.     

Adaptive decentralized control under non-uniqueness of the optimal control

We study the problem of decentralization of flow control in packet-switching networks under the isarithmic scheme. An incoming packet enters the network only if there are permits available at the entry port when it arrives. The actions of the controllers refer to the routing of permits in the network and the control variables are the corresponding probabilities. We study the behavior of adaptive algorithms implemented at the controllers to update these probabilities and seek optimal performance. This problem can be stated as a routing problem in a closed queueing network. The centralized version of a learning automation is a general framework presented along with the proof of asymptotic optimality. Decentralization of the controller gives rise to non-uniqueness of the optimal control parameters. Non-uniqueness can be exploited to construct asymptotically optimal learning algorithms that exhibit different behavior. We implement two different algorithms for the parallel operation and discuss their differences. Convergence is established using the weak convergence methodology. In addition to our theoretical results, we illustrate the main results using the flow control problem as a model example and verify the predicted behavior of the two proposed algorithms through computer simulations, including an example of tracking.The work of this author was partially supported by a grant from the Canadian Institute for Telecommunications Research under the NCE program of the Government of Canada, and partially supported by NSERC grant WFA 0139015 and FCAR-Québec grant 95-NC-1375.The work of this author was supported by a grant from the CITR under the NCE program of the Government of Canada.     

Duality of constrained Voronoi diagrams and Delaunay triangulations

We introduce theconstrained Voronoi diagram of a planar straight-line graph containingn vertices or sites where the line segments of the graph are regarded as obstacles, and show that an extended version of this diagram is the dual of theconstrained Delaunay triangulation. We briefly discussO(n logn) algorithms for constructing the extended constrained Voronoi diagram.This work was partially supported by a grant from the Natural Sciences and Engineering Research Council of Canada.     

Improved approximation algorithms for MAXk-CUT and MAX BISECTION

Polynomial-time approximation algorithms with nontrivial performance guarantees are presented for the problems of (a) partitioning the vertices of a weighted graph intok blocks so as to maximize the weight of crossing edges, and (b) partitioning the vertices of a weighted graph into two blocks of equal cardinality, again so as to maximize the weight of crossing edges. The approach, pioneered by Goemans and Williamson, is via a semidefinite programming relaxation. The first author was supported in part by NSF Grant CCR-9225008. The work described here was undertaken while the second author was visiting Carnegie Mellon University; at that time he was a Nuffield Science Research Fellow, and was supported in part by Grant GR/F 90363 of the UK Science and Engineering Research Council, and Esprit Working Group 7097 “RAND”.