PAC Analogues of Perceptron and Winnow Via Boosting the Margin

We describe a novel family of PAC model algorithms for learning linear threshold functions. The new algorithms work by boosting a simple weak learner and exhibit sample complexity bounds remarkably similar to those of known online algorithms such as Perceptron and Winnow, thus suggesting that these well-studied online algorithms in some sense correspond to instances of boosting. We show that the new algorithms can be viewed as natural PAC analogues of the online p-norm algorithms which have recently been studied by Grove, Littlestone, and Schuurmans (1997, Proceedings of the Tenth Annual Conference on Computational Learning Theory (pp. 171–183) and Gentile and Littlestone (1999, Proceedings of the Twelfth Annual Conference on Computational Learning Theory (pp. 1–11). As special cases of the algorithm, by taking p = 2 and p = we obtain natural boosting-based PAC analogues of Perceptron and Winnow respectively. The p = case of our algorithm can also be viewed as a generalization (with an improved sample complexity bound) of Jackson and Craven's PAC-model boosting-based algorithm for learning sparse perceptrons (Jackson & Craven, 1996, Advances in neural information processing systems 8, MIT Press). The analysis of the generalization error of the new algorithms relies on techniques from the theory of large margin classification.     

Cockpit cognition: Education,the military and cognitive engineering

The goals of public education, as well as conceptions of human intelligence and learning, are undergoing a transformation through the application of military-sponsored information technologies and information processing models of human thought. Recent emphases in education on thinking skills, learning strategies, and computer-based technologies are the latest episodes in the postwar military agenda to engineer intelligent components, human and artificial, for the optimal performance of complex technological systems. Public education serves increasingly as a human factors laboratory and production site for this military enterprise, whose high performance technologies and command and control paradigms have also played central roles in the emergence of the information economy.Our final hope is to develop the brain as a natural resource ... Human intelligence will be the weapon of the future.Luis Alberto MachadoThis paper will also appear, under the title Mental Material inCyborg Worlds: The Military Information Society, eds. Les Levidow and Kevin Robins, London: Free Association Press, (in press).     

Hierarchy theorems for two-way finite state transducers

Summary For a family of languages , CAL() is defined as the family of images of under nondeterministic two-way finite state transducers, while FINITE · VISIT() is the closure of under deterministic two-way finite state transducers; CAL⁰()= and for n0, CAL ⁿ⁺¹()=CAL ⁿ (CAL()). For any semiAFL , if FINITE · VISIT() CAL(), then CAL ⁿ () forms a proper hierarchy and for every n0, FINITE · VISIT(CALⁿ()) CAL ⁿ⁺¹() FINITE · VISIT(CAL ⁿ⁺¹()). If is a SLIP semiAFL or a weakly k-iterative full semiAFL or a semiAFL contained in any full bounded AFL, then FINITE · VISIT() CAL() and in the last two cases, FINITE · VISIT(). If is a substitution closed full principal semiAFL and FINITE · VISIT(), then FINITE · VISIT() CAL(). If is a substitution closed full principal semiAFL generated by a language without an infinite regular set and ₁ is a full semiAFL, then is contained in CAL^m(₁) if and only if it is contained in ₁. Among the applications of these results are the following. For the following families , CAL ⁿ () forms a proper hierarchy: =INDEXED, =ETOL, and any semiAFL contained in CF. The family CF is incomparable with CAL ^m (NESA) where NESA is the family of one-way nonerasing stack languages and INDEXED is incomparable with CAL ^m (STACK) where STACK is the family of one-way stack languages.This work was supported in part by the National Science Foundation under Grants No. DCR74-15091 and MCS-78-04725     

A note on classes of complements and the LBA problem

Summary The complements of an AFL form an AFL if and only if is closed under length-preserving universal quantification. The complements of the context-sensitive languages form a principal AFL with a hardest set L ₁. The context-sensitive languages are closed under complementation if and only if L ₁ is context-sensitive.This research was supported in part by the National Science Foundation under Grants MCS76-10076 and DCR74-15091     

Surface representation for the graphical display of structured data

In many applications one has a set of discrete points at which some variable such as pressure or velocity is measured. In order to graphically represent and display such data (say, as contours of constant pressure), the discrete data must be represented by a smooth function. This continuous surface can then be evaluated at any point for graphical display. Sometimes data are arbitrarily located except that they occur along non-intersecting lines, an example occurring in wind tunnel tests where data are recorded at plug taps on an aircraft body. An algorithm is developed for this type of structured data problem and illustrated by means of color computer graphics.     

Expert systems and computer-controlled decision making in medicine

The search for usable expert systems is leading somemedical researchers to question the appropriate role of these programs. Most current systems assume a limited role for the human user, delegating situated decision-control to the machine. As expert systems are only able to replace a narrow range of human intellectual functions, this leaves the programs unable to cope with the constructivist nature of human knowledge-use. In returning practical control to the human doctor, some researchers are abandoning focusedproblem-solving in favour of supportiveproblem-analysis. Using ONCOCIN and QMR as examples, this article contrasts these approaches and suggests that the latter avoids many of the difficulties currently facing medical expert systems.After hearing for several decades that computers will soon be able to assist with difficult diagnoses, the practicing physician may well wonder why the the revolution has not occurred. Scepticism at this point is understandable. Few, if any, programs currently have active roles as consultants to physicians (Schwartz, Patil and Szolovits, 1987, p. 685)     

Another polynomial homomorphism

Summary The current proposals for applying the so called fast O(N log^a N) algorithms to multivariate polynomials is that the univariate methods be applied recursively, much in the way more conventional algorithms are used. Since the size of the problems is rather large for which a fastrd algorithm is more efficient than a classical one, the recursive approach compounds this size completely out of any practical range.The degree homomorphism is proposed here as an alternative to this recursive approach. It is argued that methods based on the degree homomorphism and a fast algorithm may be viable alternatives to more conventional algorithms for certain multivariate problems in the setting of algebraic manipulation. Several such problems are discussed including: polynomial multiplication, powering, division (both exact and with remainder), greatest common divisors and factoring.This research was supported by NRC Grant A9284.     

Spectrographic analysis and patterns in pronunciation

Though there are good reasons to improve instruction in pronunciation, the teaching of pronunciation has lost popularity among language teachers. This is because the traditional indirect analyses of sounds according to places and manners of articulation are clumsy when applied to classroom teaching. By shifting the focus of instruction to the direct feedback of real-time acoustic analysis in the visual mode, instructors are free from the complex and often unproductive terminology of articulatory phonetics, and students are free from the burden of translating instructors' general comments such as try again or repeat after me into plans for specific changes. Garry Molholt is Assistant Professor of Linguistics and English as a Second Language, and Coordinator of Computer Assisted Instruction. His research interests are the applications of speech processing to instruction in the acquisition of second language phonology. He has published Computer Assisted Instruction in Pronunciation for Chinese Speakers of American English, in TESOL Quarterly, and (with Ari Presler), Correlation between Human and Machine Ratings of Test of Spoken English Reading Passages, in Technology and Language Testing.     

Minimization of norms and logarithmic norms by diagonal similarities

It is a commonly occurring problem to find good norms · or logarithmic norms (·) for a given matrix in the sense that they should be close to respectively the spectral radius (A) and the spectral abscissa (A). Examples may be the certification thatA is convergent, i.e. (A)A<1 or stable, i.e. (A)(A)<0. Often the ordinary norms do not suffice and one would like to try simple modifications of them such as using an ordinary norm for a diagonally transformed matrix. This paper treats this problem for some of the ordinary norms.

---

Minimisierung von Normen und Logarithmischen Normen durch Diagonale Transformationen

Zusammenfassung Ein oft vorkommendes praktisches Problem ist die Konstruktion von guten Normen · und logarithmischen Normen (·) für eine gegebene MatrixA. Mit gut wird dann verstanden, daß A den Spektralradius (A)=max |₁| und (A) die Spektralabszisse (A)=max Re _i gut approximieren. Beispiele findet man für konvergente Matrizen wo (A)A<1 gewünscht ist, und für stabile Matrizen wo (A)(A)<0 zu zeigen ist. Wir untersuchen hier, wie weit man mit Diagonaltransformationen und dengewöhnlichsten Normen kommen kann.

     

Unextendible Product Bases and Locally Unconvertible Bound Entangled States

Mutual convertibility of bound entangled states under local quantum operations and classical communication (LOCC) is studied. We focus on states associated with unextendible product bases (UPB) in a system of three qubits. A complete classification of such UPBs is suggested. We prove that for any pair of UPBs S and T the associated bound entangled states _S and _T cannot be converted to each other by LOCC, unless S and T coincide up to local unitaries. More specifically, there exists a finite precision (S,T) > 0 such that for any LOCC protocol mapping _S into a probabilistic ensemble (p, ), the fidelity between _T and any possible final state satisfies F(_T, ) = 1 - (S,T).PACS: 03.65.Bz; 03.67.-a; 89.70+c.     

Optimal multilayer channel routing with overlap

This paper presents algorithms for multiterminal net channel routing where multiple interconnect layers are available. Major improvements are possible if wires are able to overlap, and our generalized main algorithm allows overlap, but only on everyKth (K 2) layer. Our algorithm will, for a problem with densityd onL layers,L K + 3,provably use at most three tracks more than optimal: (d + 1)/L/K + 2 tracks, compared with the lower bound of d/L/K. Our algorithm is simple, has few vias, tends to minimize wire length, and could be used if different layers have different grid sizes. Finally, we extend our algorithm in order to obtain improved results for adjacent (K = 1) overlap: (d + 2)/2L/3 + 5 forL 7.This work was supported by the Semiconductor Research Corporation under Contract 83-01-035, by a grant from the General Electric Corporation, and by a grant at the University of the Saarland.     

The unified data structure capability in Madcap VI

The data structures which form an integral part of the Madcap VI programming language are described. The initialization (declarationand constructor) expressions and selector expressions of these structures are defined and their implementation using codewords is discussed. Structures, since they can contain references to other structures (including themselves), have the form of directed trees (graphs). Variables of primitive data type (real, complex, etc.) are naturally considered as degenerate graphs, merely single nodes. The possibility for both multiword and fractional-word representation of structures is evident, but the language itself is implementation-independent. Thus a field is simply a substructure. The Madcap VI data structures are compared to data structure concepts in PL/I.This work supported by the United States Atomic Energy Commission.     

Continuous reductions among combinatorial optimization problems

Summary Many reductions among combinatorial problems are known in the context of NP-completeness. These reductions preserve the optimality of solutions. However, they may change the relative error of approximative solutions dramatically. In this paper, we apply a new type of reductions, called continuous reductions. When one problem is continuously reduced to another, any approximation algorithm for the latter problem can be transformed into an approximation algorithm for the former. Moreover, the performance ratio is preserved up to a constant factor. We relate the problem Minimum Number of Inverters in CMOS-Circuits, which arises in the context of logic synthesis, to several classical combinatorial problems such as Maximum Independent Set and Deletion of a Minimum Number of Vertices (Edges) in Order to Obtain a Bipartite (Partial) Subgraph.     

Transformation of programs for fault-tolerance

In this paper we describe how a program constructed for afault-free system can be transformed into afault-tolerant program for execution on a system which is susceptible to failures. A program is described by a set of atomic actions which perform transformations from states to states. We assume that a fault environment is represented by a programF. Interference by the fault environmentF on the execution of a programP can then be described as afault-transformation which transformsP into a program (P). This is proved to be equivalent to the programPP _F , whereP _F is derived fromP andF, and defines the union of the sets of actions ofP andF _P . A recovery transformation transformsP into a program (P) =PR by adding a set ofrecovery actions R, called arecovery program. If the system isfailstop and faults do not affect recovery actions, we have ((P))=(P)R=PP _F R We illustrate this approach to fault-tolerant programming by considering the problem of designing a protocol that guarantees reliable communication from a sender to a receiver in spite of faults in the communication channel between them.     

On parsing two-level grammars

Summary Making use of the fact that two-level grammars (TLGs) may be thought of as finite specification of context-free grammars (CFGs) with infinite sets of productions, known techniques for parsing CFGs are applied to TLGs by first specifying a canonical CFG G — called skeleton grammar — obtained from the cross-reference of the TLG G. Under very natural restrictions it can be shown that for these grammar pairs (G, G) there exists a 1 — 1 correspondence between leftmost derivations in G and leftmost derivations in G. With these results a straightforward parsing algorithm for restricted TLGs is given.     

Real-time recognition of substring repetition and reversal

Real-time multitape Turing machine algorithms are presented for recognizing the languages {wxyxz^*:|w|=r|x,|z| =t|x|} and {wxyx ^R _z ^*:|w|=r|x,|z| =t|x|} for fixedr, s, andt and for string-matching with forced mismatches.This research was supported in part by the National Science Foundation under Grant MCS77-06613 (first author) and the Bat-Sheva Fund (second author). Part of the work was conducted while the second author was at the IBM Thomas J. Watson Research Center, Yorktown Heights, New York. A preliminary report was included in a paper the authors presented at the Seventeenth Annual IEEE Symposium on Foundations of Computer Science, Houston, Texas, October 1976.     

Monte Carlo modelling of electron beam lithography: a scaling law

The theorem of Dimensional Analysis, usually applied to the inference of physical laws, is for the first time applied to the derivation of interpolation curves of numerical data, leading to a simplified dependence on a reduced number of arguments , dimensionless combination of variables. In particular, Monte Carlo modelling of electron beam lithography is considered and the backscattering coefficient addressed, in case of a general substrate layer, in the elastic regime and in the energy range 5 to 100 keV. The many variables involved (electron energy, substrate physical constants and thickness) are demonstrated to ultimately enter in determining through asingle dimensionless parameter ₀. Thus, a scaling law is determined, an important guide in microsystem designing, indicating, if any part of the configuration is modified, how the other parameters should change (or scale) without affecting the result. Finally, a simple law =83 ₀ is shown to account for all variations of the parameters over all substrates of the periodic table.     

Interpretations of open default theories in non-monotonic logics

It is shown that the translation of an open default into a modal formula x(L(x)LM ₁ (x)...LM _m (x)w(x)) gives rise to an embedding of open default systems into non-monotonic logics.     

Gradually intractable problems and nondeterministic log-space lower bounds

The paper places five different problems (thek-pebble game problem, two problems aboutk finite automata, the reachability problem for Petri nets withk tokens, and the teachability problem for graphs whose k-dimensional edge sets are described by Cartesian products ofk factors) into the hierarchyNL _k of problems solvable by nondeterministic Turing machines ink-log₂ n space (and binary tape alphabet, to avoid tape speed-up). The results, when combined with the conjecture thatNL _i contains problems that requireO(n ^k ) deterministic time, show that these problems, while inP for every fixed value ofk, have polynomial deterministic time complexities with the degree of the polynomial growing linearly with the parameterk, and hence are, in this sense, gradually intractable.     

Maintaining the visibility map of spheres while moving the viewpoint on a circle at infinity

We investigate three-dimensional visibility problems for scenes that consist ofn non-intersecting spheres. The viewing point moves on a flightpath that is part of a circle at infinity given by a planeP and a range of angles {(t)¦t[01]} [02]. At timet, the lines of sight are parallel to the ray inP, which starts in the origin ofP and represents the angle(t) (orthographic views of the scene). We give an algorithm that computes the visibility graph at the start of the flight, all time parameters at which the topology of the scene changes, and the corresponding topology changes. The algorithm has running time0(n + k + p) logn), wheren is the number of spheres in the scene;p is the number of transparent topology changes (the number of different scene topologies visible along the flight path, assuming that all spheres are transparent); andk denotes the number of vertices (conflicts) which are in the (transparent) visibility graph at the start and do not disappear during the flight.The second author was supported by the ESPRIT II Basic Research Actions Program, under Contract No. 3075 (project ALCOM).