On the Axioms of Scale Space Theory

We consider alternative scale space representations beyond the well-established Gaussian case that satisfy all reasonable axioms. One of these turns out to be subject to a first order pseudo partial differential equation equivalent to the Laplace equation on the upper half plane {(x, s) ^d × | s > 0}. We investigate this so-called Poisson scale space and show that it is indeed a viable alternative to Gaussian scale space. Poisson and Gaussian scale space are related via a one-parameter class of operationally well-defined intermediate representations generated by a fractional power of (minus) the spatial Laplace operator.     

On the Complexity of Function Learning

The majority of results in computational learning theory are concerned with concept learning, i.e. with the special case of function learning for classes of functions with range {0, 1}. Much less is known about the theory of learning functions with a larger range such as or . In particular relatively few results exist about the general structure of common models for function learning, and there are only very few nontrivial function classes for which positive learning results have been exhibited in any of these models.We introduce in this paper the notion of a binary branching adversary tree for function learning, which allows us to give a somewhat surprising equivalent characterization of the optimal learning cost for learning a class of real-valued functions (in terms of a max-min definition which does not involve any learning model).Another general structural result of this paper relates the cost for learning a union of function classes to the learning costs for the individual function classes.Furthermore, we exhibit an efficient learning algorithm for learning convex piecewise linear functions from ^d into . Previously, the class of linear functions from ^d into was the only class of functions with multidimensional domain that was known to be learnable within the rigorous framework of a formal model for online learning.Finally we give a sufficient condition for an arbitrary class of functions from into that allows us to learn the class of all functions that can be written as the pointwise maximum ofk functions from . This allows us to exhibit a number of further nontrivial classes of functions from into for which there exist efficient learning algorithms.     

Constructing sets of functions which have a givenF-cardinality

The termF-cardinality of (=F-card()) is introduced whereF: ^{n n} is a partial function and is a set of partial functionsf: ^{n n} . TheF-cardinality yields a lower bound for the worst-case complexity of computingF if only functionsf can be evaluated by the underlying abstract automaton without conditional jumps. This complexity bound isindependent from the oracles available for the abstract machine. Thus it is shown that any automaton which can only apply the four basic arithmetic operations needs (n logn) worst-case time to sortn numbers; this result is even true if conditional jumps witharbitrary conditions are possible. The main result of this paper is the following: Given a total functionF: ^{n n} and a natural numberk, it is almost always possible to construct a set such that itsF-cardinality has the valuek; in addition, can be required to be closed under composition of functionsf,g . Moreover, ifF is continuous, then consists of continuous functions.     

Sets with small generalized Kolmogorov complexity

Summary We study the class of sets with small generalized Kolmogorov complexity. The following results are established: 1. A set has small generalized Kolmogorov complexity if and only if it is semi-isomorphic to a tally set. 2. The class of sets with small generalized Kolmogorov complexity is properly included in the class of self-p-printable sets. 3. The class of self-p-printable sets is properly included in the class of sets with selfproducible circuits. 4. A set S has self-producible circuits if and only if there is a tally set T such that P(T)=P(S). 5. If a set S has self-producible circuits, then NP(S)=NP _B(S), where NP _B( ) is the restriction of NP( ) studied by Book, Long, and Selman [4]. 6. If a set S is such that NP(S) =NP _B(S), then NP(S) P(SSAT).     

Generating Boolean \mu-expressions

In this paper, we consider the class of Boolean -functions, which are the Boolean functions definable by -expressions (Boolean expressions in which no variable occurs more than once). We present an algorithm which transforms a Boolean formulaE into an equivalent -expression-if possible-in time linear in E times , where E is the size ofE andn _m is the number of variables that occur more than once inE. As an application, we obtain a polynomial time algorithm for Mundici's problem of recognizing -functions fromk-formulas [17]. Furthermore, we show that recognizing Boolean -functions is co-NP-complete for functions essentially dependent on all variables and we give a bound close to co-NP for the general case.     

Dynamic half-space range reporting and its applications

We consider the half-space range-reporting problem: Given a setS ofn points in ^d, preprocess it into a data structure, so that, given a query half-space , allk points ofS can be reported efficiently. We extend previously known static solutions to dynamic ones, supporting insertions and deletions of points ofS. For a given parameterm,n m n ^d/2 and an arbitrarily small positive constant , we achieveO(m ¹⁺) space and preprocessing time, O((n/m ^d/2 logn+k) query time, and O(m¹⁺n) amortized update time (d 3). We present, among others, the following applications: an O(n¹⁺)-time algorithm for computing convex layers in ³, and an output sensitive algorithm for computing a level in an arrangements of planes in ³, whose time complexity is O((b+n) n, whereb is the size of the level.Work by the first author has been supported by National Science Foundation Grant CCR-91-06514. A preliminary version of this paper appeared in Agarwalet al. [2], which also contains the results of [20] on dynamic bichromatic closest pair and minimum spanning trees.     

Separating complexity classes related to bounded alternating ω-branching programs

We develop a theory of communication within branching programs that provides exponential lower bounds on the size of branching programs that are bounded alternating. Our theory is based on the algebraic concept of -branching programs, : , a semiring homomorphism, that generalizes ordinary branching programs, -branching programs [M2] andMOD _p-branching programs [DKMW].Due to certain exponential lower and polynomial upper bounds on the size of bounded alternating -branching programs we are able to separate the corresponding complexity classesN _ba ,co-N _{ba ba} , andMOD _p - _ba ,p prime, from each other, and from that classes corresponding to oblivious linear length-bounded branching programs investigated in the past.     

Three ∑ P 2 -complete problems in computational learning theory

The consistency problem associated with a concept classC is to determine, given two setsA andB of examples, whether there exists a conceptc inC such that eachx inA is a positive example ofc and eachy inB is a negative example ofc. We explore in this paper the following intuition: for a concept classC, if the membership problem of determining whether a given example is positive for a concept isNP-complete, then the corresponding consistency problem is likely to be _P ² -complete. To support this intuition, we prove that the following three consistency problems for concept classes of patterns, graphs and generalized Boolean formulas, whose membership problems are known to beNP-complete, are _P ² -complete: (a) given two setsA andB of strings, determine whether there exists a patternp such that every string inA is in the languageL(p) and every string inB is not in the languageL(p); (b) given two setsA andB of graphs, determine whether there exists a graphG such that every graph inA is isomorphic to a subgraph ofG and every graph inB is not isomorphic to any subgraph ofG; and (c) given two setsA andB of Boolean formulas, determine whether there exists a 3-CNF Boolean formula such that for every A, is satisfiable and for every B, is not satisfiable. These results suggest that consistendy problems in machine learning are natural candidates for _P ² -complete problems if the corresponding membership problems are known to beNP-complete.In addition, we prove that the corresponding prediction problems for concept classes of polynomial-time nondeterministic Turing machines, nondeterministic Boolean circuits, generalized Boolean formulas, patterns and graphs are prediction-complete for the classR _NP of all concept classes whose membership problems are inNP.     

Extremal Relations between Additive Loss Functions and the Kolmogorov Complexity

Conditions are presented under which the maximum of the Kolmogorov complexity (algorithmic entropy) K(₁... _N ) is attained, given the cost f( _i ) of a message ₁... _N . Various extremal relations between the message cost and the Kolmogorov complexity are also considered; in particular, the minimization problem for the function f( _i ) – K(₁... _N ) is studied. Here, is a parameter, called the temperature by analogy with thermodynamics. We also study domains of small variation of this function.     

States and the free orthogonality monoid

Let (X, #) be an orthogonality space such that the lattice C(X, #) of closed subsets of (X, #) is orthomodular and let (, ) denote the free orthogonality monoid over (X, #). Let C₀(, ) be the subset of C(, ), consisting of all closures of bounded orthogonal sets. We show that C₀(, ) is a suborthomodular lattice of C(, ) and we provide a necessary and sufficient condition for C₀(, ) to carry a full set of dispersion free states.The work of the second author on this paper was supported by National Science Foundation Grant GP-9005.     

A Fat Boundary Method for the Poisson Problem in a Domain with Holes

We consider the Poisson equation with Dirichlet boundary conditions, in a domain , where ⁿ , and B is a collection of smooth open subsets (typically balls). The objective is to split the initial problem into two parts: a problem set in the whole domain , for which fast solvers can be used, and local subproblems set in narrow domains around the connected components of B, which can be solved in a fully parallel way. We shall present here a method based on a multi-domain formulation of the initial problem, which leads to a fixed point algorithm. The convergence of the algorithm is established, under some conditions on a relaxation parameter . The dependence of the convergence interval for upon the geometry is investigated. Some 2D computations based on a finite element discretization of both global and local problems are presented.     

Sparse Finite Elements for Stochastic Elliptic Problems – Higher Order Moments

We define the higher order moments associated to the stochastic solution of an elliptic BVP in D^d with stochastic input data. We prove that the k-th moment solves a deterministic problem in D^kdk, for which we discuss well-posedness and regularity. We discretize the deterministic k-th moment problem using sparse grids and, exploiting a spline wavelet basis, we propose an efficient algorithm, of logarithmic-linear complexity, for solving the resulting system.Supported in part under the IHP network Breaking Complexity of the EC (contract number HPRN-CT-2002-00286) with support by the Swiss Federal Office for Science and Education under grant No. BBW 02.0418.     

A time-varying Beurling-Lax theorem and a related interpolation problem

This article concerns linear time-varying interpretations of the Beurling-Lax-Ball-Helton theorem and of Sarason's interpolation problem. The former characterizes shift-invariantH ₂ (Krein) subspaces. Unilateral shift invariance reflects both causality and time invariance. Removing the stationarity requirement, a generalized theorem provides a characterization of certain causal subspace families M_t L₂(t, ), t . Sarason's interpolation problem is interpreted here as a search for a (close to) minimal induced norm system, given causal input-output specifications. The Beurling-Lax theorem helps in identifying admissible specification classes. The problem is then reduced to and solved in terms of a linear time-varying Nehari problem. Technically, developments are based on timedomain, state-space methods.This research was supported by the National Science Foundation and by the Army Research Office.     

On characterizations of recursively enumerable languages

Summary Geffert has shown that earch recursively enumerable languageL over can be expressed in the formL{h(x) ^–1 g(x)x in ⁺} ^* where is an alphabet andg, h is a pair of morphisms. Our purpose is to give a simple proof for Geffert's result and then sharpen it into the form where both of the morphisms are nonerasing. In our method we modify constructions used in a representation of recursively enumerable languages in terms of equality sets and in a characterization of simple transducers in terms of morphisms. As direct consequences, we get the undecidability of the Post correspondence problem and various representations ofL. For instance,L =(L ₀) ^* whereL ₀ is a minimal linear language and is the Dyck reductiona, A.     

An Interval Linear Tolerance Problem

This paper is concerned with a tolerance problem for an interval linear system A x = b requiring inner estimation of the admissible solution set {x ⁿ | (A A)(Ax b)} formed by vectors x for which the product Ax remains within b for any possible A A. Methods for verifying the emptiness and nonemptiness of admissible solution sets are developed. Formulas for the dimensions of the interval solution of a tolerance problem with known center are derived.     

On the Quadratic Convergence of the Levenberg-Marquardt Method without Nonsingularity Assumption

Recently, Yamashita and Fukushima [11] established an interesting quadratic convergence result for the Levenberg-Marquardt method without the nonsingularity assumption. This paper extends the result of Yamashita and Fukushima by using _k=||F(x_k)||, where [1,2], instead of _k=||F(x_k)||² as the Levenberg-Marquardt parameter. If ||F(x)|| provides a local error bound for the system of nonlinear equations F(x)=0, it is shown that the sequence {x_k} generated by the new method converges to a solution quadratically, which is stronger than dist(x_k,X^*)0 given by Yamashita and Fukushima. Numerical results show that the method performs well for singular problems.     

A Lie-Group Formulation of Kinematics and Dynamics of Constrained MBS and Its Application to Analytical Mechanics

A Lie-group formulation for the kinematics and dynamics ofholonomic constrained mechanical systems (CMS) is presented. The kinematics ofrigid multibody systems (MBS) is described in terms of the screw system of theMBS. Using Lie-algebraic properties of screw algebra, isomorphicto se(3), allows a purely algebraic derivation of the Lagrangian motion equations. As such the Lie-group SE(3) ... SE(3) (n copies) is theambient space of a MBS consisting of n rigid bodies. Any parameterizationof the ambient space corresponds to a chart on the MBS configuration space ⁿ. The key to combine differential geometric and Lie-algebraic approaches is the existence of kinematic basic functions whichare push forward maps from the tangent bundle Tⁿ to the Lie-algebra of the ambient space.MBS with kinematic loops are CMS subject to holonomic constraints, holonomicCMS. Constraint equations are formulated on the ambient space based on achart on ⁿ. Explicitly considering the Lie-algebraicstructure of se(3) as semidirect sum of the algebra oftranslations and rotations enables to reduce the number of holonomicconstraints for cut joints. It is shown that third order Lie-brackets are sufficient to obtain any subalgebra of se(3).Differential geometric aspects of the kinematics of MBS with open and closedkinematic chains are considered. The distribution in any configuration andthe subalgebra generated by the screw system of a mechanism is the key todetect singularities and to analyze the structure of the set of singularpoints.     

Learning reliably and with one-sided error

Reliable and probably useful learning, proposed by Rivest and Sloan, is a variant of probably approximately correct learning. In this model the hypothesis must never misclassify an instance but is allowed to answer I don't know with a low probability. We derive upper and lower bounds for the sample complexity of reliable and probably useful learning in terms of the combinatorial characteristics of the concept class to be learned. This is done by reducing reliable and probably useful learning to learning with one-sided error. The bounds also hold for a slightly weaker model that allows the learner to output with a low probability a hypothesis that makes misclassifications. We see that in these models learning with one oracle is more difficult than learning with two oracles. Our results imply that monotone Boolean conjunctions or disjunctions cannot be learned reliably and probably usefully from a polynomial number of examples. Rectangles in ⁿ forn 2 cannot be learned from any finite number of examples.A preliminary version of this paper appeared under the title Reliable and useful learning inProceedings of the 2nd Annual Workshop on Computational Learning Theory, Morgan Kaufmann, San Mateo, CA, 1989, pp. 365–380. This work was supported by the Academy of Finland.     

On relativizations with restricted number of accesses to the oracle set

Conflicting relativizations, also known as Baker-Gill-Solovay phenomena, are common place in complexity theory. However, Book, Long, and Selman (SIAM J. Comput.,13 (1984), 461–487) have shown thatP(A) = NP _B (A) for all oraclesA if and only ifP = NP, whereNP _B (A) denotes the class of languages accepted by nondeterministic machines which possess only a polynomial number of query strings in the computation tree. It is shown in this paper that any superpolynomial bound on the number of queries already yields the BGS phenomenon. Similar results hold for theNP = co-NP andNPC = NP questions. A second objective of this paper is to point out a technique of Hartmanis with which to achieve oracle constructions by using the Kolmogorov complexity. This relatively new technique seems to be adequate for obtaining separation results for complexity classes which are not enumerable.     

Some Ergodic Results on Stochastic Iterative Discrete Events Systems

This paper deals with the asymptotic behavior of the stochastic dynamics of discrete event systems. In this paper we focus on a wide class of models arising in several fields and particularly in computer science. This class of models may be characterized by stochastic recurrence equations in ^K of the form T(n+1) = _n+1(T(n)) where _n is a random operator monotone and 1—linear. We establish that the behaviour of the extremas of the process T(n) are linear. The results are an application of the sub-additive ergodic theorem of Kingman. We also give some stability properties of such sequences and a simple method of estimating the limit points.