How to Better Use Expert Advice

This work is concerned with online learning from expert advice. Extensive work on this problem generated numerous expert advice algorithms whose total loss is provably bounded above in terms of the loss incurred by the best expert in hindsight. Such algorithms were devised for various problem variants corresponding to various loss functions. For some loss functions, such as the square, Hellinger and entropy losses, optimal algorithms are known. However, for two of the most widely used loss functions, namely the 0/1 and absolute loss, there are still gaps between the known lower and upper bounds.In this paper we present two new expert advice algorithms and prove for them the best known 0/1 and absolute loss bounds. Given an expert advice algorithm ALG, the goal is to form an upper bound on the regret L _ALG – L* of ALG, where L _ALG is the loss of ALG and L* is the loss of the best expert in hindsight. Typically, regret bounds of a canonical form C · are sought where N is the number of experts and C is a constant. So far, the best known constant for the absolute loss function is C = 2.83, which is achieved by the recent IAWM algorithm of Auer et al. (2002). For the 0/1 loss function no bounds of this canonical form are known and the best known regret bound is , where C ₁ = e – 2 and C ₂ = 2 . This bound is achieved by a P-norm algorithm of Gentile and Littlestone (1999). Our first algorithm is a randomized extension of the guess and double algorithm of Cesa-Bianchi et al. (1997). While the guess and double algorithm achieves a canonical regret bound with C = 3.32, the expected regret of our randomized algorithm is canonically bounded with C = 2.49 for the absolute loss function. The algorithm utilizes one random choice at the start of the game. Like the deterministic guess and double algorithm, a deficiency of our algorithm is that it occasionally restarts itself and therefore forgets what it learned. Our second algorithm does not forget and enjoys the best known asymptotic performance guarantees for both the absolute and 0/1 loss functions. Specifically, in the case of the absolute loss, our algorithm is canonically bounded with C approaching and in the case of the 0/1 loss, with C approaching 3/ . In the 0/1 loss case the algorithm is randomized and the bound is on the expected regret.     

Combining the Perceptron Algorithm with Logarithmic Simulated Annealing

We present results of computational experiments with an extension of the Perceptron algorithm by a special type of simulated annealing. The simulated annealing procedure employs a logarithmic cooling schedule , where is a parameter that depends on the underlying configuration space. For sample sets S of n-dimensional vectors generated by randomly chosen polynomials , we try to approximate the positive and negative examples by linear threshold functions. The approximations are computed by both the classical Perceptron algorithm and our extension with logarithmic cooling schedules. For and , the extension outperforms the classical Perceptron algorithm by about 15% when the sample size is sufficiently large. The parameter was chosen according to estimations of the maximum escape depth from local minima of the associated energy landscape.      

Tracking the Best Disjunction

Littlestone developed a simple deterministic on-line learning algorithm for learning k-literal disjunctions. This algorithm (called ) keeps one weight for each of then variables and does multiplicative updates to its weights. We develop a randomized version of and prove bounds for an adaptation of the algorithm for the case when the disjunction may change over time. In this case a possible target disjunction schedule is a sequence of disjunctions (one per trial) and the shift size is the total number of literals that are added/removed from the disjunctions as one progresses through the sequence.We develop an algorithm that predicts nearly as well as the best disjunction schedule for an arbitrary sequence of examples. This algorithm that allows us to track the predictions of the best disjunction is hardly more complex than the original version. However, the amortized analysis needed for obtaining worst-case mistake bounds requires new techniques. In some cases our lower bounds show that the upper bounds of our algorithm have the right constant in front of the leading term in the mistake bound and almost the right constant in front of the second leading term. Computer experiments support our theoretical findings.     

Two-Parameter Discrete Systems over a Residue Ring and Their Properties

A class of two-parameter discrete systems defined on the ring of class of residues of integers modulo m is studied. All solutions are shown to be periodic, stability conditions (equality of solutions to zero, beginning from a certain instant) and a controllability condition are formulated. Controllability is shown to guarantee stabilizability.     

Estimating the Optimal Margins of Embeddings in Euclidean Half Spaces

Concept classes can canonically be represented by matrices with entries 1 and –1. We use the singular value decomposition of this matrix to determine the optimal margins of embeddings of the concept classes of singletons and of half intervals in homogeneous Euclidean half spaces. For these concept classes the singular value decomposition can be used to construct optimal embeddings and also to prove the corresponding best possible upper bounds on the margin. We show that the optimal margin for embedding n singletons is and that the optimal margin for half intervals over {1,...,n} is . For the upper bounds on the margins we generalize a bound by Forster (2001). We also determine the optimal margin of some concept classes defined by circulant matrices up to a small constant factor, and we discuss the concept classes of monomials to point out limitations of our approach.     

On-line graph coloring of {\mathbb{P}_5}-free graphs

Kierstead et al. (SIAM J Discret Math 8:485–498, 1995) have shown 1 that the competitive function of on-line coloring for -free graphs (i.e., graphs without induced path on 5 vertices) is bounded from above by the exponential function . No nontrivial lower bound was known. In this paper we show the quadratic lower bound . More precisely, we prove that is the exact competitive function for ()-free graphs. In this paper we also prove that 2 - 1 is the competitive function of the best clique covering on-line algorithm for ()-free graphs.     

Stability of the Logistic Population Model with Delayed Environmental Response

The population dynamics model , was considered. For this model with uniform distribution of delays and a _n = 0, nonnegativeness and convexity of the sequence a _k (0 k n) was shown to be the sufficient stability condition. Therefore, there is no need to constrain the reproduction rate and the mean delay .     

Stability of Linear Difference and Differential Inclusions

Any given n×n matrix A is shown to be a restriction, to the A-invariant subspace, of a nonnegative N×N matrix B of spectral radius (B) arbitrarily close to (A). A difference inclusion , where is a compact set of matrices, is asymptotically stable if and only if can be extended to a set of nonnegative matrices B with or . Similar results are derived for differential inclusions.     

Stochastic Finite Learning of the Pattern Languages

The present paper proposes a new learning model—called stochastic finite learning—and shows the whole class of pattern languages to be learnable within this model.This main result is achieved by providing a new and improved average-case analysis of the Lange–Wiehagen (New Generation Computing, 8, 361–370) algorithm learning the class of all pattern languages in the limit from positive data. The complexity measure chosen is the total learning time, i.e., the overall time taken by the algorithm until convergence. The expectation of the total learning time is carefully analyzed and exponentially shrinking tail bounds for it are established for a large class of probability distributions. For every pattern containing k different variables it is shown that Lange and Wiehagen's algorithm possesses an expected total learning time of , where and are two easily computable parameters arising naturally from the underlying probability distributions, and E[] is the expected example string length.Finally, assuming a bit of domain knowledge concerning the underlying class of probability distributions, it is shown how to convert learning in the limit into stochastic finite learning.     

On the k-th root in circular arithmetic

The representation of thek-th root of a complex circular intervalZ={c;r} is considered in this paper. Thek-th root is defined by the circular intervals which include the exact regionZ ^1/k={z:z ^k ∈Z}. Two representations are given: (i) the centered inclusive disks \( \cup \{ c^{{1 \mathord{\left/ {\vphantom {1 k}} \right. \kern-0em} k}} ; \mathop {\max }\limits_{z \in Z} |z^{{1 \mathord{\left/ {\vphantom {1 k}} \right. \kern-0em} k}} - c^{{1 \mathord{\left/ {\vphantom {1 k}} \right. \kern-0em} k}} |\} \) and (ii) the diametrical inclusive disks with the diameter which is equal to the diameter of the regionZ ^1/k.     

Maximum concurrent flows and minimum cuts

In many applications, we need to find a minimum cost partition of a network separating a given pair of nodes. A classical example is the Max-Flow Min-Cut Theorem, where the cost of the partition is defined to be the sum of capacities of arcs connecting the two parts. Other similar concepts such as minimum weighted sparsest cut and flux cut have also been introduced. There is always a cost associated with a cut, and we always seek the min-cost cut separating a given pair of nodes. A natural generalization from the separation of a given pair is to find all minimum cost cuts separating all pairs of nodes, with arbitrary costs associated with all 2^n–1 — 1 cuts. In the present paper, we show thatn — 1 minimum cost cuts are always sufficient to separate all pairs of nodes.A further generalization is to considerk-way partitions rather than two-way partitions. An interesting relationship exists betweenk-way partitions, the multicommodity flow problem, and the minimum weighted sparsest cut. Namely, if the staturated arcs in a multicommodity flow problem form ak-way partition (k 4), then thek-way partition contains a two-way partition. This two-way partition is the minimum weight sparsest cut.This work is supported in part by the NSF under Grant MIP-8700767 and micro program under Grants 506205 and 506215, Intergraph, and Data General.     

Computation of Battle–Lemarie Wavelets Using an FFT-Based Algorithm

Battle and Lemarie derived independently wavelets by orthonormalizing B-splines. The scaling function _m (t) corresponding to Battle–Lemarie's wavelet _m (t) is given by , where B _m(t) is the mth-order central B-spline and the coefficients _{m, k} satisfy . In this paper, we propose an FFT-based algorithm for computing the expansion coefficients _{m, k} and the two-scale relations of the scaling functions and wavelets. The algorithm is very simple and it can be easily implemented. Moreover, the expansion coefficients can be efficiently and accurately obtained via multiple sets of FFT computations. The computational approach presented in this paper here is noniterative and is more efficient than the matrix approach recently proposed in the literature.     

Onn-column 0, 1-matrices with allk-projections surjective

This paper presents a new approach to construct a smalln-column 0, 1-matrix for two given integersn andk(k, such that everyk-column projection contains all 2 ^k possible row vectors, namely surjective on {0, 1} ^k . The number of the matrix's rows does not exceed . This approach has considerable advantage for smallk and practical sizes ofn. It can be applied to the test generation of VLSI circuits, the design of fault tolerant systems and other fields.     

Zur Charakterisierung und Berechnung von symmetrischen Kubaturformeln

The paper is concerned with the characterization and calculation of symmetric cubature formulae of degree 2k?1 for two-dimensional product-functionals. The number of knots of the cubature formulae satisfies the following relation: $$\frac{{k(k + 1)}}{2} + \left[ {\frac{k}{2}} \right] \leqslant r \leqslant \frac{{k(k + 1)}}{2} + \left[ {\frac{k}{2}} \right] + 1.$$ The systems of non-linear equations involved are either solved exactly or all solutions are computed with any precision using a program-package for symbolic and algebraic calculations.     

Conditions for Instability of One Class of Nonlinear Nonautomomous Dynamic Systems

Consideration was given to the conditions for instability of the equilibrium states of a nonlinear nonautonomous dynamic systems obeying an ordinary vector differential equation of arbitrary order whose right-hand side satisfies the following conditions: (i) for any t 0, div 0$$ " align="middle" border="0"> almost everywhere on the set H that is a neighborhood of the equilibrium point of the system and (ii) at any point . The equilibrium states of such systems can be both stable and unstable. For one class of these systems, sufficient instability conditions were given, which enables one to carry out studies using only the information about the right-hand side of the system.     

Spigot Algorithm and Reliable Computation of Natural Logarithm

The spigot approach used in the previous paper (Reliable Computing 7 (3) (2001), pp. 247–273) for root computation is now applied to natural logarithms. The logarithm ln Q with Q , Q > 1 is decomposed into a sum of two addends k ₁× ln Q ₁+k ₂× ln Q ₂ with k ₁, k ₂ , then each of them is computed by the spigot algorithm and summation is carried out using integer arithmetic. The whole procedure is not literally a spigot algorithm, but advantages are the same: only integer arithmetic is needed whereas arbitrary accuracy is achieved and absolute reliability is guaranteed. The concrete procedure based on the decomposition with p, q ( – {0}), p < q is simple and ready for implementation. In addition to the mentioned paper, means for determining an upper bound for the biggest integer occurring in the process of spigot computing are now provided, which is essential for the reliability of machine computation.     

Exact optimal grillage layouts — Part I: Combinations of free and simply supported edges

In the first instalment of this three-part study, a comprehensive treatment of analytically derived, exact optimal grillage layouts for combinations of simply supported and free edges is given. In part two, grillages with combinations of simply supported, clamped and free edges will be considered.Notation k constant in specific cost function - M beam bending moment - r radius of circular edge - R ⁺,R ^–,S ⁺,S ^–,T optimal regions - x, x _j coordinate along a beam (j) - slope of the adjoint deflection at pointD in directionDA - t, v coordinates along the free edge - adjoint deflection - angle between long beams and free edge - angle between free and simply supported edges - curvature of the adjoint deflection - , angles for layouts with circular edge - total weight (cost) of grillage - coordinate along a beam in anR ⁺ region - distance defined in Fig. 3     

Model of embedded spaces: Peculiarities of dynamical embedding and geometry

The paper is devoted to construction of a geometry of 4D space-time arising from geometrization of classical electrodynamics and associated with the Model of Embedding Spaces (MES). The basic premise of the model is the assumption that each element of matter has an eigenspace, and the spacetime of the Universe is a result of mutual dynamic embedding of these spaces, such that the magnitude of partial embedding is determined by the relative Lagrangian density of the element interactions. It is shown that the required geometry of the MES is qualified as an inhomogeneous Finsler geometry (the metric is an arbitrary function of $\dot x$ ); the basic variant of the geometry (the unique law of parallel shift of metric and non-metric tensor quantities) is obtained as a differential generalization of the Riemannian geometry by the formal replacement ${\partial \mathord{\left/ {\vphantom {\partial {\partial x^i \mapsto }}} \right. \kern-0em} {\partial x^i \mapsto }}{\partial \mathord{\left/ {\vphantom {\partial {\partial x^i }}} \right. \kern-0em} {\partial x^i }} + {{2u^k \partial ^2 } \mathord{\left/ {\vphantom {{2u^k \partial ^2 } {\partial x^{\left[ i \right.} \partial u^{\left. k \right]} }}} \right. \kern-0em} {\partial x^{\left[ i \right.} \partial u^{\left. k \right]} }}$ , u = dx/ds; and the dynamic embedding provides linearity of this generalization. Additionally, some features ofMES cosmology, which can be caused by the natural electromagnetism of the Universe, are discussed.     

Attractive energy contribution to nanoconfined fluids behavior: the normal pressure tensor

The aim of our research is to demonstrate the role of attractive intermolecular potential energy on normal pressure tensor of confined molecular fluids inside nanoslit pores of two structureless purely repulsive parallel walls in xy plane at z = 0 and z = H, in equilibrium with a bulk homogeneous fluid at the same temperature and at a uniform density. To achieve this we have derived the perturbation theory version of the normal pressure tensor of confined inhomogeneous fluids in nanoslit pores: