Converting non-parametric distance-based classification to anytime algorithms

For many real world problems we must perform classification under widely varying amounts of computational resources. For example, if asked to classify an instance taken from a bursty stream, we may have anywhere from several milliseconds to several minutes to return a class prediction. For such problems an anytime algorithm may be especially useful. In this work we show how we convert the ubiquitous nearest neighbor classifier into an anytime algorithm that can produce an instant classification, or if given the luxury of additional time, can continue computations to increase classification accuracy. We demonstrate the utility of our approach with a comprehensive set of experiments on data from diverse domains. We further show the utility of our work with two deployed applications, in classifying and counting fish, and in classifying insects.     

Harnessing the strengths of anytime algorithms for constant data streams

Anytime algorithms have been proposed for many different applications, e.g., in data mining. Their strengths are the ability to first provide a result after a very short initialization and second to improve their result with additional time. Therefore, anytime algorithms have so far been used when the available processing time varies, e.g., on varying data streams. In this paper we propose to employ anytime algorithms on constant data streams, i.e., for tasks with constant time allowance. We introduce two approaches that harness the strengths of anytime algorithms on constant data streams and thereby improve the over all quality of the result with respect to the corresponding budget algorithm. We derive formulas for the expected performance gain and demonstrate the effectiveness of our novel approaches using existing anytime algorithms on benchmark data sets.     

Parameter estimation algorithms and robust identifiability

Parameter estimation algorithms are obtained for a class of finite-dimensional systems. The algorithms are presented in a completely deterministic setting and their convergence properties are investigated by means of a fixed-point theorem. The concept of robust identifiability is introduced. The invertibility of a certain matrix is shown to be a necessary condition for robust identifiability. Some numerical results are presented.     

Designing robust volunteer-based evolutionary algorithms

This paper tackles the design of scalable and fault-tolerant evolutionary algorithms computed on volunteer platforms. These platforms aggregate computational resources from contributors all around the world. Given that resources may join the system only for a limited period of time, the challenge of a volunteer-based evolutionary algorithm is to take advantage of a large amount of computational power that in turn is volatile. The paper analyzes first the speed of convergence of massively parallel evolutionary algorithms. Then, it provides some guidance about how to design efficient policies to overcome the algorithmic loss of quality when the system undergoes high rates of transient failures, i.e. computers fail only for a limited period of time and then become available again. In order to provide empirical evidence, experiments were conducted for two well-known problems which require large population sizes to be solved, the first based on a genetic algorithm and the second on genetic programming. Results show that, in general, evolutionary algorithms undergo a graceful degradation under the stress of losing computing nodes. Additionally, new available nodes can also contribute to improving the search process. Despite losing up to 90 % of the initial computing resources, volunteer-based evolutionary algorithms can find the same solutions in a failure-prone as in a failure-free run.     

On robust online scheduling algorithms

While standard parallel machine scheduling is concerned with good assignments of jobs to machines, we aim to understand how the quality of an assignment is affected if the jobs’ processing times are perturbed and therefore turn out to be longer (or shorter) than declared. We focus on online scheduling with perturbations occurring at any time, such as in railway systems when trains are late. For a variety of conditions on the severity of perturbations, we present bounds on the worst case ratio of two makespans. For the first makespan, we let the online algorithm assign jobs to machines, based on the non-perturbed processing times. We compute the makespan by replacing each job’s processing time with its perturbed version while still sticking to the computed assignment. The second is an optimal offline solution for the perturbed processing times. The deviation of this ratio from the competitive ratio of the online algorithm tells us about the “price of perturbations”. We analyze this setting for Graham’s algorithm, and among other bounds show a competitive ratio of 2 for perturbations decreasing the processing time of a job arbitrarily, and a competitive ratio of less than 2.5 for perturbations doubling the processing time of a job. We complement these results by providing lower bounds for any online algorithm in this setting. Finally, we propose a risk-aware online algorithm tailored for the possible bounded increase of the processing time of one job, and we show that this algorithm can be worse than Graham’s algorithm in some cases.     

A paradigm for robust geometric algorithms

This paper explores a paradigm for producing geometrical algorithms in which logical decisions that depend on finite-precision numerical calculation cannot lead to failure. It applies this paradigm to the task of intersecting two convex polyhedral objects. A key tool in this work is a method of perturbing embedding polyhedra in ways consistent with their topology.Work on this paper was supported in part by NSF Grant DMC-86-17355, NSF Grant DMS-87-02070, ONR Grant N00014-86-0281, ONR Grant N00014-88-0591, and the U.S. Army Research Office through MSI, Cornell University.     

A paradigm for robust geometric algorithms

This paper explores a paradigm for producing geometrical algorithms in which logical decisions that depend on finite-precision numerical calculation cannot lead to failure. It applies this paradigm to the task of intersecting two convex polyhedral objects. A key tool in this work is a method of perturbing embedding polyhedra in ways consistent with their topology.     

Optimal algorithms theory for robust estimation and prediction

This paper deals with the theory of optimal algorithms for problems which cannot be solved exactly. The theory developed allows for the derivation of new and interesting results in parameter estimation and in time series prediction in situations where no reliable statistical hypothesis can be made on the functions and modeling errors involved, but only a bound on them is known, in particular, the derivation of computationally simple optimal algorithms for these two problems is investigated. The practical effectiveness of the algorithms obtained is illustrated by several numerical examples.     

Fast and robust fixed-point algorithms for independent componentanalysis

Independent component analysis (ICA) is a statistical method for transforming an observed multidimensional random vector into components that are statistically as independent from each other as possible. We use a combination of two different approaches for linear ICA: Comon's information theoretic approach and the projection pursuit approach. Using maximum entropy approximations of differential entropy, we introduce a family of new contrast functions for ICA. These contrast functions enable both the estimation of the whole decomposition by minimizing mutual information, and estimation of individual independent components as projection pursuit directions. The statistical properties of the estimators based on such contrast functions are analyzed under the assumption of the linear mixture model, and it is shown how to choose contrast functions that are robust and/or of minimum variance. Finally, we introduce simple fixed-point algorithms for practical optimization of the contrast functions     

Efficient parallel algorithms can be made robust

Summary The efficient parallel algorithms proposed for many fundamental problems, such as list ranking, integer sorting and computing preorder numberings on trees, are very sensitive to processor failures. The requirement of efficiency (commonly formalized usingParallel-timexProcessors as a cost measure) has led to the design of highly tuned PRAM algorithms which, given the additional constraint of simple processor failures, unfortunately become inefficient or even incorrect. We propose a new notion ofrobustness, that combines efficiency with fault tolerance. For the common case of fail-stop errors, we develop a general and easy to implement technique to make robust many efficient parallel algorithms, e.g., algorithms for all the problems listed above. More specifically,for any dynamic pattern of fail-stop errors on a CRCW PRAMwith at least one surviving processor, our method increases the original algorithm cost by at most a log² multiplicative factor. Our technique is based on a robust solution of the problem ofWrite-All, i.e., usingP processors, write 1's in all locations of anN-sized array. In addition we show that at least a log/log log multiplicative overhead will be incurred for certain patterns of failures by any algorithm that implements robust solutions toWrite-All withP=N. However, by exploiting parallel slackness, we obtain an optimal cost algorithm when Paris C. Kanellakis is a professor of computer science at Brown University. His primary research interests are in the application of logic to computer science, such as high-level query languages for database systems, parallel evaluation of logic programs, and type inference for programming languages. He has published many articles on these subjects and is the author of the chapter Elements of Relational Database Theory in theHandbook of Theoretical Computer Science (Elsevier, 1990). He has also contributed to the theory of distributed computing and to combinatorial optimization. Alex Allister Shvartsman is an engineer at Digital Equipment Corporation. His professional interests include design and development of efficient distributed systems, distributed resource management, and theoretical foundations of fault-tolerant parallel computation. At Digital he architected and managed the development of distributed control systems that automated several of Digital's manufacturing processes. He is currently on an academic leave at Brown University.An extended abstract of a part of this work appears as: Kanellakis and Shvartsman (1989) in the Proceedings of the 8th ACM Symposium on Principles of Distributed Computing, Edmonton 1989This author was supported by NSF grant IRI-8617344, an Alfred P. Sloan fellowship and ONR grant N00014-83-K-0146 ARPA Order No. 4786This author was supported by DEC GEEP Doctoral program and ONR grant N00014-91-J-1613     

Efficient algorithms for robust estimation of relative translation

One of the key challenges for structure from motion systems in order to make them robust to failure is the ability to handle outliers among the correspondences. In this paper we present two new algorithms that find the optimal solution in the presence of outliers when the camera undergoes a pure translation. The first algorithm has polynomial-time computational complexity, independently of the amount of outliers. The second algorithm does not offer such a theoretical complexity guarantee, but we demonstrate that it is magnitudes faster in practice. No random sampling approaches such as RANSAC are guaranteed to find an optimal solution, while our two methods do. We evaluate and compare the algorithms both on synthetic and real experiments. We also embed the algorithms in a larger system, where we optimize for the rotation angle as well (the rotation axis is measured by other means). The experiments show that for problems with a large amount of outliers, the RANSAC estimates may deteriorate compared to our optimal methods.     

Review of user parameter-free robust adaptive beamforming algorithms

This paper provides a comprehensive review of user parameter-free robust adaptive beamforming algorithms. We present the ridge regression Capon beamformers (RRCBs), the mid-way (MW) algorithm, and the convex combination (CC) as well as the general linear combination (GLC) approaches. The purpose of these methods is to mitigate the effect of small sample size and steering vector errors on the standard Capon beamformer (SCB). We also present sparsity based iterative beamforming algorithms, namely the iterative adaptive approach (IAA), maximum likelihood based IAA (referred to as IAA-ML) and M-SBL (multi-snapshot sparse Bayesian learning), which exploit sparsity to estimate the signal parameters. We provide a thorough evaluation of these beamforming methods in terms of power and spatial spectrum estimation accuracies, output signal-to-interference-plus-noise ratio (SINR) and resolution under various scenarios including coherent, non-coherent and distributed sources, steering vector mismatches, snapshot limitations and low signal-to-noise ratio (SNR) levels. Furthermore, we discuss the computational complexities of the algorithms and provide insights into which algorithm is the best choice under which circumstances.     

Genetic algorithms with a robust solution searching scheme

A large fraction of studies on genetic algorithms (GAs) emphasize finding a globally optimal solution. Some other investigations have also been made for detecting multiple solutions. If a global optimal solution is very sensitive to noise or perturbations in the environment then there may be cases where it is not good to use this solution. In this paper, we propose a new scheme which extends the application of GAs to domains that require the discovery of robust solutions. Perturbations are given to the phenotypic features while evaluating the functional value of individuals, thereby reducing the chance of selecting sharp peaks (i.e., brittle solutions). A mathematical model for this scheme is also developed. Guidelines to determine the amount of perturbation to be added is given. We also suggest a scheme for detecting multiple robust solutions. The effectiveness of the scheme is demonstrated by solving different one- and two-dimensional functions having broad and sharp peaks     

Classification based on fuzzy robust PCA algorithms and similarity classifier

In this article, classification method is proposed where data is first preprocessed using fuzzy robust principal component analysis (FRPCA) algorithms to obtain data in a more feasible form. After this we use similarity classifier for the classification. We tested this procedure for breast cancer data and liver-disorder data. The results were quite promising and better classification accuracy was achieved than using traditional PCA and similarity classifier. Fuzzy robust principal component analysis algorithms seem to have the effect that they project these data sets in a more feasible form, and together with similarity classifier classification on accuracy of 70.25% was achieved with liver-disorder data and 98.19% accuracy was achieved with breast cancer data. Compared to the results achieved with traditional PCA and similarity classifier about 4% higher accuracy was achieved with liver-disorder data and about 0.5% higher accuracy was achieved with breast cancer data.     

Generating robust and flexible job shop schedules using genetic algorithms

The problem of finding robust or flexible solutions for scheduling problems is of utmost importance for real-world applications as they operate in dynamic environments. In such environments, it is often necessary to reschedule an existing plan due to failures (e.g., machine breakdowns, sickness of employees, deliveries getting delayed, etc.). Thus, a robust or flexible solution may be more valuable than an optimal solution that does not allow easy modifications. This paper considers the issue of robust and flexible solutions for job shop scheduling problems. A robustness measure is defined and its properties are investigated. Through experiments, it is shown that using a genetic algorithm it is possible to find robust and flexible schedules with a low makespan. These schedules are demonstrated to perform significantly better in rescheduling after a breakdown than ordinary schedules. The rescheduling performance of the schedules generated by minimizing the robustness measure is compared with the performance of another robust scheduling method taken from literature, and found to outperform this method in many cases.     

无线传感器网络中的数据传输精简算法

在部署无线传感器网络的相关应用中,由于无线带宽、计算能力、电池能源和意外干扰等限制,通讯环境十分严峻。为了能减少数据传输量并较为精确地由源端传感器向汇聚节点（sink）传输数据,已有方法提出只向sink节点传输无法预测的数据。然而,很少有算法研究在这种严峻的环境中,丢包对数据精简的影响。基于线性预测模型和Heartbeat机制提出LRPH算法来抵制丢包带来的影响,并且及时监测传感器是否故障。另外,提出LRSH算法来优化LRPH,减少冗余信息。实验结果表明LRPH方式可以在一定的误差阈值内,通过只传输4.15%的数据来预测所有的数据。而LRSH只需要3.63%的传输。同已有的一些方法相比,这两种算法都可以在条件严峻的通讯环境下,有效地抑制丢包带来的影响。     

Constrained robustness analysis by randomized algorithms

This paper shows that many robust control problems can be formulated as constrained optimization problems and can be tackled by using randomized algorithms. Two different approaches in searching reliable solutions to robustness analysis problems under constraints are proposed, and the minimum computational efforts for achieving certain reliability and accuracy are investigated and bounds for sample size are derived. Moreover, the existing order statistics distribution theory is extended to the general case in which the distribution of population is not assumed to be continuous and the order statistics is associated with certain constraints     

Competitive randomized algorithms for nonuniform problems

Competitive analysis is concerned with comparing the performance of on-line algorithms with that of optimal off-line algorithms. In some cases randomization can lead to algorithms with improved performance ratios on worst-case sequences. In this paper we present new randomized on-line algorithms for snoopy caching and the spin-block problem. These algorithms achieve competitive ratios approachinge/(e–1) 1.58 against an oblivious adversary. These ratios are optimal and are a surprising improvement over the best possible ratio in the deterministic case, which is 2. We also consider the situation when the request sequences for these problems are generated according to an unknown probability distribution. In this case we show that deterministic algorithms that adapt to the observed request statistics also have competitive factors approachinge/(e–1). Finally, we obtain randomized algorithms for the 2-server problem on a class of isosceles triangles. These algorithms are optimal against an oblivious adversary and have competitive ratios that approache/(e–1). This compares with the ratio of 3/2 that can be achieved on an equilateral triangle.Supported in part by the Center for Discrete Mathematics and Theoretical Computer Science (DIMACS), an NSF Science and Technology Center funded under NSF Contract STC-88-09648 and supported by the New Jersey Commission on Science and Technology.     

Layered reasoning for randomized distributed algorithms

This paper adopts the communication closed layer (CCL) concept of Elrad and Francez to the formal reasoning of randomized distributed algorithms. We do so by enriching probabilistic automata (PA) with a layered composition operator, an intermediate between parallel and sequential composition. Layered composition is used to establish probabilistic counterparts of the CCL laws that exploit independence and/or precedence conditions between the constituent PA. The probabilistic CCL laws enable partial order (po-) equivalence when layered composition is replaced by sequential composition. Such po-equivalence induces a purely syntactic partial-order state space reduction via layered separation in compositions of PA while preserving probabilistic next-free linear-time properties. The feasibility of such layered separation is demonstrated on a randomized mutual exclusion algorithm by Kushilevitz and Rabin, complementing an algebraic approach (for analyzing this algorithm) by McIver, Gonzalia, Cohen, and Morgan.     

Fast robust regression algorithms for problems with Toeplitz structure

The problem of computing an approximate solution of an overdetermined system of linear equations is considered. The usual approach to the problem is least squares, in which the 2-norm of the residual is minimized. This produces the minimum variance unbiased estimator of the solution when the errors in the observations are independent and normally distributed with mean 0 and constant variance. It is well known, however, that the least squares solution is not robust if outliers occur, i.e., if some of the observations are contaminated by large error. In this case, alternate approaches have been proposed which judge the size of the residual in a way that is less sensitive to these components. These include the Huber M-function, the Talwar function, the logistic function, the Fair function, and the ?₁ norm. New algorithms are proposed to compute the solution to these problems efficiently, in particular, when the matrix A has small displacement rank. Matrices with small displacement rank include matrices that are Toeplitz, block-Toeplitz, block-Toeplitz with Toeplitz blocks, Toeplitz plus Hankel, and a variety of other forms. For exposition, only Toeplitz matrices are considered, but the ideas apply to all matrices with small displacement rank. Algorithms are also presented to compute the solution efficiently when a regularization term is included to handle the case when the matrix of the coefficients is ill-conditioned or rank-deficient. The techniques are illustrated on a problem of FIR system identification.