SMO-based pruning methods for sparse least squares support vector machines

Solutions of least squares support vector machines (LS-SVMs) are typically nonsparse. The sparseness is imposed by subsequently omitting data that introduce the smallest training errors and retraining the remaining data. Iterative retraining requires more intensive computations than training a single nonsparse LS-SVM. In this paper, we propose a new pruning algorithm for sparse LS-SVMs: the sequential minimal optimization (SMO) method is introduced into pruning process; in addition, instead of determining the pruning points by errors, we omit the data points that will introduce minimum changes to a dual objective function. This new criterion is computationally efficient. The effectiveness of the proposed method in terms of computational cost and classification accuracy is demonstrated by numerical experiments.     

Sequential approaches for learning datum-wise sparse representations

In supervised classification, data representation is usually considered at the dataset level: one looks for the ??best?? representation of data assuming it to be the same for all the data in the data space. We propose a different approach where the representations used for classification are tailored to each datum in the data space. One immediate goal is to obtain sparse datum-wise representations: our approach learns to build a representation specific to each datum that contains only a small subset of the features, thus allowing classification to be fast and efficient. This representation is obtained by way of a sequential decision process that sequentially chooses which features to acquire before classifying a particular point; this process is learned through algorithms based on Reinforcement Learning. The proposed method performs well on an ensemble of medium-sized sparse classification problems. It offers an alternative to global sparsity approaches, and is a natural framework for sequential classification problems. The method extends easily to a whole family of sparsity-related problem which would otherwise require developing specific solutions. This is the case in particular for cost-sensitive and limited-budget classification, where feature acquisition is costly and is often performed sequentially. Finally, our approach can handle non-differentiable loss functions or combinatorial optimization encountered in more complex feature selection problems.     

Computational methods for sparse matrices

This paper is a survey of methods currently available for processing sparse matrices in a digital computer; specifically in the solution of linear algebraic equations and the eigenproblem.     

Context-based semantical vector representations for animal trajectories

ABSTRACT

Behaviour could be expressed as a set of specific movement patterns in time. An animal's movement or trajectory could characterise its behaviours and provide information about its internal states. Recent advances in GPS-based sensor technologies led to drastic increase in volume of the data collected from animals' movements which enables researchers to analyse and model their behaviours using data-driven methods. However, having compact, discriminative, semantical and independent numerical representations of trajectories as features, is essential for employing the most of available off-the-shelf machine learning and deep learning techniques. Inspired by language processing, the approach presented in this study utilizes Skip-gram model to create contextual vector embeddings or representations of key-points in animal trajectories to be used as input features. Here, a key-point is defined as a location which represents a trajectory segment. It is assumed that these key-points encapsulate contextual information which is attributed to a certain behaviour or specific group of animals with similar behavioural features. So, the vector embeddings could be interpreted as contextual semantical representations of trajectory key-points independent of their spatial coordinates. With these representations, it would be possible to predict likelihood of preceding or subsequent key-points given a context or an internal state, or vice versa. To test this hypothesis, an experiment was conducted on birds' trajectories logged from a seabird species, Streaked Shearwater (Calonectris leucomelas). In this experiment, vector representations of the key-points in birds' trajectories were constructed and optimized using candidate sampling. The experimental results showcased the utility of these vector embeddings in both exploration of Streaked Shearwater trajectory data and improvement of gender-based trajectory classification. In summary, the proposed method provided a novel approach for numerical representation of animal trajectories and, it was illustrated to be semantically more explanatory for analysis as well as being more informative as features for modelling of animal movement data.     

A variational method for learning sparse and overcomplete representations.

An expectation-maximization algorithm for learning sparse and overcomplete data representations is presented. The proposed algorithm exploits a variational approximation to a range of heavy-tailed distributions whose limit is the Laplacian. A rigorous lower bound on the sparse prior distribution is derived, which enables the analytic marginalization of a lower bound on the data likelihood. This lower bound enables the development of an expectation-maximization algorithm for learning the overcomplete basis vectors and inferring the most probable basis coefficients.     

Properties of gray and binary representations

Representations are formalized as encodings that map the search space to the vertex set of a graph. We define the notion of bit equivalent encodings and show that for such encodings the corresponding Walsh coefficients are also conserved. We focus on Gray codes as particular types of encoding and present a review of properties related to the use of Gray codes. Gray codes are widely used in conjunction with genetic algorithms and bit-climbing algorithms for parameter optimization problems. We present new convergence proofs for a special class of unimodal functions; the proofs show that a steepest ascent bit climber using any reflected Gray code representation reaches the global optimum in a number of steps that is linear with respect to the encoding size. There are in fact many different Gray codes. Shifting is defined as a mechanism for dynamically switching from one Gray code representation to another in order to escape local optima. Theoretical results that substantially improve our understanding of the Gray codes and the shifting mechanism are presented. New proofs also shed light on the number of unique Gray code neighborhoods accessible via shifting and on how neighborhood structure changes during shifting. We show that shifting can improve the performance of both a local search algorithm as well as one of the best genetic algorithms currently available.     

Efficient sparse nonparallel support vector machines for classification

In this paper, we propose a novel nonparallel classifier, named sparse nonparallel support vector machine (SNSVM), for binary classification. Different with the existing nonparallel classifiers, such as the twin support vector machines (TWSVMs), SNSVM has several advantages: It constructs two convex quadratic programming problems for both linear and nonlinear cases, which can be solved efficiently by successive overrelaxation technique; it does not need to compute the inverse matrices any more before training; it has the similar sparseness with standard SVMs; it degenerates to the TWSVMs when the parameters are appropriately chosen. Therefore, SNSVM is certainly superior to them theoretically. Experimental results on lots of data sets show the effectiveness of our method in both sparseness and classification accuracy and, therefore, confirm the above conclusions further.     

A unified framework for the construction of various matrix multisplitting iterative methods for large sparse system of linear equations

A unified framework for the construction of various synchronous and asynchronous parallel matrix multisplitting iterative methods, suitable to the SIMD and MIMD multiprocessor systems, respectively, is presented, and its convergence theory is established under rather weak conditions. These afford general method models and systematical convergence criterions for studying the parallel iterations in the sense of matrix multisplitting. In addition, how the known parallel matrix multisplitting iterative methods can be classified into this new framework, and what novel ones can be generated by it are shown in detail.     

On complexity of optimal recombination for binary representations of solutions

We consider the optimization problem of finding the best possible offspring as a result of a recombination operator in an evolutionary algorithm, given two parent solutions. The optimal recombination is studied in the case where a vector of binary variables is used as a solution encoding. By means of efficient reductions of the optimal recombination problems (ORPs) we show the polynomial solvability of the ORPs for the maximum weight set packing problem, the minimum weight set partition problem, and for linear Boolean programming problems with at most two variables per inequality, and some other problems. We also identify several NP-hard cases of optimal recombination: the Boolean linear programming problems with three variables per inequality, the knapsack, the set covering, the p-median, and some other problems.     

Graphic representations of three-component vector fields

A technique is described for graphically displaying three-component vector fields in a display plane. Direction of the vector field is indicated by displaying, at various points of the plane, sketches of an ‘arrowhead’, a readily identified standard object, in the correct orientation. Magnitude is indicated by scaling the size of the arrowhead.

For use in raster graphics, arrowheads must be simple and have an unabiguous appearance in any orientation. Cones, cylinders, and spheres have been investigated, and results are given. A short, hockey-puck shaped cylinder with faces of contrasting colours appears to have good visual impact even when plotted to small scale in a discrete (raster graphics) display.

This shape is also easy to plot, and makes good use of available screen space. It is therefore recommended over the other candidate shapes.     

The optimal synapse for sparse, binary signals in the rod pathway

The sparsity of photons at very low light levels necessitates a nonlinear synaptic transfer function between the rod photoreceptors and the rod-bipolar cells. We examine different ways to characterize the performance of the pathway: the error rate, two variants of the mutual information, and the signal-to-noise ratio. Simulation of the pathway shows that these approaches yield substantially different performance at very low light levels and that maximizing the signal-to-noise ratio yields the best performance when judged from simulated images. The results are compared to recent data.     

General methods for determining projective invariants in imagery

This paper presents the results of a study of projective invariants and their applications in image analysis and object recognition. The familiar cross-ratio theorem, relating collinear points in the plane to the projections through a point onto a line, provides a starting point for their investigation. Methods are introduced in two dimensions for extending the cross-ratio theorem to relate noncollinear object points to their projections on multiple image lines. The development is further extended to three dimensions. It is well known that, for a set of points distributed in three dimensions, stereo pairs of images can be made and relative distances of the points from the film plane computed from measurements of the disparity of the image points in the stereo pair. These computations require knowledge of the effective focal length and baseline of the imaging system. It is less obvious, but true, that invariant metric relationships among the object points can be derived from measured relationships among the image points. These relationships are a generalization into three dimensions of the invariant cross-ratio of distances between points on a line. In three dimensions the invariants are cross-ratios of areas and volumes defined by the object points. These invariant relationships, which are independent of the parameters of the imaging system, are derived and demonstrated with examples.     

Comparator networks for binary heap construction

Comparator networks for constructing binary heaps of size n are presented which have size and depth . A lower bound of for the size of any heap construction network is also proven, implying that the networks presented are within a constant factor of optimal. We give a tight relation between the leading constants in the size of selection networks and in the size of heap construction networks.     

Feature fusion based on joint sparse representations and wavelets for multiview classification

Pattern Analysis and Applications - Feature-level-based fusion has attracted much interest. Generally, a dataset can be created in different views, features, or modalities. To improve the...     

On representations of 2-increasing binary aggregation functions

In this paper, we provide two different representations of 2-increasing binary aggregation functions by means of their lower and upper margins and a suitable copula.     

Efficient sparse matrix-delayed vector multiplication for discretized neural field model

Computational models of the human brain provide an important tool for studying the principles behind brain function and disease. To achieve whole-brain simulation, models are formulated at the level of neuronal populations as systems of delayed differential equations. In this paper, we show that the integration of large systems of sparsely connected neural masses is similar to well-studied sparse matrix-vector multiplication; however, due to delayed contributions, it differs in the data access pattern to the vectors. To improve data locality, we propose a combination of node reordering and tiled schedules derived from the connectivity matrix of the particular system, which allows performing multiple integration steps within a tile. We present two schedules: with a serial processing of the tiles and one allowing for parallel processing of the tiles. We evaluate the presented schedules showing speedup up to \(2\,\times \) on single-socket CPU, and \(1.25\,\times \) on Xeon Phi accelerator.     

一种快速稀疏最小二乘支持向量回归机

将Jiao法直接应用于最小二乘支持向量回归机上的效果并不理想,为此采用不完全抛弃的策略,提出了改进的Jiao法,并将其应用于最小二乘支持向量回归机.数据集测试的结果表明,基于改进Jiao法的稀疏最小二乘支持向量回归机,无论在支持向量个数和训练时间上都取得了一定的优势.与其他剪枝算法相比,在不丧失回归精度的情况下,改进的Jiao法可大大缩短训练时间.另外,改进的Jiao法同样适用于分类问题.     

Linear quadtrees from vector representations of polygons

A new algorithm is presented which produces various forms of linear quadtrees directly from a vector representation of a polygon. This algorithm takes advantage of specific properties of linear quadtrees and associated linear keys to infer the colors of all parts of the region not cut by the polygon boundary. The method is further extended to multicolored (rather than binary) linear quadtrees which may be useful in geographic information systems applications.     

Preconditioned iterative methods for the large sparse symmetric eigenvalue problem

In general, two strategies are used in methods for the solution of large sparse eigensystems. The former are transformation and elimination methods which may change the structure of the original matrix, destroy sparsity and are only suitable if all or most of the eigenvalues are required. The latter includes methods which are iterative and no change in the structure of the original matrix occurs. These methods are often employed when one or several of the eigenvalues are required. In this paper we study iterative methods whereby the extreme eigenvalues and their corresponding eigenvectors are evaluated by a new preconditioning method which with a suitable choice of shift of origin and preconditioning parameter produces a powerful convergent method to cope with problems in this class.