Feature selection via sensitivity analysis of SVM probabilistic outputs

Feature selection is an important aspect of solving data-mining and machine-learning problems. This paper proposes a feature-selection method for the Support Vector Machine (SVM) learning. Like most feature-selection methods, the proposed method ranks all features in decreasing order of importance so that more relevant features can be identified. It uses a novel criterion based on the probabilistic outputs of SVM. This criterion, termed Feature-based Sensitivity of Posterior Probabilities (FSPP), evaluates the importance of a specific feature by computing the aggregate value, over the feature space, of the absolute difference of the probabilistic outputs of SVM with and without the feature. The exact form of this criterion is not easily computable and approximation is needed. Four approximations, FSPP1-FSPP4, are proposed for this purpose. The first two approximations evaluate the criterion by randomly permuting the values of the feature among samples of the training data. They differ in their choices of the mapping function from standard SVM output to its probabilistic output: FSPP1 uses a simple threshold function while FSPP2 uses a sigmoid function. The second two directly approximate the criterion but differ in the smoothness assumptions of criterion with respect to the features. The performance of these approximations, used in an overall feature-selection scheme, is then evaluated on various artificial problems and real-world problems, including datasets from the recent Neural Information Processing Systems (NIPS) feature selection competition. FSPP1-3 show good performance consistently with FSPP2 being the best overall by a slight margin. The performance of FSPP2 is competitive with some of the best performing feature-selection methods in the literature on the datasets that we have tested. Its associated computations are modest and hence it is suitable as a feature-selection method for SVM applications. Editor: Risto Miikkulainen.     

Design and Implementation of an Agent Home Scheme Strategy for Prefetch-Based DSM Systems

In recent years, cluster computing has been widely investigated and there is no doubt that it can provide a cost-effective computing infrastructure by aggregating computational power, communication, and storage resources. Moreover, it is also considered to be a very attractive platform for low-cost supercomputing. Distributed shared memory (DSM) systems utilize the physical memory of each computing node interconnected in a private network to form a global virtual shared memory. Since this global shared memory is distributed among the computing nodes, accessing the data located in remote computing nodes is an absolute necessity. However, this action will result in significant remote memory access latencies which are major sources of overhead in DSM systems. For these reasons, in order to increase overall system performance and decrease this overhead, a number of strategies have been devised. Prefetching is one such approach which can reduce latencies, although it always increases the workload in the home nodes. In this paper, we propose a scheme named Agent Home Scheme. Its most noticeable feature, when compared to other schemes, is that the agent home distributes the workloads of each computing nodes when sending data. By doing this, we can reduce not only the workload of the home nodes by balancing the workload for each node, but also the waiting time. Experimental results show that the proposed method can obtain about 20% higher performance than the original JIAJIA, about 18% more than History Prefetching Strategy (HPS), and about 10% higher than Effective Prefetch Strategy (EPS).     

The Impact of Speculative Execution on SMT Processors

By executing two or more threads concurrently, Simultaneous MultiThreading (SMT) architectures are able to exploit both Instruction-Level Parallelism (ILP) and Thread-Level Parallelism (TLP) from the increased number of in-flight instructions that are fetched from multiple threads. However, due to incorrect control speculations, a significant number of these in-flight instructions are discarded from the pipelines of SMT processors (which is a direct consequence of these pipelines getting wider and deeper). Although increasing the accuracy of branch predictors may reduce the number of instructions so discarded from the pipelines, the prediction accuracy cannot be easily scaled up since aggressive branch prediction schemes strongly depend on the particular predictability inherently to the application programs. In this paper, we present an efficient thread scheduling mechanism for SMT processors, called SAFE-T (Speculation-Aware Front-End Throttling): it is easy to implement and allows an SMT processor to selectively perform speculative execution of threads according to the confidence level on branch predictions, hence preventing wrong-path instructions from being fetched. SAFE-T provides an average reduction of 57.9% in the number of discarded instructions and improves the instructions per cycle (IPC) performance by 14.7% on average over the ICOUNT policy across the multi-programmed workloads we simulate. This paper is an extended version of the paper, “Speculation Control for Simultaneous Multithreading,” which appeared in the Proceedings of the 18th International Parallel and Distributed Processing Symposium, Santa Fe, New Mexico, April 2004.     

Boosted Bayesian network classifiers

The use of Bayesian networks for classification problems has received a significant amount of recent attention. Although computationally efficient, the standard maximum likelihood learning method tends to be suboptimal due to the mismatch between its optimization criteria (data likelihood) and the actual goal of classification (label prediction accuracy). Recent approaches to optimizing classification performance during parameter or structure learning show promise, but lack the favorable computational properties of maximum likelihood learning. In this paper we present boosted Bayesian network classifiers, a framework to combine discriminative data-weighting with generative training of intermediate models. We show that boosted Bayesian network classifiers encompass the basic generative models in isolation, but improve their classification performance when the model structure is suboptimal. We also demonstrate that structure learning is beneficial in the construction of boosted Bayesian network classifiers. On a large suite of benchmark data-sets, this approach outperforms generative graphical models such as naive Bayes and TAN in classification accuracy. Boosted Bayesian network classifiers have comparable or better performance in comparison to other discriminatively trained graphical models including ELR and BNC. Furthermore, boosted Bayesian networks require significantly less training time than the ELR and BNC algorithms.     

Inductive transfer with context-sensitive neural networks

Context-sensitive Multiple Task Learning, or csMTL, is presented as a method of inductive transfer which uses a single output neural network and additional contextual inputs for learning multiple tasks. Motivated by problems with the application of MTL networks to machine lifelong learning systems, csMTL encoding of multiple task examples was developed and found to improve predictive performance. As evidence, the csMTL method is tested on seven task domains and shown to produce hypotheses for primary tasks that are often better than standard MTL hypotheses when learning in the presence of related and unrelated tasks. We argue that the reason for this performance improvement is a reduction in the number of effective free parameters in the csMTL network brought about by the shared output node and weight update constraints due to the context inputs. An examination of IDT and SVM models developed from csMTL encoded data provides initial evidence that this improvement is not shared across all machine learning models.     

Performance Evaluation of a Multi-Zone Application in Different OpenMP Approaches

We describe a performance study of a multi-zone application benchmark implemented in several OpenMP approaches that exploit multi-level parallelism and deal with unbalanced workload. The multi-zone application was derived from the well-known NAS Parallel Benchmarks (NPB) suite that involves flow solvers on collections of loosely coupled discretization meshes. Parallel versions of this application have been developed using the Subteam concept and Workqueuing model as extensions to the current OpenMP. We examine the performance impact of these extensions to OpenMP and compare with hybrid and nested OpenMP approaches on several large parallel systems.     

A Rectangular Finite Volume Element Method for a Semilinear Elliptic Equation

In this paper we extend the idea of interpolated coefficients for semilinear problems to the finite volume element method based on rectangular partition. At first we introduce bilinear finite volume element method with interpolated coefficients for a boundary value problem of semilinear elliptic equation. Next we derive convergence estimate in H ¹-norm and superconvergence of derivative. Finally an example is given to illustrate the effectiveness of the proposed method. This work is supported by Program for New Century Excellent Talents in University of China State Education Ministry, National Science Foundation of China, the National Basic Research Program under the Grant (2005CB321703), the key project of China State Education Ministry (204098), Scientific Research Fund of Hunan Provincial Education Department, China Postdoctoral Science Foundation (No. 20060390894) and China Postdoctoral Science Foundation (No. 20060390894).     

Color Subspaces as Photometric Invariants

Complex reflectance phenomena such as specular reflections confound many vision problems since they produce image ‘features’ that do not correspond directly to intrinsic surface properties such as shape and spectral reflectance. A common approach to mitigate these effects is to explore functions of an image that are invariant to these photometric events. In this paper we describe a class of such invariants that result from exploiting color information in images of dichromatic surfaces. These invariants are derived from illuminant-dependent ‘subspaces’ of RGB color space, and they enable the application of Lambertian-based vision techniques to a broad class of specular, non-Lambertian scenes. Using implementations of recent algorithms taken from the literature, we demonstrate the practical utility of these invariants for a wide variety of applications, including stereo, shape from shading, photometric stereo, material-based segmentation, and motion estimation.     

Analysis of Two-Dimensional Non-Rigid Shapes

Analysis of deformable two-dimensional shapes is an important problem, encountered in numerous pattern recognition, computer vision and computer graphics applications. In this paper, we address three major problems in the analysis of non-rigid shapes: similarity, partial similarity, and correspondence. We present an axiomatic construction of similarity criteria for deformation-invariant shape comparison, based on intrinsic geometric properties of the shapes, and show that such criteria are related to the Gromov-Hausdorff distance. Next, we extend the problem of similarity computation to shapes which have similar parts but are dissimilar when considered as a whole, and present a construction of set-valued distances, based on the notion of Pareto optimality. Finally, we show that the correspondence between non-rigid shapes can be obtained as a byproduct of the non-rigid similarity problem. As a numerical framework, we use the generalized multidimensional scaling (GMDS) method, which is the numerical core of the three problems addressed in this paper.     

Model of Frequency Analysis in the Visual Cortex and the Shape from Texture Problem

This paper addresses the question: at the level of cortical cells present in the primary area V1, is the information sufficient to extract the local perspective from the texture? Starting from a model of complex cells in visual area V1, we propose a biologically plausible algorithm for frequency analysis applied to the shape from texture problem. First, specific log-normal filters are designed in replacement of the classical Gabor filters because of their theoretical properties and of their biological plausibility. These filters are separable in frequency and orientation and they better sample the image spectrum which makes them appropriate for any pattern analysis technique. A method to estimate the local frequency in the image, which discards the need to choose the best local scale, is designed. Based on this frequency analysis model, a local decomposition of the image into patches leads to the estimation of the local frequency variation which is used to solve the problem of recovering the shape from the texture. From the analytical relation between the local frequency and the geometrical parameters, under perspective projection, it is possible to recover the orientation and the shape of the original image. The accuracy of the method is evaluated and discussed on different kind of textures, both regular and irregular, with planar and curved surfaces and also on natural scenes and psychophysical stimuli. It compares favorably to the best existing methods, with in addition, a low computational cost. The biological plausibility of the model is finally discussed.