Sparse Support Vector Machine with <Emphasis Type="Italic">L</Emphasis><Subscript><Emphasis Type="Italic">p</Emphasis></Subscript> Penalty for Feature Selection

We study the strategies in feature selection with sparse support vector machine (SVM). Recently, the socalled L _p -SVM (0 < p < 1) has attracted much attention because it can encourage better sparsity than the widely used L ₁-SVM. However, L _p -SVM is a non-convex and non-Lipschitz optimization problem. Solving this problem numerically is challenging. In this paper, we reformulate the L _p -SVM into an optimization model with linear objective function and smooth constraints (LOSC-SVM) so that it can be solved by numerical methods for smooth constrained optimization. Our numerical experiments on artificial datasets show that LOSC-SVM (0 < p < 1) can improve the classification performance in both feature selection and classification by choosing a suitable parameter p. We also apply it to some real-life datasets and experimental results show that it is superior to L ₁-SVM.     

Upper and lower estimates for the number of some <Emphasis Type="Italic">k</Emphasis>-dimensional subspaces of a given weight over a finite field

A fast simulation method is proposed that makes it possible to construct upper and lower estimates for the number of k-measurable subspaces (of arbitrary weight ω) of an n-measurable vector space over a Galois field containing q elements. Numerical examples demonstrate the accuracy of the estimates.     

Exploiting multi-channels deep convolutional neural networks for multivariate time series classification

Time series classification is related to many different domains, such as health informatics, finance, and bioinformatics. Due to its broad applications, researchers have developed many algorithms for this kind of tasks, e.g., multivariate time series classification. Among the classification algorithms, k-nearest neighbor (k-NN) classification (particularly 1-NN) combined with dynamic time warping (DTW) achieves the state of the art performance. The deficiency is that when the data set grows large, the time consumption of 1-NN with DTWwill be very expensive. In contrast to 1-NN with DTW, it is more efficient but less effective for feature-based classification methods since their performance usually depends on the quality of hand-crafted features. In this paper, we aim to improve the performance of traditional feature-based approaches through the feature learning techniques. Specifically, we propose a novel deep learning framework, multi-channels deep convolutional neural networks (MC-DCNN), for multivariate time series classification. This model first learns features from individual univariate time series in each channel, and combines information from all channels as feature representation at the final layer. Then, the learnt features are applied into a multilayer perceptron (MLP) for classification. Finally, the extensive experiments on real-world data sets show that our model is not only more efficient than the state of the art but also competitive in accuracy. This study implies that feature learning is worth to be investigated for the problem of time series classification.     

Supervised Vessels Classification Based on Feature Selection

Arterial-venous classification of retinal blood vessels is important for the automatic detection of cardiovascular diseases such as hypertensive retinopathy and stroke. In this paper, we propose an arterial-venous classification (AVC) method, which focuses on feature extraction and selection from vessel centerline pixels. The vessel centerline is extracted after the preprocessing of vessel segmentation and optic disc (OD) localization. Then, a region of interest (ROI) is extracted around OD, and the most efficient features of each centerline pixel in ROI are selected from the local features, grey-level co-occurrence matrix (GLCM) features, and an adaptive local binary patten (A-LBP) feature by using a max-relevance and min-redundancy (mRMR) scheme. Finally, a feature-weighted K-nearest neighbor (FW-KNN) algorithm is used to classify the arterial-venous vessels. The experimental results on the DRIVE database and INSPIRE-AVR database achieve the high accuracy of 88.65% and 88.51% in ROI, respectively.     

Metamodel selection based on stepwise regression

Metamodels are often used to replace expensive simulations of engineering problems. When a training set is given, a series of metamodels can be constructed, and then there are two strategies to deal with these metamodels: (1) picking out the best one with the highest accuracy as an approximation of the computationally intensive simulation; and (2) combining all of them into an ensemble model. However, since the choice of approximate model depends on design of experiments (DOEs), employing of the first strategy thus increases the risk of adopting an inappropriate model. Nevertheless, the second strategy also seems not to be a good choice, since adding redundant metamodels may lead to loss of accuracy. Therefore, it is a necessary step to eliminate the redundant metamodels from the set of the candidates before constructing the final ensemble. Illuminated by the method of variable selection widely used in polynomial regression, a metamodel selection method based on stepwise regression is proposed. In our method, just a subset of n ones (n ≤ p, where p is the number of all of the candidate metamodels) is used. In addition, a new ensemble technique is proposed from the view of polynomial regression in this work. This new ensemble technique, combined with metamodel selection method, has been evaluated using six benchmark problems. The results show that eliminating the redundant metamodels before constructing the ensemble can provide more ideal prediction accuracy than directly constructing the ensemble by utilizing all of the candidates.     

Locally adaptive <Emphasis Type="Italic">k</Emphasis> parameter selection for nearest neighbor classifier: one nearest cluster

The k nearest neighbors (k-NN) classification technique has a worldly wide fame due to its simplicity, effectiveness, and robustness. As a lazy learner, k-NN is a versatile algorithm and is used in many fields. In this classifier, the k parameter is generally chosen by the user, and the optimal k value is found by experiments. The chosen constant k value is used during the whole classification phase. The same k value used for each test sample can decrease the overall prediction performance. The optimal k value for each test sample should vary from others in order to have more accurate predictions. In this study, a dynamic k value selection method for each instance is proposed. This improved classification method employs a simple clustering procedure. In the experiments, more accurate results are found. The reasons of success have also been understood and presented.     

Speaker discrimination based on fuzzy fusion and feature reduction techniques

In this paper, we propose a research work on speaker discrimination using a multi-classifier fusion with focus on feature reduction effects. Speaker discrimination consists in the automatic distinction between two speakers using the vocal characteristics of their speeches. A number of features are extracted using Mel Frequency Spectral Coefficients and then reduced using Relative Speaker Characteristic (RSC) along with the Principal Components Analysis (PCA). Several classification methods are implemented to ensure the discrimination task. Since different classifiers are employed, two fusion algorithms at the decision level, referred to as Weighted Fusion and Fuzzy Fusion, are proposed to boost the classification performances. These algorithms are based on the weighting of the different classifiers outputs. Furthermore, the effects of speaker gender and feature reduction on the speaker discrimination task have been examined too. The evaluation of our approaches was conducted on a subset of Hub-4 Broadcast-News. The experimental results have shown that the speaker discrimination accuracy is improved by 5–15% using the (RSC–PCA) feature reduction. In addition, the proposed fusion methods recorded an improvement of about 10% compared to the individual scores of the classifiers. Finally, we noticed that the gender has an important impact on the discrimination performances.     

Guaranteeing Response Times for Aperiodic Tasks in Global Multiprocessor Scheduling

We provide a constant time schedulability test for an on-line multiprocessor server handling aperiodic tasks. Dhall's effect is avoided by dividing the tasks in two priority classes based on task utilization: heavy and light. We prove that if the load on the multiprocessor server stays below U _threshold = 3 ? √7 ≈ 35.425%, the server can accept an incoming aperiodic task and guarantee that the deadlines of all accepted tasks will be met. The same number 35.425% is also a threshold for a task to be characterized as heavy.The bound U _threshold = 3 ? √7≈ 35.425% is easy-to-use, but not sharp if we know the number of processors in the multiprocessor system. Assuming the server to be equipped with m processors, we calculate a formula for the sharp bound U _threshold (m), which converges to U _threshold from above as m → ∞.The results are based on a utilization function u(x) = 2(1 ? x)/(2 + √2+2x). By using this function, the performance of the multiprocessor server can in some cases be improved beyond U _threshold(m) by paying the extra overhead of monitoring the individual utilization of the current tasks.     

On the optimal choice of quality metric in image compression: a soft computing approach

Images take lot of computer space; in many practical situations, we cannot store all original images, we have to use compression. Moreover, in many such situations, compression ratio provided by even the best lossless compression is not sufficient, so we have to use lossy compression. In a lossy compression, the reconstructed image ? is, in general, different from the original image I. There exist many different lossy compression methods, and most of these methods have several tunable parameters. In different situations, different methods lead to different quality reconstruction, so it is important to select, in each situation, the best compression method. A natural idea is to select the compression method for which the average value of some metric d(I,?) is the smallest possible. The question is then: which quality metric should we choose? In this paper, we show that under certain reasonable symmetry conditions, L ^p metrics d(I,?)=∫|I(x)??(x)| ^p dx are the best, and that the optimal value of p can be selected depending on the expected relative size r of the informative part of the image.     

Beyond the structure of SAT formulas

Nowadays, many real-world problems are encoded into SAT instances and efficiently solved by modern SAT solvers. These solvers, usually known as Conflict-Driven Clause Learning (CDCL) SAT solvers, include a variety of sophisticated techniques, such as clause learning, lazy data structures, conflict-based adaptive branching heuristics, or random restarts, among others. However, the reasons of their efficiency in solving real-world, or industrial, SAT instances are still unknown. The common wisdom in the SAT community is that these technique exploit some hidden structure of real-world problems.In this thesis, we characterize some important features of the underlying structure of industrial SAT instances. Namely, they are the community structure and the self-similar structure. We observe that most industrial SAT formulas, viewed as graphs, have these two properties. This means that (i) in a graph with a clear community structure, i.e. having high modularity, we can find a partition of its nodes into communities such that most edges connect nodes of the same community; and (ii) in a graph with a self-similar pattern, i.e. being fractal, its shape is kept after re-scalings, i.e., grouping sets of nodes into a single node. We also analyze how these structures are affected by the effects of CDCL techniques during the search.Using the previous structural studies, we propose three applications. First, we face the problem of generating pseudo-industrial random SAT instances using the notion of modularity. Our model generates instances similar to (classical) random SAT formulas when the modularity is low, but when this value is high, our model is also adequate to model realistic pseudo-industrial problems. Second, we propose a method based on the community structure of the instance to detect relevant learnt clauses. Our technique augments the original instance with this set of relevant clauses, and this results into an overall improvement of the efficiency of several state-of-the-art CDCL SAT solvers. Finally, we analyze the classification of industrial SAT instances into families using the previously analyzed structure features, and we compare them to other classifiers commonly used in portfolio SAT approaches.In summary, this dissertation extends the understandings of the structure of SAT instances, with the aim of better explaining the success of CDCL techniques and possibly improve them, and propose a number of applications based on this analysis of the underlying structure of SAT formulas.     

Quantum Relief algorithm

Relief algorithm is a feature selection algorithm used in binary classification proposed by Kira and Rendell, and its computational complexity remarkably increases with both the scale of samples and the number of features. In order to reduce the complexity, a quantum feature selection algorithm based on Relief algorithm, also called quantum Relief algorithm, is proposed. In the algorithm, all features of each sample are superposed by a certain quantum state through the CMP and rotation operations, then the swap test and measurement are applied on this state to get the similarity between two samples. After that, Near-hit and Near-miss are obtained by calculating the maximal similarity, and further applied to update the feature weight vector WT to get \({\overline{WT}}\) that determine the relevant features with the threshold \(\tau \). In order to verify our algorithm, a simulation experiment based on IBM Q with a simple example is performed. Efficiency analysis shows the computational complexity of our proposed algorithm is O(M), while the complexity of the original Relief algorithm is O(NM), where N is the number of features for each sample, and M is the size of the sample set. Obviously, our quantum Relief algorithm has superior acceleration than the classical one.     

Discerning individual interests and shared interests for social user profiling

Traditionally, research about social user profiling assumes that users share some similar interests with their followees. However, it lacks the studies on what topic and to what extent their interests are similar. Our study in online sharing sites reveals that besides shared interests between followers and followees, users do maintain some individual interests which differ from their followees. Thus, for better social user profiling we need to discern individual interests (capturing the uniqueness of users) and shared interests (capturing the commonality of neighboring users) of the users in the connected world. To achieve this, we extend the matrix factorization model by incorporating both individual and shared interests, and also learn the multi-faceted similarities unsupervisedly. The proposed method can be applied to many applications, such as rating prediction, item level social influence maximization and so on. Experimental results on real-world datasets show that our work can be applied to improve the performance of social rating. Also, it can reveal some interesting findings, such as who likes the “controversial” items most, and who is the most influential in attracting their followers to rate an item.     

Selective A<Emphasis Type="Italic">n</Emphasis>DE for large data learning: a low-bias memory constrained approach

Learning from data that are too big to fit into memory poses great challenges to currently available learning approaches. Averaged n-Dependence Estimators (AnDE) allows for a flexible learning from out-of-core data, by varying the value of n (number of super parents). Hence, AnDE is especially appropriate for learning from large quantities of data. Memory requirement in AnDE, however, increases combinatorially with the number of attributes and the parameter n. In large data learning, number of attributes is often large and we also expect high n to achieve low-bias classification. In order to achieve the lower bias of AnDE with higher n but with less memory requirement, we propose a memory constrained selective AnDE algorithm, in which two passes of learning through training examples are involved. The first pass performs attribute selection on super parents according to available memory, whereas the second one learns an AnDE model with parents only on the selected attributes. Extensive experiments show that the new selective AnDE has considerably lower bias and prediction error relative to A\(n'\)DE, where \(n' = n-1\), while maintaining the same space complexity and similar time complexity. The proposed algorithm works well on categorical data. Numerical data sets need to be discretized first.     

Multi-label classification via learning a unified object-label graph with sparse representation

Automatic annotation is an essential technique for effectively handling and organizing Web objects (e.g., Web pages), which have experienced an unprecedented growth over the last few years. Automatic annotation is usually formulated as a multi-label classification problem. Unfortunately, labeled data are often time-consuming and expensive to obtain. Web data also accommodate much richer feature space. This calls for new semi-supervised approaches that are less demanding on labeled data to be effective in classification. In this paper, we propose a graph-based semi-supervised learning approach that leverages random walks and ? ₁ sparse reconstruction on a mixed object-label graph with both attribute and structure information for effective multi-label classification. The mixed graph contains an object-affinity subgraph, a label-correlation subgraph, and object-label edges with adaptive weight assignments indicating the assignment relationships. The object-affinity subgraph is constructed using ? ₁ sparse graph reconstruction with extracted structural meta-text, while the label-correlation subgraph captures pairwise correlations among labels via linear combination of their co-occurrence similarity and kernel-based similarity. A random walk with adaptive weight assignment is then performed on the constructed mixed graph to infer probabilistic assignment relationships between labels and objects. Extensive experiments on real Yahoo! Web datasets demonstrate the effectiveness of our approach.     

Efficient feature combination techniques for emotional speech classification

The challenge to enhance the naturalness and efficiency of spoken language man–machine interface, emotional speech identification and its classification has been a predominant research area. The reliability and accuracy of such emotion identification greatly depends on the feature selection and extraction. In this paper, a combined feature selection technique has been proposed which uses the reduced features set artifact of vector quantizer (VQ) in a Radial Basis Function Neural Network (RBFNN) environment for classification. In the initial stage, Linear Prediction Coefficient (LPC) and time–frequency Hurst parameter (pH) are utilized to extract the relevant feature, both exhibiting complementary information from the emotional speech. Extensive simulations have been carried out using Berlin Database of Emotional Speech (EMO-DB) with various combination of feature set. The experimental results reveal 76 % accuracy for pH and 68 % for LPC using standalone feature set, whereas the combination of feature sets, (LP VQC and pH VQC) enhance the average accuracy level up to 90.55 %.     

Multi-algorithmic cancelable fingerprint template generation based on weighted sum rule and <Emphasis Type="Italic">T</Emphasis>-operators

With the emergence of biometric-based authentication systems in real-world applications, template protection in biometrics is a significant issue to be considered in the recent years. This paper presents two feature set computation algorithms, namely nearest neighbor feature set (NNFS) and Delaunay triangle feature set (DTFS), for a fingerprint sample. Further, the match scores obtained from these algorithms are fused using weighted sum rule and T-operators (T-norms and T-conorms). The experimental evaluation done on FVC 2002 databases confirms the credibility of fusion method compared to each individual algorithm used for fusing. The EER obtained for proposed method is 0 %, 0.059 %, and 3.93 % for FVC 2002 DB1, DB2, and DB3 databases, respectively. This paper also aims to prove the effectiveness of applying T-operators for fusion at score level in fingerprint template protection.     

Integrated microblog sentiment analysis from users’ social interaction patterns and textual opinions

Traditional post-level opinion classification methods usually fail to capture a person’s overall sentiment orientation toward a topic from his/her microblog posts published for a variety of themes related to that topic. One reason for this is that the sentiments connoted in the textual expressions of microblog posts are often obscure. Moreover, a person’s opinions are often influenced by his/her social network. This study therefore proposes a new method based on integrated information of microblog users’ social interactions and textual opinions to infer the sentiment orientation of a user or the whole group regarding a hot topic. A Social Opinion Graph (SOG) is first constructed as the data model for sentiment analysis of a group of microblog users who share opinions on a topic. This represents their social interactions and opinions. The training phase then uses the SOGs of training sets to construct Sentiment Guiding Matrix (SGM), representing the knowledge about the correlation between users’ sentiments, Textual Sentiment Classifier (TSC), and emotion homophily coefficients of the influence of various types of social interaction on users’ mutual sentiments. All of these support a high-performance social sentiment analysis procedure based on the relaxation labeling scheme. The experimental results show that the proposed method has better sentiment classification accuracy than the textual classification and other integrated classification methods. In addition, IMSA can reduce pre-annotation overheads and the influence from sampling deviation.     

Attracting ellipsoids and synthesis of oscillatory regimes

An approach to stabilization of nonlinear oscillations in multidimensional spaces is proposed on the basis of the V.I. Zubov’s stability theory for invariant sets. As a special case, the derived controls make it possible to excite self-oscillating regimes in specified state subspaces R ^2k ? R ²ⁿ with simultaneous oscillation damping on Cartesian products R ^2n?2k .     

Efficiently Storing Well-Composed Polyhedral Complexes Computed Over 3D Binary Images

A 3D binary image I can be naturally represented by a combinatorial-algebraic structure called cubical complex and denoted by Q(I), whose basic building blocks are vertices, edges, square faces and cubes. In Gonzalez-Diaz et al. (Discret Appl Math 183:59–77, 2015), we presented a method to “locally repair” Q(I) to obtain a polyhedral complex P(I) (whose basic building blocks are vertices, edges, specific polygons and polyhedra), homotopy equivalent to Q(I), satisfying that its boundary surface is a 2D manifold. P(I) is called a well-composed polyhedral complex over the picture I. Besides, we developed a new codification system for P(I), encoding geometric information of the cells of P(I) under the form of a 3D grayscale image, and the boundary face relations of the cells of P(I) under the form of a set of structuring elements. In this paper, we build upon (Gonzalez-Diaz et al. 2015) and prove that, to retrieve topological and geometric information of P(I), it is enough to store just one 3D point per polyhedron and hence neither grayscale image nor set of structuring elements are needed. From this “minimal” codification of P(I), we finally present a method to compute the 2-cells in the boundary surface of P(I).