IP-LSSVM: A two-step sparse classifier

We present in this work a two-step sparse classifier called IP-LSSVM which is based on Least Squares Support Vector Machine (LS-SVM). The formulation of LS-SVM aims at solving the learning problem with a system of linear equations. Although this solution is simpler, there is a loss of sparseness in the feature vectors. Many works on LS-SVM are focused on improving support vectors representation in the least squares approach, since they correspond to the only vectors that must be stored for further usage of the machine, which can also be directly used as a reduced subset that represents the initial one. The proposed classifier incorporates the advantages of either SVM and LS-SVM: automatic detection of support vectors and a solution obtained simply by the solution of systems of linear equations. IP-LSSVM was compared with other sparse LS-SVM classifiers from literature, and RRS+LS-SVM. The experiments were performed on four important benchmark databases in Machine Learning and on two artificial databases created to show visually the support vectors detected. The results show that IP-LSSVM represents a viable alternative to SVMs, since both have similar features, supported by literature results and yet IP-LSSVM has a simpler and more understandable formulation.