Structured large margin machines: sensitive to data distributions |
| |
Authors: | Daniel S Yeung Defeng Wang Wing W Y Ng Eric C C Tsang Xizhao Wang |
| |
Affiliation: | (1) Department of Computing, The Hong Kong Polytechnic University, Kowloon, Hong Kong;(2) Faculty of Mathematics and Computer Science, Hebei University, Baoding, 071002, China |
| |
Abstract: | This paper proposes a new large margin classifier—the structured large margin machine (SLMM)—that is sensitive to the structure
of the data distribution. The SLMM approach incorporates the merits of “structured” learning models, such as radial basis
function networks and Gaussian mixture models, with the advantages of “unstructured” large margin learning schemes, such as
support vector machines and maxi-min margin machines. We derive the SLMM model from the concepts of “structured degree” and
“homospace”, based on an analysis of existing structured and unstructured learning models. Then, by using Ward’s agglomerative
hierarchical clustering on input data (or data mappings in the kernel space) to extract the underlying data structure, we
formulate SLMM training as a sequential second order cone programming. Many promising features of the SLMM approach are illustrated,
including its accuracy, scalability, extensibility, and noise tolerance. We also demonstrate the theoretical importance of
the SLMM model by showing that it generalizes existing approaches, such as SVMs and M4s, provides novel insight into learning models, and lays a foundation for conceiving other “structured” classifiers.
Editor: Dale Schuurmans.
This work was supported by the Hong Kong Research Grant Council under Grants G-T891 and B-Q519. |
| |
Keywords: | Large margin learning Weighted Mahalanobis distance (WMD) Homospace Structured learning Agglomerative hierarchical clustering Second order cone programming (SOCP) |
本文献已被 SpringerLink 等数据库收录! |
|