从神经网络到支撑矢量机（上）

在统计模式识别中，Bayes决策规则从理论上解决了最优分类器的设计问题，然后其实施却必须首先解决更困难的概率密度估计问题。BP神经网络直接从观测数据（训练样本）学习，是更简便有效的方法，因而获得了广泛的应用，但它是一种启发式技术，缺乏指导工程实践的坚实理论基础。统计推断理论研究所取得的突破性成果导致现代统计学习理论－VC理论的建立，该理论不仅在严格的数学基础上圆满地回答了人工神经网络中出现的理论问题，而且导出了一种新的学习方法－支撑矢量机（SVM），SVM已经在一些实际问题中获得了成功的应用，性能优于传统的神经网络方法，本文以模式识别问题为背景，介绍VC理论的主要内容及支撑矢量机方法。

From neural networks to support vector machines(A)

In the field of statistical pattern recognition, optimal classifiers may be designed theoretically based on the Bayesian decision rule, however, it is necessary for the implementation of the design to so1ve a more difficu1t problem of density estimation first. The strategy adopted in HP neural networks is learning di-rectly front the measurement data( training samples), which is more efficient and effective. Therefore the methodology of neural networks has been widely used in real life applications, but like other heuristic meth-ods, it lacks a solid theoretical foundation to direct engineering practice. As the result of the breakthrough in the research of statistical inference, VC theory has been established and accepted as the modern statistical learning theory. The behavior of neural networks may he explained by VC theory with mathematical rigor. in addition, a more powerful learning method-the support vector machine has been constructed based on the theory and gained real life applications. This paper is a tutorial in which the basic concepts of VC theory and the methodology of SVMs as applied to pattern recognition problems are reviewed.