Prior Support Vector Machines: minimum-bound vs. maximum-margin classifiers

In this paper we introduce a new algorithm to train Support Vector Machines that aims at the minimisation of the PAC-Bayes bound on the error instead of at the traditional maximisation of the margin. The training of the classifier proceeds in two stages. First some data are used to estimate a prior distribution of classifiers. Then, an optimisation procedure based on quadratic programming determines the classifier as the centre of the posterior distribution that minimises the PAC-Bayes according to the previously obtained prior.