Learning the prior for the PAC-Bayes bound

This paper presents a bound on the performance of a Support Vector Machine obtained within the PAC-Bayes framework. The bound is computed by means of the estimation of a prior of the distribution of SVM classifiers given a particular dataset, and the use of this prior in the PAC-Bayes generalisation bound. The quality of the bound is tested in a model selection task, where it is compared against other procedures to select models based on other PAC-Bayes bounds and ten fold cross-validation. Furthermore, we introduce an algorithm to approximately optimise the new bound and test it against a standard SVM both in terms of bound value and test set error.