Basic Statistical Learning Theory

Basic Statistical Learning Theory Page 1 Basic Statistical Learning Theory John Shawe-Taylor School of Electronics and Computer Science University of Southampton jst@ecs.soton.ac.uk September, 2004 Berder Island Summer School, September 2004 Page 2 STRUCTURE 1. General Statistical Considerations 2. Basic PAC Ideas and proofs 3. Real-valued Function Classes and the Margin 4. Concentration and Stability 5. Rademacher complexity and Main Theory 6. Applications to classification and kernel PCA 7. Conclusions Berder Island Summer School, September 2004 1 Page 3 Aim: • Some thoughts on why theory • Basic Techniques with some deference to history • Insights into proof techniques and statistical learning approaches • Complete proof of SVM bound using Rademacher approach Berder Island Summer School, September 2004 2 Page 4 What won’t be included: • The most general results • …