An Introduction to Statistical Learning Theory

Introduction to Statistical Learning Theory Page 1 Introduction to Statistical Learning Theory John Shawe-Taylor Department of Computer Science University College London jst@cs.ucl.ac.uk May, 2008 Porquerolles Machine Learning School, May 2008 Page 2 STRUCTURE 1. General Statistical Considerations 2. Basic PAC Ideas and proofs 3. Real-valued Function Classes and the Margin 4. Concentration Inequalities 5. Rademacher complexity and Main Theory 6. Applications to classification 7. Conclusions Porquerolles Machine Learning School, May 2008 1 Page 3 Aim: • Some thoughts on why theory • Basic Techniques with some deference to history • Insights into proof techniques and statistical learning approaches • Complete proof of SVM bound using Rademacher approach Porquerolles Machine Learning School, May 2008 2 Page 4 What won’t be included: • The most general results • Complete History • …