Learning Theory

Learning Theory Page 1 Learning Theory John Shawe-Taylor Centre for Computational Statistics and Machine Learning University College London jst@cs.ucl.ac.uk September, 2009 Machine Learning Summer School (MLSS’09) Tutorial, Sept 2009 Page 2 STRUCTURE PART A 1. General Statistical Considerations 2. Basic PAC Ideas and proofs 3. Real-valued Function Classes and the Margin PART B 1. Rademacher complexity and Main Theory 2. Applications to classification 3. Conclusions Machine Learning Summer School (MLSS’09) Tutorial, Sept 2009 1 Page 3 Aim: • Some thoughts on why theory • Basic Techniques with some deference to history • Insights into proof techniques and statistical learning approaches • Concentration inequalities and relation to Rademacher approach Machine Learning Summer School (MLSS’09) Tutorial, Sept 2009 2 Page 4 What won’t be included: • The most general results • …