Risk Minimization and Tail Bounds

Advanced Introduction to Machine Learning - 10-715

Content

  • Risk and loss

    • Loss functions

    • Risk

    • Empirical risk vs True risk

    • Empirical Risk minimization

    • Approximation error

    • Estimation error

  • Underfitting and Overfitting

  • Examples

    • Classification

    • Regression

Supplementary material

Slides in PDF

Recommended books and papers

  • Slides about Stochastic Convergence in PDF.

  • L. Devroye, L. Gyorfi, G. Lugosi: A Probabilistic Theory of Pattern Recognition. Springer, New York, 1996.

  • L. Gyorfi (editor): Principles of Nonparametric Learning, Springer-Verlag Wien New York, 2002

  • L. Gyorfi, M. Kohler, A. Krzyzak, H. Walk: A Distribution-Free Theory of Nonparametric Regression, Springer-Verlag, New York, 2002

  • A. Tsybakov: Introduction to Nonparametric Estimation, Springer Series in Statistics, 2008

  • Robert D. Nowak: Lecture notes about Statistical Learning Theory