ECE 901 Summer '09: Statistical Learning Theory
Prerequisites:Background in applied mathematics, probability, and statistics
Instructor:
Robert Nowak
E-mail: nowak@engr.wisc.edu
Web: http://nowak.ece.wisc.edu
Phone: 608 265 3914
3627 Engineering Hall
Office Hours: email for appointment
Lectures:
May 26, 2009-Jul 19, 2009 2009
Time/Place: 9am-12:30pm Wednesdays / MECH ENGR 1152
Course Format:
The course will meet once per week for 3.5 hours. Each meeting
period will be divided into two subperiods, each approximately 1.5 hours
in duration. We will take a short break between the subperiods. Each
subperiod will focus on one of the lectures below.
Lectures:
Lecture 1 A Probabilistic Approach to Pattern Recognition
Lecture 2 Introduction to Classification and Regression
Lecture 3 Introduction to Complexity Regularization
Lecture 4 Denoising in Smooth Function Spaces
Lecture 5 Plug-in Rules and Histogram Classifiers
Lecture 6 Probably Approximately Correct (PAC) Learning
Lecture 7 Chernoff's Bound and Hoeffding's Inequality
Lecture 8 Classification Error Bounds
Lecture 9 Error Bounds in Countably Infinite Models Spaces
Lecture 10 Complexity Regularization
Lecture 11 Decision Trees
Lecture 12 Complexity Regularization for Squared Error Loss
Lecture 13 Maximum Likelihood Estimation
Lecture 14 Maximum Likelihood and Complexity Regularization
Lecture 15 Denoising II: Adapting to Unknown Smoothness
Lecture 16 Wavelet Approximation Theory
Lecture 17 Denoising III: Spatial Adaptivity
Lecture 18 Introduction to VC Theory
Lecture 19 The VC Inequality
Lecture 20 Applications of VC Theory
Homework Problems: TBA
Readings: TBA
Textbooks and References:
A textbook will not be followed in this course. A collection of
notes, relevant papers and materials will be prepared and distributed.
Textbooks recommended for further reading are listed below.
A probabilistic theory of pattern recognition, Devroye, Gyorfi, Lugosi, Springer
Nonparameteric Estimation Theory, Iain Johnstone, unpublished monograph
The Elements of Statistical Learning, Hastie, et al, Springer
An introduction to support vector machines, Cristianini and Shawe-Taylor, Cambridge Press
Combinatorial methods in density estimation, Devroye and Lugosi, Springer
Statistical Learning Theory, Vapnik, Wiley
An Introduction to Computational Learning Theory, Kearns and Vazirani, MIT Press
Empirical Processes in M-Estimation, van de Geer, Cambridge PressGrading and Evaluation:
Grades will be based on course participation and lecture presentations.