ECE 901 Summer '09: Statistical Learning TheoryPrerequisites:Background in applied mathematics, probability, and statistics

Instructor:

Robert Nowak

E-mail: nowak@engr.wisc.edu

Web: http://nowak.ece.wisc.edu

Phone: 608 265 3914

3627 Engineering Hall

Office Hours: email for appointmentLectures:

May 26, 2009-Jul 19, 2009 2009

Time/Place: 9am-12:30pm Wednesdays / MECH ENGR 1152Course Format:

The course will meet once per week for 3.5 hours. Each meeting

period will be divided into two subperiods, each approximately 1.5 hours

in duration. We will take a short break between the subperiods. Each

subperiod will focus on one of the lectures below.Lectures:Lecture 1 A Probabilistic Approach to Pattern RecognitionLecture 2 Introduction to Classification and RegressionLecture 3 Introduction to Complexity RegularizationLecture 4 Denoising in Smooth Function SpacesLecture 5 Plug-in Rules and Histogram ClassifiersLecture 6 Probably Approximately Correct (PAC) LearningLecture 7 Chernoff's Bound and Hoeffding's InequalityLecture 8 Classification Error BoundsLecture 9 Error Bounds in Countably Infinite Models SpacesLecture 10 Complexity RegularizationLecture 11 Decision TreesLecture 12 Complexity Regularization for Squared Error LossLecture 13 Maximum Likelihood EstimationLecture 14 Maximum Likelihood and Complexity RegularizationLecture 15 Denoising II: Adapting to Unknown SmoothnessLecture 16 Wavelet Approximation TheoryLecture 17 Denoising III: Spatial AdaptivityLecture 18 Introduction to VC TheoryLecture 19 The VC InequalityLecture 20 Applications of VC TheoryHomework Problems: TBAReadings: TBATextbooks and References:

A textbook will not be followed in this course. A collection of

notes, relevant papers and materials will be prepared and distributed.

Textbooks recommended for further reading are listed below.

A probabilistic theory of pattern recognition, Devroye, Gyorfi, Lugosi, Springer

Nonparameteric Estimation Theory, Iain Johnstone, unpublished monograph

The Elements of Statistical Learning, Hastie, et al, Springer

An introduction to support vector machines, Cristianini and Shawe-Taylor, Cambridge Press

Combinatorial methods in density estimation, Devroye and Lugosi, Springer

Statistical Learning Theory, Vapnik, Wiley

An Introduction to Computational Learning Theory, Kearns and Vazirani, MIT Press

Empirical Processes in M-Estimation, van de Geer, Cambridge Press

Grading and Evaluation:

Grades will be based on course participation and lecture presentations.