CS 527A Lecture Notes

Lecture 1 - Introduction. This lecture was written using the notes of Carla Brodey (Purdue) as a starting point.

Lecture 2 - Example for Reinforcement Learning: Playing Checkers.

Lecture 3 and 4 - Concept Learning and Version Spaces.

Lecture 5 - Decision Trees.

Lecture 6 - Decision Trees (cont).

Lecture 7 - Artificial Neural Networks.

Lecture 8 - Artificial Neural Networks (cont).

Lecture 9 - Evaluating Hypothesis.

Lecture 10 - Evaluating Hypothesis (cont).

Lecture 11 - Learning Theory.

Lectures 12 and 13 - Learning Theory (cont).

Lecture 14 - Boosting. For additional coverage the following survey paper is very good.

Lecture 15 - Support Vector Machines. For a more in depth coverage the following Techincal Report is very good.

Lecture 16 - Bayesian Learning.

Lectures 17,18 - Naive Bayes applied to text categorization, Bayesian Belief networks, EM, and a very brief intro to Hidden Markov Models.

Additional Lecture Notes on Learning Theory

Introduction to PAC model (Chapter 1, pages 9-16)

Two-Button PAC Model (Chapter 2, pages 17-22)

Learning k-term-DNF (Chapter 3, pages 23-28)

Handling an Unknown Size Parameter and Hypothesis Testing (Chapter 4, pages 29-32)

Learning with Noise (Chapter 5, pages 33-42 and Chapter 15, pages 113-122)

Occam's Razor (Chapter 6, pages 43-46)

Vapnik-Chervonenkis (VC) Dimension (Chapter 7, pages 47-56 and Chapter 14, 107-112)

Representation Independent Hardness (Chapter 8, pages 57-60)

Weak Learning and Boosting (Chapter 9, pages 61-68)

Learning with Queries (Chapter 10, pages 69-76)

Learning Horn Sentences (Chapter 11, pages 77-82)

Learning with Many Irrelevant Attributes using Winnow (Chapter 12, pages 83-94)

Learning Regular Sets (Chapter 13, pages 95-106)

Inferring Graphs from Walks (Chapter 16, pages 123-136)

Learning in an Infinite Attribute Space (Chapter 17, pages 137-142)

Weighted Majority Algorithm (Chapter 18, pages 143-150)

Efficiently Implementing the Halving Algorithm (Chapter 19, pages 151-156)