Time: Tue 13:00-16:00

Location: Cluster Bd. (학연산클러스터) R. 507

References: no textbook is required, but following references will be helpful:

• • Elements of Statistical Learning, Springer, 2009 -- free PDF available

  • Machine Learning, Murphy, MIT Press, 2012

  • Infomation theory, inference, and learning algorithms, Mackay, Cambridge, 2017 (9th printing) -- free PDF available

  • Foundations of machine learning, Mohri, Rostamizadeh, and Talwalar, MIT Press, 2018 (2nd Ed)

  • All of statistics, Wasserman, Springer, 2005

  • CS229 Stanford lecture by Andrew Ng

Grading:

• • Homework/discussion, Midterm, Final Exam = 30%, 30%, 30%

  • Attendance: 10%

Homework

• Homework 1 (due April 2)

• Homework 2 (due April 16)

  • lr_debug.py, data_a.txt, data_b.txt

• Homework 3 (due May 28)

Lecture Notes

Topics will be selected from learning models, learning theory, and large-scale optimization.

• Lecture 1. Introduction (pdf)

• Lecture 2. Supervised Learning (pdf)

  • Linear Regression, Weighted Linear Regression, Perceptron, Exponential Family, Generalized Linear Models

  • Suppl : Linear algebra review (pdf)

• Lecture 3. Generative Models (pdf)

  • Gaussian discriminative analysis, naive Bayes, Laplace smoothing

  • Suppl : Probability theory review (pdf)

  • HW1 is out!

• Lecture 4. Support vector machines (pdf)

  • Suppl: Convex optimization (pdf1, pdf2)

  • Suppl: SMO paper by John Platt 1998 (link), String kernel by Leslie & Kuang 2004 (link)

• Lecture 5. HW1 Discussion & Learning Theory part I (pdf)

  • Suppl: Hoeffding's inequality (pdf)

  • HW2 is out!

• Lecture 6. Learning Theory part 2

• Lecture 7. HW2 Discussion, SMO algorithm, Regularization and Model Selection (pdf)

• Midterm (April 23, in class)

• Lecture 8. EM algorithm (pdf1, pdf2)

  • Bayesian inference (remainder of L7)

  • Midterm review

• Lecture 9. Mixture of Gaussian, Factor Analysis (pdf)

• Lecture 10. PCA (pdf) and ICA (pdf)

  • Eigenfaces

  • HW3 is out!

• Lecture 11. Reinforcement learning and control (pdf)

• Lecture 12. LQR, DDP, and LQG (pdf)

• Lecture 13. POMDP, Policy Search, Reinforce, Pegasus

• Final Exam (June 11, in class)