Main Page
From Btry6790
This is the course web page for BTRY 6790 (CS 6782), "Probabilistic Graphical Models" (Fall 2010).
Please check this page frequently throughout the semester. It will continually be updated with information you will need. Keep in mind that the schedules for lectures and homeworks are subject to change as the semester progresses.
Contents |
Announcements
- We will use the discussion section this week (9/22) to catch up on topics that we haven't gotten to in lecture, viz., linear classification and some loose ends on the max-product algorithm.
- The lecture and reading schedule for the period between now and Fall Break has been updated
- Notes for recitation #3 have been posted.
- Notes from the second recitation have been posted (see bottom of page). Thanks to scribe Ronan LeBras
- Notes from the first recitation have been posted (see bottom of page).
- Homework #1 has been posted (Sep 5). It will be due Fri, Sep 17.
- Note the new location for the lectures: 224 Weill Hall
- Note the new time and location for the discussion section: Wed 3:35-4:25 in 262 Uris
- Be sure to join the course mailing list: btry6790-l (Instructions for joining lyris lists)
Course Description
A thorough introduction to probabilistic graphical models, a flexible and powerful graph-based framework for probabilistic modeling. Covers directed and undirected models, exact and approximate inference, and learning in the presence of latent variables. Hidden Markov models, conditional random fields, and Kalman filtering are explored in detail.
General Information
- Lectures: Tues/Thurs, 11:40-12:55, 224 Weill
- Recitations: Wed, 3:35-4:25, 262 Uris
- Credit Hours: 4 (S/U or letter)
- Instructor: Adam Siepel, 102E Weill
- TA: Andre Luis Martins, 102 Weill
- Office Hours: Tues, 4:00-5:00PM
Prerequisites
Required prerequisites are a course in probability theory (BTRY 4080 or equivalent) and a course in intermediate programming/data structures (CS 2110 or equivalent). A course in mathematical statistics (BTRY 4090 or equivalent) is recommended but not required.
Resources
- Syllabus
- Information about the class project
- Errata from Jordan's manuscript
- Papers by Yedidia, Freeman, and Weiss on connections between belief propagation and variational inference: Bethe free energy, Kikuchi approximations, and belief propagation algorithms, Understanding Belief Propagation and its Generalizations
Books
- Primary textbook:
- Bishop CM, Pattern Recognition and Machine Learning, Springer, 2006.
- Supplementary readings:
- Jordan MI, An Introduction to Probabilistic Graphical Models, in preparation
- Recommended reference books:
- Casella G, Berger RL, Statistical Inference. Duxbury Press, 2001.
Lecture Schedule
(Lecture slides and homework assignments are password protected.)
| Date | Readings [optional]* | Topics | Slides |
|---|---|---|---|
| Aug 26 | 8.0-8.1; Jordan & Weiss review; Kevin Murphy tutorial | Course introduction | |
| Aug 31 | 1.2-1.3; 2.1-2.3 [1.1; 1.4-1.5; 2.5] | Probability and statistics background | |
| Sep 2 | 8.2; J2.0-2.1 | Conditional independence and factorization | |
| Sep 7 | 8.3; J2.2-2.4 [J16] | D-separation, undirected models | |
| Sep 9 | 8.4.0-8.4.2; J3 | Probabilistic inference by graph elimination | |
| Sep 14 | 8.4.3; J4 | Belief propagation / sum-product algorithm | |
| Sep 16 | 8.4.4-8.4.5; J5 | Max-product algorithm | |
| Sep 21 | J5-7 [3, 4] | Linear regression and classification | |
| Sep 23 | J8 | Exponential family, generalized linear models | |
| Sep 28 | J9 | Learning with completely observed models | |
| Sep 30 | J10-11 [9] | Expectation maximization (EM) | |
| Oct 5 | – | More on EM | |
| Oct 7 | 13.0-13.2; J12 | Hidden Markov models (HMMs) | |
| Oct 12 | – | Happy Fall Break! | – |
| Oct 14 | – | More on HMMs and related models | |
| Oct 19 | J13-14 | Finish HMMs; Factor analysis | |
| Oct 21 | 12 | PCA and probabilistic PCA | |
| Oct 26 | 13.3; J15 | Kalman filtering | |
| Oct 28 | Sutton & McCallum review; Original Lafferty et al. paper | Conditional random fields | |
| Nov 2 | 11.0-11.1 | Basic sampling | |
| Nov 4 | 11.2-11.6 | Introduction to MCMC | |
| Nov 9 | – | More on MCMC | |
| Nov 11 | – | Variational inference | |
| Nov 16 | Yedidia, Freeman, and Weiss: paper #1, paper#2 | More on variational inference, expectation propagation | |
| Nov 18 | J17 | Junction tree algorithm | |
| Nov 23 | – | More on junction tree algorithm | |
| Nov 25 | – | Happy Thanksgiving! | – |
| Nov 30 | – | Neural networks and Boltzmann machines | |
| Dec 2 | – | Learning graph structure |
*Prefix of "J" indicates Jordan's draft manuscript; other readings are from Bishop
Assignment Schedule
| Assignment | Date Assigned | Date Due | Topics | Data |
|---|---|---|---|---|
| HW#1 | Sep 4 | Sep 17 | Factorization, D-separation, graph elimination | – |
| HW#2 | Sep 18 | Oct 1 | Sum-product, max-product algorithms | column.fa multicolumn.fa tree.nh example-code.tgz |
| HW#3 | Oct 2 | Oct 18 | Fully observed learning, expectation maximization | survey-labeled.dat survey-unlabeled.dat |
| HW#4 | Oct 16 | Nov 5 | HMMs | riverdale.dat across-the-riverdale.dat |
| Proposal | Oct 29 | Nov 5 | Detailed project proposal | – |
| HW#5 | Nov 6 | Nov 19 | Image processing and Gibbs sampling | orig.txt orig.png noisy_10.txt noisy_10.png noisy_20.txt noisy_20.png image2Text.pl text2Image.pl computeError.pl image2Text.24bit.pl |
Recitation Notes
| Date | Section Number | Topics |
|---|---|---|
| Sep 1 | Section 1 | Probability and statistics background |
| Sep 8 | Section 2 | Factorization and D-separation |
| Sep 15 | Section 3 | Graph Elimination and Sum-Product Algorithms |
| Sep 22 | Section 4 | Sum-Product Algorithm and Logistic regression |
| Sep 29 | Section 5 | Learning with Completely Observed Models |
| Oct 6 | Section 6 | Expectation Maximization |
| Oct 13 | Section 7 | Expectation Maximization and HMMs |
