Bayesian Data Analysis course

This book serves as a collection of my notes and exercises completed for the Bayesian Data Analysis course taught by Aki Vehtari. It is normally taught as CS-E5710 at Aalto University, but the lectures and assignments have been made freely available online and the course is based around the text book Bayesian Data Analysis (3rd ed.) by Gelman et al (Gelman et al. 2013).

The source code for my course work is available on GitHub: jhrcook/bayesian-data-analysis-course. If you see any problems (big or small), please let me know by opening an Issue. All R Markdown files are self-contained an can be executed on their own (i.e. do not need to be run in a series or specific order).

0.2 How to study

The following are recommendations from the course creators on how to take the course.

The recommended way to go through the material is:

  1. Read the reading instructions for a chapter in the chapter notes.
  2. Read the chapter in BDA3 and check that you find the terms listed in the reading instructions.
  3. Watch the corresponding video lecture to get explanations for most important parts.
  4. Read corresponding additional information in the chapter notes.
  5. Run the corresponding demos in R demos or Python demos.
  6. Read the exercise instructions and make the corresponding assignments. Demo codes in R demos and Python demos have a lot of useful examples for handling data and plotting figures. If you have problems, visit TA sessions or ask in course slack channel.
  7. If you want to learn more, make also self study exercises listed below.

0.3 Course sections

Below are the main sections of the course where each section should take about a week to complete.

  1. Course Introduction
  2. Basics of Bayesian Inference
  3. Multidimensional Posterior
  4. Monte Carlo
  5. Markov chain Monte Carlo
  6. HMC, NUTS, and Stan
  7. Hierarchical models and exchangeability
  8. Model checking & Cross-validation
  9. Model comparison and selection
  10. Decision analysis
  11. Normal approximation & Frequency properties
  12. Extended topics

0.4 Additional notes

I took additional notes on other chapters in the book. I appended these chapters after the primary course sections.

  1. Ch. 14 Introduction to regression models
  2. Ch. 15 Hierarchical linear models
  3. Ch. 19 Parametric nonlinear models
  4. Ch. 20 Basis function models
  5. Ch. 21 Gaussian process models
  6. Ch. 22 Finite mixture models
  7. Ch. 23 Dirichlet process models

I plan to also read through and take notes on chapters 16 and 18 on “Generalized linear models” and “Models for missing data” in the future.

0.5 Assignments and exercises

I completed the assignments that accompany the course and also did some of additional exercises from the book. These are available in their own sections of the book: Assignments and Exercises.

0.6 Stan models

Below is a list of the models built during the course. The original code is available in the GitHub repo “models” directory and they are also copied in the Models section of this website.

References

Gelman, Andrew, John B Carlin, Hal S Stern, David B Dunson, Aki Vehtari, and Donald B Rubin. 2013. Bayesian Data Analysis, Third Edition. 3rd ed. CRC Press.