MATH5835M — Statistical Computing

Syllabus
Lecture Notes
Handouts
Software
- Resources
- R Code from Lectures (live and recorded)
References
Links

Syllabus

Statistical computing is the branch of computational mathematics which studies computational techniques for situations which either directly involve randomness, or where randomness is used as part of a mathematical method. This module gives an introduction to statistical computing, with a focus on Monte Carlo methods. The following topics will be covered:

Monte-Carlo methods
Random number generation
Markov Chain Monte Carlo (MCMC) methods
Resampling methods
Implementation of different methods in R

Lecture Notes

In place of lecture notes, we will use the book An Introduction to Statistical Computing: A Simulation-Based Approach (see References, below), which was specially written for this module. The book is available online via the university library:

access the book

We will cover sections 3.1-3.3, 1.1, 1.3, 1.4, 2.3, 4.1, 4.2 and 5.2 of this book. The exact page ranges are:

videos	sections	pages	topic
1, 2, 3, 4	3.1	69--74	Monte Carlo methods
5, 6, 7	3.2	74--84	Monte Carlo error
8, 9	3.3.1	84--88	Importance Sampling
10, 11, 12	3.3.2	88--93	Antithetic Variables method
13, 14	3.3.3	93--96	Control Variates method
15, 16, 17	1.1	1--8	Pseudo Random Number Generators
18, 19	1.3	11--15	Inverse Transform method
20, 21, 22	1.4.1	15--18	basic rejection sampling
23, 24, 25	1.4.2	18--22	envelope rejection sampling
26, 27, 28	2.3	50--58	Markov Chains
29, 30, 31	4.1.1--4.1.2	110--116	Metropolis-Hastings (MH) algorithm
32, 33, 34	4.1.3--4.1.4	116--120	special cases of the MH algorithm
35, 36, 37	4.2.2	129--137	Convergence of MCMC estimates
38 -- 43	(intro of 4.3)	137--139	Application to Bayesian Inference
44, 45, 46	5.2.1	192--197	Bootstrap sampling
47, 48	5.2.2.1, 5.2.2.2	198--203	Applications to statistical inference

Handouts

The following links contain pdf copies of the handouts from the lectures.

Software

For the module we will use the statistical computing package R. This program is free software, and I would recommend that you install R on your own laptop. There are different versions of R available:

R itself, together with a lot of additional information, can be found on the R project homepage.
A more polished version is RStudio, which can be found at the RStudio homepage. (Choose the open source version, RStudio Desktop, on the download page.)

Alternatively you can use RStudio or plain R on the university computers.

Resources

Useful resources for learning R include to following:

The stats department provides a short, four-page introduction to R. This covers, amongst other things, how to start R on the university's computers and has some hints on how to install R on your own computer.
The basics of R are explained in a bit more detail in my Short Introduction to R.
The official R manual contains a lot of information.
The R online help, accessed by typing help() or help.start() in R, can be used to remind yourself about indivdual commands.
An R tutorial can be found in appendix B of my book An Introduction to Statistical Computing: A Simulation-Based Approach.

R Code from Lectures (live and recorded)

References

The main reference for the module is the following book:

Jochen Voss,
An Introduction to Statistical Computing: A Simulation-Based Approach.
Wiley, 2014 (Library, Amazon)

More in-depth information, beyond what we will be able to cover in the lectures, is for example contained in the following texts.

Maria L. Rizzo,
Statistical Computing with R.
Chapman & Hall/CRC, 2008 (Library, Amazon)
Brian D. Ripley,
Stochastic Simulation.
Wiley, 1987 (Library, Amazon)
Christian P. Robert and George Casella,
Monte Carlo Statistical Methods.
Springer, 2004 (Library, Amazon)
Wally R. Gilks, Silvia Richardson and David J. Spiegelhalter,
Markov Chain Monte Carlo in Practice.
Chapman & Hall/CRC, 1995 (Library, Amazon)
Anthony C. Davison and David V. Hinkley,
Bootstrap methods and their application.
Cambridge University Press, 1997 (Library, Amazon)
Andrew Gelman, et al.,
Bayesian Data Analysis.
Chapman & Hall/CRC, 3rd edition, 2013 (Library, Amazon)