Probability
The probability material can be mostly found in (Casella and Berger 2002).
For the probability-focused material, we may also refer to a handful of other references including (Blitzstein and Hwang 2019; Bishop 2006; Gut 2009; Stoyanov 2013; Feller 1968; DasGupta 2011; Durrett 2019).
At a high-level, how do we distinguish Probability and Statistics?
In general, these are answers to dual problems: Probability theory gives us the tools to carry out deductive reasoning, where based on perfect, complete knowledge of random processes, we can derive true statements about the result (e.g., statements about expectation, variance, etc.). On the other hand, statistical inference allows us to go from observed data and combine it with assumptions to draw conclusions inductively, which may or may not be correct. For example, a statistical statement might be that a particular way of constructing a confidence interval (under assumptions) has a 95% probability of containing the true population parameter when repeatedly performing the same experiment (e.g., in the Frequentist paradigm).
Outline of Probability:
- Introduction/Background
- Probability Basics
- Random Variables
- Expected Values
- Families of Distributions
- Multiple Random Variables
- Independence and Conditional Expectation
- Bivariate Dependence
- Inequalities
- Multivariate Normal
- Statistics of Random Samples
- Laws of Large Numbers and Central Limit Theorem
- Stochastic Processes