Inference

Largely the inference material comes from (Casella and Berger 2002), though some of the later theory around point-estimation and hypothesis testing comes from (E. L. Lehmann and Romano 2022; Erich L. Lehmann and Casella 1998). Unfortunately, for much of the material towards the end (on the asymptotic properties of maximum likelihood estimates and the Wald, Score, and Likelihood Ratio Tests) the course materials refer heavily to notes from Robert Gray (google scholar), and I don’t believe his notes are publicly available, nor do I have permission to share them. As such, I’ll try to find corroborating references for the same material when I get to that point.

Outline of Inference:

  1. Introduction
    • Identifiability
    • Scale Families
    • Exponential Families
    • Moments
  2. Principles of Data Reduction
    • Data Reduction
    • Sufficiency
      • Order Statistics
      • Factorization Theorem
      • Minimal Sufficiency
      • Ratio of Densities
    • Ancillary Statistics
      • Location-Scale Families
      • Conditioning / Conditionality Principle
    • Complete Families
      • Basu’s Theorem
    • Likelihood Principle
  3. Methods of Estimation
    • Method of Moments
    • Maximum Likelihood
      • Multiple Parameters
      • Invariance Property
      • Exponential Families
    • Other Methods
  4. Evaluating Estimators
    • Mean Squared Error and Bias-Variance Tradeoff
    • Best Unbiased Estimation
      • Cramér-Rao Inequality
      • Rao-Blackwell
      • Complete Families
  5. Asymptotic Distributions
    • Consistency
    • Convergence in Distribution
      • Slutsky’s Theorem
      • Delta Method
      • Lyapunov Central Limit Theorem
    • Multivariate Distributions
      • Asymptotics
  6. Intro to Hypothesis Testing
    • Basic Concepts
    • Likelihood Ratio Test
    • Type I and Type II Error Rate
    • P-Values
  7. Theory of Optimal Tests
    • UMP Tests
      • Neyman-Pearson Lemma
      • Randomized Tests
      • More on Neyman-Pearson
      • Karlin-Rubin
      • Two-Sided Alternatives
    • Locally Most Powerful Tests
    • Multiple Parameters
  8. Distribution of MLEs and Asymptotic Efficiency
    • Notation and Assumptions
      • Multiple Parameters
    • Consistency of MLEs
    • Regularity Conditions
    • Asymptotic Distribution of MLEs
      • Non-IID Data
      • Multi-Parameter Models
    • Asymptotic Relative Efficiency
    • Asymptotic Efficiecny
    • Asymptotic Confidence Intervals
  9. Likelihood Ratio, Score, and Wald Tests
    • Likelihood Ratio, Score, and Wald Tests
    • Multidimensional Parameters
    • Composite Null

Some particularly nice looking inference course materials are available online from Stanfords STATS 200: https://web.stanford.edu/class/archive/stats/stats200/stats200.1172/lectures.html

References

Casella, George, and Roger L Berger. 2002. Statistical Inference. Vol. 2. Duxbury Pacific Grove, CA.
Lehmann, E. L., and Joseph P. Romano. 2022. Testing Statistical Hypotheses. Cham: Springer. https://doi.org/10.1007/978-3-030-70578-7.
Lehmann, Erich L., and George Casella. 1998. Theory of Point Estimation. 2nd ed. New York: Springer.