Fisher information helps us find better estimators.

  1. Cramer-Rao Lower Bound shows what the best estimators can do with their precision
  2. Reaching the CRLB means it’s finite-sample efficient
  3. The Maximum Likelihood Estimator is also a very good estimator.

def. Fisher Information is the amount of information we have about the unknown parameter. It’s the Expected Value of score.

  • Given , if has a high peak we may assume that carries a lot of information about .
  • If is spread out a lot, we may assume that carries little information. Thus:
  • (2) → (3) as we know that

thm. Addition of fisher information. if , then:

thm. Score and Fisher Information. if is twice differentiable wrt , and under certain regularity conditions:

  • Knowing this we also know that .