Fisher information helps us find better estimators.
- Cramer-Rao Lower Bound shows what the best estimators can do with their precision
- Reaching the CRLB means it’s finite-sample efficient
- The Maximum Likelihood Estimator is also a very good estimator.
def. Fisher Information is the amount of information we have about the unknown parameter. It’s the Expected Value of score.
- Given , if has a high peak we may assume that carries a lot of information about .
- If is spread out a lot, we may assume that carries little information. Thus:
- (2) → (3) as we know that
thm. Addition of fisher information. if , then:
thm. Score and Fisher Information. if is twice differentiable wrt , and under certain regularity conditions:
- Knowing this we also know that .