Recall Exponential Family
Sufficiency
def. let statistic ; and depends on unknown parameter . Then is a sufficient statistic for if does not depend on .
Info
If the distribution of ’s depends on unknown parameter , but the distribution of given does not depend on , it must be the case that all information about is in . Once the value of is given, then becomes irrelevant in determining the value of .
Info
There’s only one minimally sufficient statistic, and all other sufficient staistics are a function of this.
thm. Fischer-Neymann factorization. is a sufficient statistic iff:
i.e. the joint density function can be factored into a a function of where:
- does not depend on
- depends on , and depend on only through statistic ⇒ Intuitively, this menas when
Info
Very convenient for determining if statistic is sufficient or not.
lem 1. let an estimator for and a sufficient statistic for
then is also a sufficient estimator.
thm. Rao-Blackwell theorem. Continuing from above,
Info
i.e. if you throw more data into a statistic, it often becomes a better statistic.
- In the exponential family of distributions, you can mostly do one iteration of Rao-Blackwell algorithm to get a pretty good estimator.
- Bias of and Bias of Rao-Blackwellized is the same
- If is a minimally sufficient statistic, is the best you will do