Recall Exponential Family

Sufficiency

def. let statistic ; and depends on unknown parameter . Then is a sufficient statistic for if does not depend on .

Info

If the distribution of ’s depends on unknown parameter , but the distribution of given does not depend on , it must be the case that all information about is in . Once the value of is given, then becomes irrelevant in determining the value of .

Info

There’s only one minimally sufficient statistic, and all other sufficient staistics are a function of this.

thm. Fischer-Neymann factorization. is a sufficient statistic iff:

i.e. the joint density function can be factored into a a function of where:

  • does not depend on
  • depends on , and depend on only through statistic ⇒ Intuitively, this means when

Info

Very convenient for determining if statistic is sufficient or not.

lem 1. let an estimator for and a sufficient statistic for

then is also a sufficient estimator.

thm. Rao-Blackwell theorem. Continuing from above,

Info

i.e. if you throw more data into a statistic, it often becomes a better statistic.

  • In the exponential family of distributions, you can mostly do one iteration of Rao-Blackwell algorithm to get a pretty good estimator.
    • Bias of and Bias of Rao-Blackwellized is the same
  • If is a minimally sufficient statistic, is the best you will do