Simple Definition

def. Confidence Interval of the parameter in the probability distribution of a random variable is defined as the following:

  • where functions are able to be derived from the random variable .

The most common use of confidence interval problems is with CLT, where a parameter of a binomial distribution is estimated. e.g.: let where is the parameter to be estimated.

Formal Definition

def. Confidence Interval. For distributed with an unknown parameter , a -level confidence interval:

  • …is an interval in which the probability of being in the interval is [-level CI]
    • think of big is good, small is good.
  • …is formalized as the following where is a statistic of :

def. Observed Information. For , observed information is:

And for

Remark. This is Fisher Information, but gathered on data ()

thm. (approximating a confidence interval using Fisher Information) In general, if can be found and the log-likelihood is twice-differentiable, an approximate CI of level can be approximated by:

thm Delta Method. Let unknown parameter of r.v. ’s distribution. If Fisher’s approximation [=asymptotically efficient and asymptotically unbiased] conditions are satisfied, the following holds for all which :

and due to Fisher’s Approximation (asymptotic normality):

Example. To construct a standard normal distribution’s confidence interval of level :