def. Statistic. let observable random variables [= data of an experiment]. then statistic is:

  • i.e. is a real-valued function
  • cannot contain unknown variables

def. Estimator [= point estimate] is a statistic used to estimate the parameter of the model we think the data is showing. Note the following notation convention:

  • Assume as an r.v. of an experiment, whose model includes parameter .
  • To estimate ground truth parameter , we can use an estimator r.v.
  • A specific estimate for a particular observed value is denoted
  • An estimator has to be a function of known variables & data only.
  • , NOT ← This is MSE

How Good is Your Estimator?

  1. Accuracy is higher. Increased as Bias (Statistics) is decreased
  2. Precision is higher. Increased as Variance is decreased
  3. Efficiency (Statistics) is higher. If estimators have the same accuracy, but then the former is more efficient than the latter.
  4. Consistency.
  5. Mean Squared Error is lower.
  6. Likelihood (Statistics) is higher.

→ In general, making sure to reduce bias of estimators is important. Note that:

  • If you can write down what the bias is mathematically [= characterize the bias], then you can make a new estimator that doesn’t have the bias.
  • Bias usually decreases as the data points increase


let and let estimator where

  • are weights that sum to 1. [= weighted average]
  • is estimating . is known.

  1. How accurate is ? [=what is the bias?]

  1. How precise is ? What are the best ?

→ Thus is minimized when .