Mean Squared Error

def. Loss function is a function that encapsulates the “closeness” of the estimate and ground truth. For example a loss function may measured the geometric closeness by:

$$\mathbb{L}(\theta ,e)=(\theta -a{)}^2$$

def. Mean Squared Error (MSE) is the arithmetic mean of the loss function defined:

$$\begin{array}{rl}\text{MSE}(\hat{\theta })& ={\mathbb{E}}_X[(\hat{\theta }-\theta {)}^2]\\ & ={\int }_{x=-\infty }^{\infty }(\hat{\theta }-\theta {)}^2{f}_{X|\theta =\hat{\theta }}(x)\end{array}$$

Info

The following shows why MSE is useful; it is just a sum of variance and bias squared.

$$>{\mathbb{E}}_X[(\hat{\theta }-\theta {)}^2]=\text{Var}[\hat{\theta }]+\text{Bias}[\hat{\theta }{]}^2>$$