Moment (Probability)

→ An alternative way to get an estimator quick and dirty.

Method of Moments §

alg. Method of Moments (MOM):

let $X_1,\dots ,X_n\stackrel{iid}{\sim }f(x;{\theta }_1,\dots ,{\theta }_p)$. Then to get estimators for ${\theta }_1,\dots ,{\theta }_p$:

1. let ${\mu }_i=E[X^i]=g_i({\theta }_1,\dots ,{\theta }_n)$
2. gather data on ${\bar{X}}^i$ to get an emperical estimate $m_i$

Observe that because $E[X^i]\approx {\bar{X}}^i$, this means that ${\mu }_i\approx m_i$

1. let function $g_i$ which maps ${\theta }_1,\dots ,{\theta }_n\stackrel{g_i}{\mapsto }{m}_i$ can be inverted
2. Get system of equations for as many $i$’s as necessary
3. Solve the system of equations as ${\hat{\theta }}_i=g_i^{-1}(m_1,\dots )$

Limitations on MoM:

• Estimators may not make sense (negative numbers, complex number, etc.)
• Illegal

def. Moment Generating Functions

(alternative specification of pdf. Makes it easy to calculate things.)

Moment generating functions are defined as such:

$$M_X(t)=E[\exp (tX)]$$

…where $M_X$ is differentiable $k$-times around zero to be able to generate the $k$-th moment.

You can build an MGF from a pmf of pdf:

• for pmfs: $M_X(t)={\sum }_{}^{}{e}^{t{x}_i}p(x_i)$
• for pdfs: $M_X(t)={\int }_{-\infty }^{\infty }{e}^{tx}f(x)dx$

It can generate the $k$-th moment like such:

$$E(X^k)=\frac{{\partial }^k}{\partial x^k}{M}_X(0)=M_X^{(k)}(0)$$

MGFs are unique to each distribution. If an RV has the same MGF, they are distributed identically.

thm. Linear Combination of MGFs. Given r.v. $W=X_1,\dots ,X_n$, and $X_i$ are iid,:

$$M_W(t)=M_{X_1}\cdot M_{X_2}\cdots M_{X_n}$$