Expected Value

Expected Value §

def. Expected Value. For random variable $X$ with countably many outcomes, its expected value $\mathbb{E}(X)$ is defined as:

$$\begin{array}{rlr}\mathbb{E}(X)& :=\sum _{\forall x\in \text{range(X)}}x\cdot \mathbb{P}(X)& \text{for discrete}\\ \mathbb{E}(X)& :={\int }_{-\infty }^{\infty }x\cdot f_X(x)dx& \text{for continuous}\end{array}$$

Properties. The following identities hold for expected values, with constant $k$, random variables $X,Y$.

• $\mathbb{E}(k)=k$
• $\mathbb{E}(X+Y)=\mathbb{E}(X)+\mathbb{E}(Y)$ Linearity
• $\mathbb{E}(k\cdot X)=k\cdot \mathbb{E}(X)$
• $\mathbb{E}(X\cdot Y)={\sum }_{\forall z}z\cdot \mathbb{P}(X\cdot Y=z)$
  • If $X\perp Y$ then $\mathbb{E}(X\cdot Y)\Rightarrow \mathbb{E}(X)\cdot \mathbb{E}(Y)$ (reverse does not hold)
• let $g$ be a function over $\text{range}(X)$. Then, $\mathbb{E}(g(x))={\sum }_{\forall x}g(x)\cdot \mathbb{P}(X=x)$ (Law of Unconscious Statistician)
  • ! $\mathbb{E}(g(x))\ne g(\mathbb{E}(x))$
  • $\mathbb{E}(X^k)={\sum }_{\forall x}{x}^k\cdot \mathbb{P}(X=x)$

thm. Tail Sum Formula. when $X$ is a non-negative discrete random variable:

$$\mathbb{E}(X)=\sum _{i=0}^{\infty }\mathbb{P}(X\ge i)$$

Remark. The Tail Sum formula is useful when the random variable is defined as the minimum or maximum of a certain set of events (e.g. minimum of multiple dice rolls, etc.)

Expectation Manipulation from class:

Conditional Expected Value §

$$\mathbb{E}(g(X,Y))={\iint }_{{\mathbb{R}}^2}g(x,y)f_{X,Y}(x,y)dA$$

def. Conditional Expectation. let $X,Y$ be jointly distributed. Then the conditional expected value is defined…

• …over an event: $\mathbb{E}(X|A)={\sum }_{\forall A}x\cdot \mathbb{P}(X=x|A)$
• …over an event on a random variable $\mathbb{E}(X|Y=y)={\int }_{-\infty }^{\infty }x\cdot f_{X|Y=y}(x)dx$
• …over a random variable: $\mathbb{E}(X|Y)={\sum }_{\forall x\forall y}x\cdot \mathbb{P}(X=x|Y=y)$
  • ! While expectation conditioned on an event is a value, an expectation conditioned over a random variable is another random variable
  • with more rigour: $\mathbb{E}(X|Y):=\mathbb{E}(X|\sigma (Y))$
  • Intuition. Think of it as “given all information by $Y$, what’s the new random variable?”

Properties.

• $\mathbb{E}(aX+bY|A)=a\cdot \mathbb{E}(X|A)+b\cdot \mathbb{E}(Y|A)$ linearity
• $\mathbb{E}(X)={\sum }_{\forall i}\mathbb{E}(X|A_i)\cdot \mathbb{P}(A_i)$ where $A_1,\dots ,A_n$ is a partition of $\Omega$.weighted summation
• If $X$ is $\mathcal{F}$-measurable, $\mathbb{E}(X\mid \mathcal{F})=X$
• If $X$ is independent of $\mathcal{H}$ then $\mathbb{E}(X|\mathcal{H})=\mathbb{E}(X)$ Taking out independent factors.
• if $X$ is $\mathcal{F}$-measurable, $\mathbb{E}(XY|\mathcal{F})=X\mathbb{E}(Y|\mathcal{F})$ Taking out what’s known
  • e.g. $\mathbb{E}(X_{t+0.2}{X}_t\mid \mathcal{F})=X_t\mathbb{E}(X_{t+0.2})$
• Tower Property: if $X$ is a random variable, and $\mathcal{F}\subset \mathcal{G}$ then: $\mathbb{E}(\mathbb{E}(X|\mathcal{G})|\mathcal{F})=\mathbb{E}(X|\mathcal{F})$
  • & Think of it as “high-res camera” then “low-res camera”; the final picture is low-res.

thm. Conditional Joint Expectation. $X,Y$ and $R\in [\text{Range(X)}\times \text{Range}(Y)]$. Then:

$$\begin{array}{rl}\mathbb{E}(X|x,y\in R)& ={\iint }_{R}x\cdot f_{X,Y|x,y\in R}(x,y)dA\\ \text{where}{f}_{X,Y|x,y\in R}(x,y)& =\frac{f_{X,Y}(x,y)}{\mathbb{P}(x,y\in R)}\end{array}$$

thm. Calculating Expected Value from Conditional Expected Value. (Identity 2 above) let $X,Y$ be jointly distributed. Then the expected value of $X$ is calculated:

$$\begin{array}{rl}\mathbb{E}(X)& =\sum _{\forall y}\mathbb{E}(X|Y=y)\cdot \mathbb{P}(Y=y)=\mathbb{E}(\mathbb{E}(X|Y))\\ \mathbb{E}(X)& ={\int }_{-\infty }^{\infty }\mathbb{E}(X|Y=y)\cdot f_Y(y)\cdot dy=\mathbb{E}(\mathbb{E}(X|Y))\end{array}$$

• Useful for computing $\mathbb{E}(X)$ when $X$ depends on $Y$.
• Works regardless of whether $X,Y$ are random or discrete, and when mixed.