Expected Value

Expected Value §

def. Expected Value. For random variable $X$ with countably many outcomes, its expected value $\mathbb{E}(X)$ is defined as:

$$\begin{array}{rlr}\mathbb{E}(X)& :=\sum _{\forall x\in \text{range(X)}}x\cdot \mathbb{P}(X)& \text{for discrete}\\ \mathbb{E}(X)& :={\int }_{-\infty }^{\infty }x\cdot f_X(x)dx& \text{for continuous}\end{array}$$

Properties. The following identities hold for expected values, with constant $k$, random variables $X,Y$.

• $\mathbb{E}(k)=k$
• $\mathbb{E}(X+Y)=\mathbb{E}(X)+\mathbb{E}(Y)$ Linearity
• $\mathbb{E}(k\cdot X)=k\cdot \mathbb{E}(X)$
• $\mathbb{E}(X\cdot Y)={\sum }_{\forall z}z\cdot \mathbb{P}(X\cdot Y=z)$
  • If $X\perp Y$ then $\mathbb{E}(X\cdot Y)\Rightarrow \mathbb{E}(X)\cdot \mathbb{E}(Y)$ (reverse does not hold)
• let $g$ be a function over $\text{range}(X)$. Then, $\mathbb{E}(g(x))={\sum }_{\forall x}g(x)\cdot \mathbb{P}(X=x)$ (Law of Unconscious Statistician)
  • ! $\mathbb{E}(g(x))\ne g(\mathbb{E}(x))$
  • $\mathbb{E}(X^k)={\sum }_{\forall x}{x}^k\cdot \mathbb{P}(X=x)$

thm. Tail Sum Formula. when $X$ is a non-negative discrete random variable:

$$\mathbb{E}(X)=\sum _{i=0}^{\infty }\mathbb{P}(X\ge i)$$

Remark. The Tail Sum formula is useful when the random variable is defined as the minimum or maximum of a certain set of events (e.g. minimum of multiple dice rolls, etc.)

Expectation Manipulation from class:

Conditional Expected Value §

$$\mathbb{E}(g(X,Y))={\iint }_{{\mathbb{R}}^2}g(x,y)f_{X,Y}(x,y)dA$$

def. Conditional Expectation. let $X,Y$ be jointly distributed. Then the conditional expected value is defined…

1. …over an event: $\mathbb{E}(X|A)={\sum }_{\forall A}x\cdot \mathbb{P}(X=x|A)$
2. …over an event on a random variable $\mathbb{E}(X|Y=y)={\int }_{-\infty }^{\infty }x\cdot f_{X|Y=y}(x)dx$
3. …over a random variable: $\mathbb{E}(X|Y):=\mathbb{E}(X|\sigma (Y))={\sum }_{\forall x\forall y}x\cdot \mathbb{P}(X=x|Y=y)$
   • ! While expectation conditioned on an event is a value, an expectation conditioned over a random variable is another random variable
   • Intuition. Think of it as “given all information by $Y$, what’s the new random variable?”

• $\mathbb{E}(aX+bY|A)=a\cdot \mathbb{E}(X|A)+b\cdot \mathbb{E}(Y|A)$ linearity
• $\mathbb{E}(X)={\sum }_{\forall i}\mathbb{E}(X|A_i)\cdot \mathbb{P}(A_i)$ where $A_1,\dots ,A_n$ is a partition of $\Omega$.weighted summation thm. Conditional Joint Expectation. $X,Y$ and $R\in [\text{Range(X)}\times \text{Range}(Y)]$. Then:

$$\begin{array}{rl}\mathbb{E}(X|x,y\in R)& ={\iint }_{R}x\cdot f_{X,Y|x,y\in R}(x,y)dA\\ \text{where}{f}_{X,Y|x,y\in R}(x,y)& =\frac{f_{X,Y}(x,y)}{\mathbb{P}(x,y\in R)}\end{array}$$

thm. Calculating Expected Value from Conditional Expected Value. (Identity 2 above) let $X,Y$ be jointly distributed. Then the expected value of $X$ is calculated:

$$\begin{array}{rl}\mathbb{E}(X)& =\sum _{\forall y}\mathbb{E}(X|Y=y)\cdot \mathbb{P}(Y=y)=\mathbb{E}(\mathbb{E}(X|Y))\\ \mathbb{E}(X)& ={\int }_{-\infty }^{\infty }\mathbb{E}(X|Y=y)\cdot f_Y(y)\cdot dy=\mathbb{E}(\mathbb{E}(X|Y))\end{array}$$

• Useful for computing $\mathbb{E}(X)$ when $X$ depends on $Y$.
• Works regardless of whether $X,Y$ are random or discrete, and when mixed.

…for Stochastic Calculus §

Intuition. Conditioning is done only by two objects: a sigma algebra, or an event. Each are defined:

$$\begin{array}{rlr}\mathbb{E}[X|A]& ={\int }_{A}XdP& \text{A Number}\\ \mathbb{E}[X|\mathcal{F}]& =\{\begin{array}{l}⋮\\ \mathbb{E}[X|A]\text{ for every }\text{atom }A\in \mathcal{F}\\ ⋮\end{array}& \text{A New Random Variable}\end{array}$$

In both cases, it’s useful to think of $\mathbb{E}$ as the best guess.

\mathbb{E}[X|Y] on an RV, it's shorthand for \mathbb{E}[X|\sigma(Y)].

def. Measurability. R.V. $X$ is $\mathcal{F}$-measurable iff:

$$\mathbb{E}[X|\mathcal{F}]=X$$

Intuition. Looking at the above definition, it simply means $\mathbb{E}[X|\mathcal{F}]$ also takes the same value for each atom of $\mathcal{F}$ as $X$.

thm. For any R.V. and filtration $X,\mathcal{G}$, $\mathbb{E}[X|\mathcal{G}]$, which is a random variable, is $\mathcal{G}$-measurable.

def. Independence. R.V. $X$ is independent from $\sigma$-alg $\mathcal{G}$ iff:

$$\mathbb{E}[X|\mathcal{G}]=\mathbb{E}[X]$$

i.e. giving the information on $\mathcal{G}$ has made zero difference Properties.

• If $X$ is independent of $\mathcal{H}$ then $\mathbb{E}(X|\mathcal{H})=\mathbb{E}(X)$ Taking out independent factors.
• if $X$ is $\mathcal{F}$-measurable, $\mathbb{E}(XY|\mathcal{F})=X\mathbb{E}(Y|\mathcal{F})$ Taking out what’s known
  • e.g. $\mathbb{E}(X_{t+0.2}{X}_t\mid \mathcal{F})=X_t\mathbb{E}(X_{t+0.2})$
• Tower Property: $\mathbb{E}(\mathbb{E}(X|\mathcal{F}))=\mathbb{E}(X)$