Applications of Discrete Markov Chains

Gamber’s Ruin §

game. Gambler’s Ruin. Gambler starts out with $i$ units. They have probability $p$ of earning one unit, $1-p$ losing one unit, iid per round. Game ends when gambler reaches either $0$ (lose) or a preset $N$ (win). What’s the probability the gambler wins/loses the game? let $P_i=\mathbb{P}(\text{reaches }N\mid X_0=i)$. Then:

1. $P_0=0$, $P_N=1$
2. $P_i=p{P}_{i-1}+(1-p)P_{i+1}$ Then identity: $P_i=p{P}_i+(1-p)P_i$ rearrange the second into the following:

$$\begin{array}{rlr}{P}_2-P_1& =\frac{1-p}{p}(P_1-P_0)& =\frac{1-p}{p}{P}_1\\ P_3-P_2& =\frac{1-p}{p}(P_3-P_2)& ={\left(\frac{1-p}{p}\right)}^2{P}_1\\ ⋮& & \\ P_i-P_{i-1}& =\frac{1-p}{p}(P_{i+1}-P_i)& ={\left(\frac{1-p}{p}\right)}^{i-1}{P}_1\end{array}$$

Summing all and canceling:

$$\begin{array}{rl}{P}_i-P_1& =\left(\frac{1-p}{p}+{\left(\frac{1-p}{p}\right)}^2+\dots {\left(\frac{1-p}{p}\right)}^{i-1}\right)P_1\\ P_i& =\{\begin{array}{ll}\frac{1-{\left(\frac{1-p}{p}\right)}^i}{1-\frac{1-p}{p}}{P}_1& \text{if}\frac{1-p}{p}\ne 1\\ i{P}_1& \text{if}\frac{1-p}{p}=1\end{array}\end{array}$$

Try substituting $i=N$ for boundary condition $P_N=1$:

$$\begin{array}{rl}1=P_N& =\{\begin{array}{ll}\frac{1-{\left(\frac{1-p}{p}\right)}^N}{1-\frac{1-p}{p}}{P}_1& \text{if}\frac{1-p}{p}\ne 1\\ N\cdot P_1& \text{if}\frac{1-p}{p}=1\end{array}\end{array}$$

• $P_1=\frac{1}{N}$ if $1-p=p$
• $P_1=\frac{1-(1-p)/p}{1-((1-p)/p{)}^N}$ if $1-p\ne p$ Thus the general formula:

$$P_i=\{\begin{array}{ll}\frac{1-((1-p)/p{)}^i}{1-((1-p)/p{)}^N}& \text{if }p\ne 1-p\\ \frac{i}{N}& \text{if }p=1-p=\frac{1}{2}\end{array}$$

Hypothesis Testing For a Drug §

This can totally be used to construct a hypothesis test in the following way: