Confidence Intervals and Hypothesis Testing in OLS Linear Regression

See also: Confidence Intervals and Hypothesis Testing

Motivation. Assume we have our estimators for our sample size $N$ using OLS, $\widehat{{\beta }_0,\hat{{\beta }_1}}$. Now, assuming we have the true population data (impossible in real life) and take $100$ samples of size $N$ from the whole population, we get $100$ different tuple of estimators $(\widehat{{\beta }_0},\widehat{{\beta }_1})$. If we plot these on a graph, we get an approximate bell curve. This is due to the Central Limit Theorem. Knowing this fact, we can deduce if there is a correlation between $X$ and $Y$.

Remark. $N\ge 30$ is the minimum required for CLT. $N\ge 100$ is a conservative requirement for CLT to apply. Remark. We will only look at $\widehat{{\beta }_1}$ since it is the more important parameter.

Hypothesis Testing §

def. The Null hypothesis in regression is $H_0:{\beta }_1=0$, i.e. there is no correlation.

def. Regression T-test. See Student’s t-test. A T-test is a test for rejecting the null hypothesis. let the T-statistic $T=\frac{\widehat{{\beta }_1}-{\beta }_1^{\text{Null}}}{\sqrt{\text{Var}(\widehat{{\beta }_1})}}$. Then

$$\{\begin{array}{ll}{H}_0& \text{if }|T|>K\\ H_1& \text{otherwise}\end{array}$$

• The cutoff value $K$ is determined by how powerful (=$\alpha$) you want the test to be. This is determined by the Student’s T-Distribution.
  • This is because $T{=}_d{t}_{N-1}$
• This is Student’s t-test but with only one random variable.
• Normally, we set the cutoff $K=2$, i.e. two standard deviations away. This is around an $\alpha =0.05$ test.

Table of common T-Statistic values

Value of T	Significance	Sidedness
$\Vert T\Vert >1.95$	$\mathbb{P}({\beta }_1=0)<0.05$	Two-side
$\Vert T\Vert >2.58$	$\mathbb{P}({\beta }_1=0)<0.01$	Two-side

A Large-sample critical value is simply using the idea that as $N\to \infty$ the $T{\to }_d^{N\to \infty }\mathcal{N}$ i.e. the $T$ statistic becomes a normal distribution. Thus the cutoff values are using the standard normal table.

Intuition.

Confidence Interval §

Intuition.

Bias:

standard error of regression: mean squared of residuals; also the estimator for the error

Standard error of…

• Residuals → standard error of regression
• $be$ Varaince of $\widehat{{\beta }_1}$ is also the variance of CLT limit with the same sample size

Remark. Substantive significance does not imply statistical significance, nor does vice versa. The two are irrelevant.