# ML Wiki

## Relationship between Confidence Intervals and Tests

How to connect Hypothesis Testing and Confidence Intervals?

## Example

Observations:

• $n = 1000$
• $\hat{p} = 0.576$
• under $H_0$, is difference $| \hat{p} - p | = | \hat{p} - 0.5 | = 0.076$ too large to reject H_0?

We calculate $p$-value

$P(|\hat{p} - 0.5| \geqslant 0.076) \approx P(|N(0, 1)| \geqslant 4.81 ) \approx 1 / 663000 \leqslant 0.05$

And reject the $H_0$ because the $p$-value is small

Now let's calculate 95% CI:

• $\hat{p} \pm 1.96 \sqrt{p(1-p)/n} = [0.532, 0.620]$
• The CI misses 0.5
• and this is not a coincidence!

## General Test

Suppose we have a test of the following form

• $H_0: \mu = \mu_0, H_A: \mu \neq \mu_0$
• Our observations are: $n$, $\bar{X}$, $s$,
• We're interested in the difference between observed mean and the true mean:
$\Delta = |\bar{X} - \mu_0|$

So we reject $H_0$ if

• $P(|\bar{X} - \mu_0| \geqslant \Delta) \leqslant \alpha$
• $P\left(\left| \cfrac{\bar{X} - \mu_0}{\sqrt{s^2 / n}} \right| \geqslant \cfrac{\Delta}{\sqrt{s^2 / n}} \right) = P\left(\left| t_{n - 1} \right| \geqslant \cfrac{\Delta}{\sqrt{s^2 / n}} \right) \leqslant \alpha$

This will only happen if

• $\cfrac{\Delta}{\sqrt{s^2 / n}} \geqslant T_{\alpha/2, n-1}$
• where $T_{\alpha/2, n-1}$ is critical value s.t.
$P(|t_{n-1}| \geqslant T_{\alpha/2, n-1} ) = \alpha$

And $(1 - \alpha)$ Confidence Intervals for $\mu$ is

• $\bar{X} \pm T_{\alpha/2, n-1} \cdot \sqrt{s^2 / n}$
• This misses $\mu_0$ when
$|\bar{X} - \mu_0 | \geqslant T_{\alpha/2, n-1} \cdot \sqrt{s^2 / n}$

So these are equivalent:

Reject $H_0$ when C.I. misses $\mu_0$