24 Confidence Intervals

24.1 Goal

Once we have a point estimate of a parameter, we would like a measure of its uncertainty.

Given that we are working within a probabilistic framework, the natural language of uncertainty is through probability statements.

We interpret this measure of uncertainty in terms of hypothetical repetitions of the sampling scheme we used to collect the original data set.

24.2 Formulation

Confidence intervals take the form

$(\hat{\mu} - C_{\ell}, \hat{\mu} + C_{u})$

where

${\rm Pr}(\mu - C_{\ell} \leq \hat{\mu} \leq \mu + C_{u})$

forms the “level” or coverage probability of the interval.

24.3 Interpretation

If we repeat the study many times, then the CI $$(\hat{\mu} - C_{\ell}, \hat{\mu} + C_{u})$$ will contain the true value $$\mu$$ with a long run frequency equal to $${\rm Pr}(\mu - C_{\ell} \leq \hat{\mu} \leq \mu + C_{u})$$.

A CI calculated on an observed data set is not intepreted as: “There is probability $${\rm Pr}(\mu - C_{\ell} \leq \hat{\mu} \leq \mu + C_{u})$$ that $$\mu$$ is in our calculated $$(\hat{\mu} - C_{\ell}, \hat{\mu} + C_{u})$$.” Why not?

24.4 A Normal CI

If $$Z \sim$$ Normal(0,1), then $${\rm Pr}(-1.96 \leq Z \leq 1.96) = 0.95.$$

$\begin{eqnarray} 0.95 & = & {\rm Pr} \left(-1.96 \leq \frac{\hat{\mu} - \mu}{\sigma/\sqrt{n}} \leq 1.96 \right) \\ \ & = & {\rm Pr} \left(-1.96 \frac{\sigma}{\sqrt{n}} \leq \hat{\mu} - \mu \leq 1.96\frac{\sigma}{\sqrt{n}} \right) \\ \ & = & {\rm Pr} \left(\mu-1.96\frac{\sigma}{\sqrt{n}} \leq \hat{\mu} \leq \mu+1.96\frac{\sigma}{\sqrt{n}} \right) \end{eqnarray}$

Therefore, $$\left(\hat{\mu} - 1.96\frac{\sigma}{\sqrt{n}}, \hat{\mu} + 1.96\frac{\sigma}{\sqrt{n}}\right)$$ forms a 95% confidence interval of $$\mu$$.

24.5 A Simulation

> mu <- 5
> n <- 20
> x <- replicate(10000, rnorm(n=n, mean=mu)) # 10000 studies
> m <- apply(x, 2, mean) # the estimate for each study
> ci <- cbind(m - 1.96/sqrt(n), m + 1.96/sqrt(n))
[,1]     [,2]
[1,] 4.797848 5.674386
[2,] 4.599996 5.476534
[3,] 4.472930 5.349468
[4,] 4.778946 5.655485
[5,] 4.778710 5.655248
[6,] 4.425023 5.301561
> cover <- (mu > ci[,1]) & (mu < ci[,2])
> mean(cover)
 0.9512

24.6 Normal$$(0,1)$$ Percentiles

Above we constructed a 95% CI. How do we construct (1-$$\alpha$$)-level CIs?

Let $$z_{\alpha}$$ be the $$\alpha$$ percentile of the Normal(0,1) distribution.

If $$Z \sim$$ Normal(0,1), then

$\begin{eqnarray*} 1-\alpha & = & {\rm Pr}(z_{\alpha/2} \leq Z \leq z_{1-\alpha/2}) \\ \ & = & {\rm Pr}(-|z_{\alpha/2}| \leq Z \leq |z_{\alpha/2}|) \end{eqnarray*}$

24.7 Commonly Used Percentiles

> # alpha/2 upper and lower percentiles for alpha=0.05
> qnorm(0.025)
 -1.959964
> qnorm(0.975)
 1.959964
> # alpha/2 upper and lower percentiles for alpha=0.10
> qnorm(0.05)
 -1.644854
> qnorm(0.95)
 1.644854

24.8$$(1-\alpha)$$-Level CIs

If $$Z \sim$$ Normal(0,1), then $${\rm Pr}(-|z_{\alpha/2}| \leq Z \leq |z_{\alpha/2}|) = 1-\alpha.$$

Repeating the steps from the 95% CI case, we get the following is a $$(1-\alpha)$$-Level CI for $$\mu$$:

$\left(\hat{\mu} - |z_{\alpha/2}| \frac{\sigma}{\sqrt{n}}, \hat{\mu} + |z_{\alpha/2}| \frac{\sigma}{\sqrt{n}}\right)$

24.9 One-Sided CIs

The CIs we have considered so far are “two-sided”. Sometimes we are also interested in “one-sided” CIs.

If $$Z \sim$$ Normal(0,1), then $$1-\alpha = {\rm Pr}(Z \geq -|z_{\alpha}|)$$ and $$1-\alpha = {\rm Pr}(Z \leq |z_{\alpha}|).$$ We can use this fact along with the earlier derivations to show that the following are valid CIs:

$(1-\alpha)\mbox{-level upper: } \left(-\infty, \hat{\mu} + |z_{\alpha}| \frac{\sigma}{\sqrt{n}}\right)$

$(1-\alpha)\mbox{-level lower: } \left(\hat{\mu} - |z_{\alpha}| \frac{\sigma}{\sqrt{n}}, \infty\right)$