# 18 Convergence of Random Variables

## 18.1 Sequence of RVs

Let $$Z_1, Z_2, \ldots$$ be an infinite sequence of rv’s.

An important example is

$Z_n = \overline{X}_n = \frac{\sum_{i=1}^n X_i}{n}.$

It is useful to be able to determine a limiting value or distribution of $$\{Z_i\}$$.

## 18.2 Convergence in Distribution

$$\{Z_i\}$$ converges in distribution to $$Z$$, written

$Z_n \stackrel{D}{\longrightarrow} Z$

if

$F_{Z_n}(y) = \Pr(Z_n \leq y) \rightarrow \Pr(Z \leq y) = F_{Z}(y)$

as $$n \rightarrow \infty$$ for all $$y \in \mathbb{R}$$.

## 18.3 Convergence in Probability

$$\{Z_i\}$$ converges in probability to $$Z$$, written

$Z_n \stackrel{P}{\longrightarrow} Z$

if

$\Pr(|Z_n - Z| \leq \epsilon) \rightarrow 1$

as $$n \rightarrow \infty$$ for all $$\epsilon > 0$$.

Note that it may also be the case that $$Z_n \stackrel{P}{\longrightarrow} \theta$$ for a fixed, nonrandom value $$\theta$$.

## 18.4 Almost Sure Convergence

$$\{Z_i\}$$ converges almost surely (or “with probability 1”) to $$Z$$, written

$Z_n \stackrel{a.s.}{\longrightarrow} Z$

if

$\Pr\left(\{\omega: |Z_n(\omega) - Z(\omega)| \stackrel{n \rightarrow \infty}{\longrightarrow} 0 \}\right) = 1.$

Note that it may also be the case that $$Z_n \stackrel{a.s.}{\longrightarrow} \theta$$ for a fixed, nonrandom value $$\theta$$.

## 18.5 Strong Law of Large Numbers

Suppose $$X_1, X_2, \ldots, X_n$$ are iid rv’s with population mean $${\operatorname{E}}[X_i] = \mu$$ where $${\operatorname{E}}[|X_i|] < \infty$$. Then

$\overline{X}_n \stackrel{a.s.}{\longrightarrow} \mu.$

## 18.6 Central Limit Theorem

Suppose $$X_1, X_2, \ldots, X_n$$ are iid rv’s with population mean $${\operatorname{E}}[X_i] = \mu$$ and variance $${\operatorname{Var}}(X_i) = \sigma^2$$. Then as $$n \rightarrow \infty$$,

$\sqrt{n}(\overline{X}_n - \mu) \stackrel{D}{\longrightarrow} \mbox{Normal}(0, \sigma^2).$

We can also get convergence to a $$\mbox{Normal}(0, 1)$$ by dividing by the standard deviation, $$\sigma$$:

$\frac{\overline{X}_n - \mu}{\sigma/\sqrt{n}} \stackrel{D}{\longrightarrow} \mbox{Normal}(0, 1).$

We write the second convergence result as above rather than $\frac{\sqrt{n} (\overline{X}_n - \mu)}{\sigma} \stackrel{D}{\longrightarrow} \mbox{Normal}(0, 1)$ because $$\sigma/\sqrt{n}$$ is the “standard error” of $$\overline{X}_n$$ when $$\overline{X}_n$$ is treated as an estimator, so $$\sigma/\sqrt{n}$$ is kept intact.

Note that for fixed $$n$$, ${\operatorname{E}}\left[ \frac{\overline{X}_n - \mu}{1/\sqrt{n}} \right] = 0 \mbox{ and } {\operatorname{Var}}\left[ \frac{\overline{X}_n - \mu}{1/\sqrt{n}} \right] = \sigma^2,$

${\operatorname{E}}\left[ \frac{\overline{X}_n - \mu}{\sigma/\sqrt{n}} \right] = 0 \mbox{ and } {\operatorname{Var}}\left[ \frac{\overline{X}_n - \mu}{\sigma/\sqrt{n}} \right] = 1.$

## 18.7 Example: Calculations

Let $$X_1, X_2, \ldots, X_{40}$$ be iid Poisson($$\lambda$$) with $$\lambda=6$$.

We will form $$\sqrt{40}(\overline{X} - 6)$$ over 10,000 realizations and compare their distribution to a Normal(0, 6) distribution.

> x <- replicate(n=1e4, expr=rpois(n=40, lambda=6),
+                simplify="matrix")
> x_bar <- apply(x, 2, mean)
> clt <- sqrt(40)*(x_bar - 6)
>
> df <- data.frame(clt=clt, x = seq(-18,18,length.out=1e4),
+                  y = dnorm(seq(-18,18,length.out=1e4),
+                            sd=sqrt(6)))

## 18.8 Example: Plot

> ggplot(data=df) +
+   geom_histogram(aes(x=clt, y=..density..), color="blue",
+                  fill="lightgray", binwidth=0.75) +
+   geom_line(aes(x=x, y=y), size=1.5)