# 27 MLE Examples: One Sample

## 27.1 Exponential Family Distributions

Suppose $$X_1, X_2, \ldots, X_n$$ are iid from some EFD. Then,

$\frac{\partial}{\partial \eta_k} \ell(\boldsymbol{\eta} ; \boldsymbol{x}) = \sum_{i=1}^n T_k(x_i) - n \frac{\partial}{\partial \eta_k} A(\boldsymbol{\eta})$ Setting the second equation to 0, it follows that the MLE of $$\eta_k$$ is the solution to

$\frac{1}{n} \sum_{i=1}^n T_k(x_i) = \frac{\partial}{\partial \eta_k} A(\boldsymbol{\eta}).$

where note that $$\frac{\partial}{\partial \eta_k} A(\boldsymbol{\eta}) = {\operatorname{E}}[T_k(X)]$$.

## 27.2 Summary of MLE Statistics

In all of these scenarios, $$Z$$ converges in distribution to Normal$$(0,1)$$ for large $$n$$.

Distribution MLE Std Err $$Z$$ Statistic
Binomial$$(n,p)$$ $$\hat{p} = X/n$$ $$\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}$$ $$\frac{\hat{p} - p}{\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}}$$
Normal$$(\mu, \sigma^2)$$ $$\hat{\mu} = \overline{X}$$ $$\frac{\hat{\sigma}}{\sqrt{n}}$$ $$\frac{\hat{\mu} - \mu}{\hat{\sigma}/\sqrt{n}}$$
Poisson$$(\lambda)$$ $$\hat{\lambda} = \overline{X}$$ $$\sqrt{\frac{\hat{\lambda}}{n}}$$ $$\frac{\hat{\lambda} - \lambda}{\sqrt{\hat{\lambda}/n}}$$

## 27.3 Notes

• For the Normal and Poisson distributions, our model is $$X_1, X_2, \ldots, X_n$$ iid from each respective distribution
• For the Binomial distribution, our model is $$X \sim \mbox{Binomial}(n, p)$$
• In the Normal model, $$\hat{\sigma} = \sqrt{\frac{\sum_{i=1}^n (X_i - \overline{X})^2}{n}}$$ is the MLE of $$\sigma$$
• The above formulas were given in terms of the random variable probability models; on observed data the same formulas are used except we observed data lower case letters, e.g., replace $$\overline{X}$$ with $$\overline{x}$$

## 27.4 Binomial

Approximate $$(1-\alpha)$$-level two-sided CI:

$\left(\hat{p} - |z_{\alpha/2}| \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}, \hat{p} + |z_{\alpha/2}| \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \right)$

Hypothesis test, $$H_0: p=p_0$$ vs $$H_1: p \not= p_0$$:

$z = \frac{\hat{p} - p_0}{\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}} \mbox{ and } \mbox{p-value} = {\rm Pr}(|Z^*| \geq |z|)$

where $$Z^*$$ is a Normal$$(0,1)$$ random variable.

## 27.5 Normal

Approximate $$(1-\alpha)$$-level two-sided CI:

$\left(\hat{\mu} - |z_{\alpha/2}| \frac{\hat{\sigma}}{\sqrt{n}}, \hat{\mu} + |z_{\alpha/2}| \frac{\hat{\sigma}}{\sqrt{n}} \right)$

Hypothesis test, $$H_0: \mu=\mu_0$$ vs $$H_1: \mu \not= \mu_0$$:

$z = \frac{\hat{\mu} - \mu_0}{\hat{\sigma}/\sqrt{n}} \mbox{ and } \mbox{p-value} = {\rm Pr}(|Z^*| \geq |z|)$

where $$Z^*$$ is a Normal$$(0,1)$$ random variable.

## 27.6 Poisson

Approximate $$(1-\alpha)$$-level two-sided CI:

$\left(\hat{\lambda} - |z_{\alpha/2}| \sqrt{\frac{\hat{\lambda}}{n}}, \hat{\lambda} + |z_{\alpha/2}| \sqrt{\frac{\hat{\lambda}}{n}} \right)$

Hypothesis test, $$H_0: \lambda=\lambda_0$$ vs $$H_1: \lambda \not= \lambda_0$$:

$z = \frac{\hat{\lambda} - \lambda_0}{\sqrt{\frac{\hat{\lambda}}{n}}} \mbox{ and } \mbox{p-value} = {\rm Pr}(|Z^*| \geq |z|)$

where $$Z^*$$ is a Normal$$(0,1)$$ random variable.

## 27.7 One-Sided CIs and HTs

The one-sided versions of these approximate confidence intervals and hypothesis tests work analogously.

The procedures shown for the $$\mbox{Normal}(\mu, \sigma^2)$$ case with known $$\sigma^2$$ from last week are utilzied with the appropriate subsitutions as in the above examples.