# 39 Why Numerical Methods for Likelihood

## 39.1 Challenges

Frequentist model:

$X_1, X_2, \ldots, X_n {\; \stackrel{\text{iid}}{\sim}\;}F_{{\boldsymbol{\theta}}}$

Bayesian model:

$X_1, X_2, \ldots, X_n | {\boldsymbol{\theta}}{\; \stackrel{\text{iid}}{\sim}\;}F_{{\boldsymbol{\theta}}} \mbox{ and } {\boldsymbol{\theta}}\sim F_{\boldsymbol{\tau}}$

Sometimes it’s not possible to find formulas for $$\hat{{\boldsymbol{\theta}}}_{\text{MLE}}$$, $$\hat{{\boldsymbol{\theta}}}_{\text{MAP}}$$, $${\operatorname{E}}[{\boldsymbol{\theta}}| {\boldsymbol{x}}]$$, or $$f({\boldsymbol{\theta}}| {\boldsymbol{x}})$$. We have to use numerical methods instead.

## 39.2 Approaches

We will discuss the following numerical approaches to likelihood based inference:

• Expectation-maximization (EM) algorithm
• Variational inference
• Markov chain Monte Carlo (MCMC)
• Metropolis sampling
• Metropolis-Hastings sampling
• Gibbs sampling