continuously continuing review

Yesterday I crammed a chapter on continuous probability distributions. Let's see what I can remember before I open the book again.

First, can I remember what they are? Here's what I got on paper at first:

• The continuous distributions

  Uniform; Exponential; Gamma; Normal; Log-normal; Pareto; Weibull; Beta

• The discrete distributions

  Binomial; Hypergeometric; Poisson; Geometric; Negative binomial; I feel like I forgot one

First pass

The continuous distributions

Uniform

$$\begin{aligned} \mathrm dP(x) &= \begin{cases} \frac{\mathrm dx}{b-a} & a<x<b \\ 0 & \text{elsewhere} \end{cases} \\ \mu &= \frac{a+b}{2} \\ \sigma^2 &= \frac{(b-a)^2}{12} \end{aligned}$$

Exponential

$$\begin{aligned} \mathrm dP(x) &= \lambda e^{-\lambda x} \mathrm dx \\ \mu &= \lambda \\ \sigma^2 &= \lambda^2 \end{aligned}$$

Gamma

This one came slowly, but I got it (if I did) by analogy with the exponential distribution. Remembering the mean and the variance was the key here.

$$\begin{aligned} \mathrm dP(x) &= \frac{\beta}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x} \mathrm dx \\ \mu &= \alpha\lambda \\ \sigma^2 &= \alpha\lambda^2 \end{aligned}$$

Normal

$$\begin{aligned} \mathrm dP(x) &= \frac{1}{\sqrt{\sigma^2 2\pi}} \exp\left( -\frac{(x-\mu)^2}{2\sigma^2} \right) \mathrm dx \\ \mu &= \mu \\ \sigma^2 &= \sigma^2 \end{aligned}$$

Log-normal

$$\begin{aligned} \mathrm dP(\ln y) &= \frac{1}{\sqrt{\sigma_Y^2 2\pi}} \exp\left( -\frac{(\ln y-\mu)^2}{2\sigma_Y^2} \right) \mathrm d(\ln y) \\ \mu_{\ln y} &= \mu \\ \sigma_Y^2 &= \cdots \end{aligned}$$

If the expectation value of $\ln y$ is $E(\ln y) = \mu$, I think that doesn't directly translate into $E(y) = e^\mu$. I think the mean and variance are complicated and that folks don't use them

Pareto

I drew a blank here. Annoying, because I know the name and I actually wanted to know what this one was.

It's "tail-heavy."

It's the exponential decay modified to be tail-heavy, for "independent" events which become more likely over time, like aging machine parts.

I can't recall the probability density expression at all.

Weibull

I'm pretty sure this is the one-sided, zero-excluding distribution which goes

$$\begin{aligned} \mathrm dP &= \frac \alpha\beta \left( \frac\beta x \right)^{\alpha -1} \mathrm dx \end{aligned}$$

I remember the mean and variance being hairy here.

Beta

This is the one for events bounded on $x \in [0,1]$.

$$\begin{aligned} \mathrm dP(x) &= A x^{\alpha-1} (1-x)^{\beta-1} \mathrm dx \end{aligned}$$

I think the normalization is $ A = \frac{\Gamma(\alpha+\beta)}{\Gamma(\alpha)\Gamma(\beta)} $.

The discrete distributions

For fun, let's recall these, too.

Binomial

$$\begin{aligned} P(k) &= {n\choose k} p^k q^{n-k} \\ \mu &= np \\ \sigma^2 &= npq \end{aligned}$$

Hypergeometric

$$\begin{aligned} P(k) &= \frac{ {K \choose k}{N-K \choose n-k} }{ {N \choose n} } \\ \mu &= n \cdot \frac KN \\ \sigma^2 &= n \left( \frac KN \right)^2 ? \end{aligned}$$

Oh, this is the one which goes like the binomial variance, but with the population size replacement.

$$\begin{aligned} \sigma^2 &= n \times \frac KN \times \frac{N-K}{N} \times \frac{ N-n }{ N-1 } \end{aligned}$$

Poisson

$$\begin{aligned} P(k) &= \frac{ e^{-\lambda} \lambda^k }{ k! } \\ \mu &= \lambda \\ \sigma^2 &= \lambda \end{aligned}$$

Geometric

$$\begin{aligned} P(k) &= p q^{k} \\ \mu &= \frac pq \text{ or } \frac 1q\text{, fencepost} \\ \sigma^2 &= \cdots ? \end{aligned}$$

Negative binomial

I've lost this one, too. No recollection.

I feel like I forgot one

Corrections

Continuous

Exponential and gamma: unit error

The mean and variance for the exponential distribution are

$$\begin{aligned} \mu &= \frac1\lambda & \sigma^2 &= \frac1{\lambda^2}, \end{aligned}$$ which I would have remembered if I'd been using my units correctly.

That means I also got the geometric distribution wrong; it should be $$\begin{aligned} \mu &= \frac\alpha\beta & \sigma^2 &= \frac{\alpha}{\beta^2} \end{aligned}$$

Log-normal: complicated as expected.

$$\begin{aligned} E(Y) &= \exp\left(\mu + \frac{\sigma^2}{2}\right) \\ V(Y) &= e^{2\mu + \sigma^2} \left( e^{\sigma^2} - 1 \right) \end{aligned}$$

I have no plan to commit these to memory.

Pareto and Weibull

I mixed up the Pareto and the Weibull, by remembering only one of them and also misnaming it. Also a sign error. A mess, all in all; but I don't actually need these for my current purposes.

The Pareto is

$$\begin{aligned} \mathrm dP(x) &= \frac{\alpha}{\beta}\left( \frac\beta x \right)^{\alpha+1} \mathrm dx \end{aligned}$$

It's the Weibull that is the failure-increases-with-age distribution. In that case the failure rate is

$$\begin{aligned} \lambda(x) &= \alpha\beta x^{\alpha-1} \end{aligned}$$

which corresponds to a probability density

$$\begin{aligned} \mathrm dP(x) &= \alpha\beta\ x^{\alpha-1} e^{-\beta x^\alpha} \mathrm dx \end{aligned}$$

Discrete

Geometric

I flipped the expectation value; it's either $\frac 1p$ or $\frac qp$, depending on whether your count starts at zero or at one. Likewise, the variance is $\frac{q}{p^2}$.

Negative binomial

This is the one that extends the geometric series to the $n$th event.

$$\begin{aligned} P(k) &= {r+k-1 \choose r-1} q^k p^r \\ \mu &= \frac{rq}{p} \\ \sigma^2 &= \frac{rq}{p^2}. \end{aligned}$$

I forgot UNIFORM

because it's BORING