Probability Distributions

Study guide: Exam MAS-I

External Resources

1. ProbOnto (Probability Distribution Ontology) for data about distributions and their relationships.
2. SIPmath 3.0 Standard for data structures for probability distributions.

Probability Distributions

• Uniform (a die roll)
• Bernoulli (a coin flip)
• Binomial (the number of heads in n coin flips)
• Poisson (the number of heads in many flips of a coin that rarely lands heads)
• Normal (the number of heads in many coin flips, approximately)
  • t-Distribution
• Exponential (the waiting time between events of a Poisson Process)
• Gamma (the waiting time to the n-th event of a Poisson Process)

Relationships

• Binomial → Poisson: As n \to \infty, p \to 0, np = \lambda.
• Binomial → Normal: As n \to \infty, np \text{ and } n(1-p) are large.
• Poisson → Normal: As \lambda becomes large.

Exponential Family

f(x \mid \theta) = h(x)\exp\big(\eta(\theta)\,T(x) - A(\theta)\big)

• This is the form f(y; \theta) = \exp[a(y)b(\theta) + c(\theta) + d(y)] in Larsen's study note (which supplements Dobson & Barnett §3.2), with h = e^{d}, T = a, \eta = b and A = -c.

Binomial

Tossing a coin n times, how likely to get k heads? If each toss lands heads with probability p, independently of the others:

P(X = k) = \binom{n}{k} p^k (1-p)^{n-k}

• k = 0, 1, \ldots, n, with E[X] = np and \text{Var}(X) = np(1-p)

Poisson

P(N = k) = \frac{e^{-\lambda}\lambda^k}{k!}

• k = 0, 1, 2, \ldots, with E[N] = \text{Var}(N) = \lambda

Normal

f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-(x-\mu)^2/(2\sigma^2)}

• -\infty < x < \infty; \mu is the mean and \sigma the standard deviation

Exponential

f(x) = \frac{1}{\theta}\, e^{-x/\theta}

• x > 0, with F(x) = 1 - e^{-x/\theta} and E[X] = \theta
• It is the gamma distribution with \alpha = 1, and the sum of n independent exponentials with mean \theta is gamma with \alpha = n and the same \theta

Central Concepts

• Central Limit Theorem