Probability and Statistics

Probability

Set theory is foundational to probability. If you need a refresher, try https://www.statlect.com/mathematical-tools/set-theory.

$\Omega$ is the set of possible outcomes, called the sample space.
An element $\omega \in \Omega$ is called a sample point, or possible outcome.
When we learn that $\overline{\omega}$ has happened, $\overline{\omega}$ is called the realized outcome.
A subset $E \subseteq \Omega$ is called an event.
- $\Omega$ is an event, called the sure event.
- $\emptyset$ is an event, called the impossible event.
The space of all events is $\mathscr{F}$.
A probability measure is a function $P : \mathscr{F} \to \mathbb{R}$. To be a probability measure, $P$ MUST have three properties:
1. $P(E) \in [0, 1]$
2. $P(\Omega) = 1$; $P \left( \emptyset \right) = 0$
3. $P \left( \bigcup_{i=1}^{\infin} E_i \right) = \sum_{i=1}^{\infin} P \left( E_i \right)$, for any sequence of mutually-exclusive events, i.e. $E_i \cap E_j = \emptyset$ when $i \ne j$.
$P(E)$ is called the probability of $E$.
$R_X$ is the support of a random variable; it is the set of values the random variable can have.
- For a six-sided die, $R_X = \lbrace 1, 2, 3, 4, 5, 6 \rbrace$
- For a Bernoulli trial, $R_X = \lbrace 0, 1 \rbrace$
- For a continuous, real random variable, $R_X$ would be $\mathbb{R}$ or some restriction of $\mathbb{R}$.

Binomial Univariate discrete distribution that models the number of favorable outcomes obtained in a repeated Bernoulli experiment.
Geometric Univariate discrete distribution that models either the number of attempts or the number of failures from repeating a Bernoulli experiment until the first success is obtained.
Poisson Univariate discrete distribution that models the number of occurrences of a random event.
Normal (Gaussian) Continuous distribution
Chi Squared ($\chi^2$) Continuous distribution that models the sum of the squares of $k$ independent standard normal random variables.
Non-central Chi Squared Non-central generalization (i.e. $\mu \ne 0$) of the Chi Squared distribution.

Distribution

Parameters

PDF/PMF

CDF

Binomial

$\binom{n}{k}p^k q^{n-k}$

Geometric

$p \in (0, 1)$

$X$ = Trials for success	$Y = X-1$ failures before success
$(1-p)^{k-1} p$	$(1-p)^k p$

$X$ = Trials for success	$Y = X-1$ failures before success
$x < 1 : 0$ $x \ge 1 : 1 - (1-p)^{\lfloor x \rfloor}$	$x < 0 : 0$ $x \ge 0 : 1 - (1-p)^{\lfloor x \rfloor + 1}$

Poisson

$\lambda \in (0, \infin)$ (rate)

$\frac{\lambda^k \euler^{-k}}{k!}$

$\frac{\Gamma \left( \lfloor k + 1 \rfloor, \lambda \right)}{\lfloor k \rfloor !}$
$\Gamma$ is the upper incomplete Gamma function

Normal

$f(x)=\frac{1}{\sigma \sqrt{2 \pi}} \euler^{-\frac{1}{2} \left( \frac{x - \mu}{\sigma} \right)^2}$

$\Phi \left( \frac{x - \mu}{\sigma} \right) = \frac{1}{2} \left[ 1 + \rm{erf} \left( \frac{x - \mu}{\sigma \sqrt{2} } \right) \right]$
$\rm{erf}$ is the Error Function

$\chi^2$

$k \in \mathbb{N}$ (degrees of freedom)

Non-central $\chi^2$

$\frac{1}{2}\euler^{-(x+\lambda)/2} \left( \frac{x}{\lambda} \right)^{k/4 - 1/2} I_{k/2-1}(\sqrt{\lambda x})$
$I$ is the modified Bessel function of the first kind

Function	Notation	Definition
Error Function	$\rm{erf}$	$\rm{erf} = \frac{1}{\sqrt{\pi}} \int_{-x}^{x} \euler^{-t^2} \diff{t}$
Regularized Beta Function	$I_z(a,b)$	$I_z(a,b) = \frac{B_z(a, b)}{B(a,b)}$
Incomplete Beta Function	$B_z(a,b)$	https://mathworld.wolfram.com/IncompleteBetaFunction.html
Complete Beta Function	$B(a,b)$	https://mathworld.wolfram.com/BetaFunction.html
Gamma Function	$\Gamma(n)$	$n \in \mathbb{N}: \Gamma(n) = (n - 1)!$ $n \in \mathbb{C}: \Gamma(z) = \int_0^\infin t^{z-1} \euler^{-t} \diff{t}, \Re(z) > 0$
Lower Incomplete Gamma Function	$\gamma(s,x)$	$\gamma(s,x) = \int_0^x t^{s-1} \euler^{-t} \diff{t}$
Upper Incomplete Gamma Function	$\Gamma(s,x)$	$\Gamma(s,x) = \int_x^\infin t^{s-1} \euler^{-t} \diff{t}$
Modified Bessel function of the First Kind	$I_v(y)$	$I_v(y)= \left( \frac{y}{2} \right)^v \sum_{j=0}^\infin \frac{(y^2/4)^j}{j! \Gamma(v + j + 1)}$
Marcum Q Function	$Q_v(a,b)$	$Q_v(a,b) = \frac{1}{a^{v-1}} \int_b^\infin x^v \rm{exp} \left( - \frac{x^2 + a^2}{2} \right) I_{v-1}(ax) \diff{x}$

$ \E[\cdot] $ is the Expected Value operator, so $ \E[X] $ is the expected value of $ X $.

$ X, Y, X_1, X_2 \mathellipsis X_k $ are real random variables
$ Z, W, Z_1, Z_2 \mathellipsis Z_k $ are complex random variables
$ a, b, a_1, a_2 \mathellipsis a_k $ are constants $ \{ a, b, a_1, a_2 \mathellipsis a_k \in \R \} $
$ \bold{\Sigma} $ is a $ k \times l $ matrix of random variables
$ \bold{A} $ is a $ k \times l $ matrix of constants
$ \bold{B} $ is an $ m \times k $ matrix of constants
$ \bold{C} $ is an $ l \times n $ matrix of constants
$ k, l, m, n \in \N $

Given the probability density function $ f(X) $, then $ \E[X] = \int_{-\infin}^{\infin} X \cdot f(X) \diff x $
Given the probability mass function $ P(X) $, then $ \E[X] = \sum_{x_k \in \mathbb{R_x} } X \cdot P(X) $
$ \E[X] = \mu_X $ (The expected value of a random variable is the arithmetic mean of that variable.)
$ \E[a] = a $
$ \E[aX] = a \E[X] $
$ \E[a_1 X_1 + a_2 X_2 + \mathellipsis a_k X_k = a_1 \E[X_1] + a_2 \E[X_2] + \mathellipsis a_k \E[X_k] $
$ \E[X \cdot Y] = \E[X] \cdot \E[Y] $
$ \E[\bold{A} + \bold{\Sigma} ] = \bold{A} + \E[ \bold{\Sigma} ] $
$ \E[\bold{B} \cdot \bold{\Sigma}] = \bold{B} \cdot \E[ \bold{\Sigma} ] $
$ \E[\bold{\Sigma} \cdot \bold{C}] = \E[\bold{\Sigma}] \cdot \bold{C} $
In general, for a non-linear function $ g(\cdot) $, $ \E[g(X)] \neq g(\E[X]) $
If g(X) is convex, then $\E[g(X)] \ge g(\E[X])$
If g(X) is concave, then $\E[g(X)] \le g(\E[X])$

A complex random variable can always be treated as a vector of real random variables (real and imaginary parts).

$ \E[Z] = \E[\Re(Z)] + \imath \E[\Im(Z)] $
$ \overline{\E[Z]} = \E[\bar{Z}] $
If the complex random variable Z has a probability density function $ f_Z(z) $, then $ \E[Z] = \iint_{\mathbb{C}} z \cdot f_Z(z) \diff x \diff y $

Taboga, Marco (2021). "The mathematics of probability", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/probability
Taboga, Marco (2021). "Properties of the expected value", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/fundamentals-of-probability/expected-value-properties
Wikipedia contributors, "Complex random variable," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Complex_random_variable&oldid=1185345835 (accessed January 11, 2024)
Wikipedia contributors, "Probability axioms," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Probability_axioms&oldid=1194504777 (accessed January 11, 2024)
Wikipedia contributors, "Random variable," Wikipedia, The Free Encyclopedia, https://en.wikipedia.org/w/index.php?title=Random_variable&oldid=1189421773 (accessed January 11, 2024)