Probability
Markov's Inequality
A non-negative random variable rarely exceeds a multiple of its expectation
Chebyshev's Inequality
A random variable rarely deviates from its mean by more than a few standard deviations
Bayes' Theorem
Reversing conditional probability to update beliefs from evidence
Jensen's Inequality
For a convex function phi, phi(E[X]) <= E[phi(X)] — the image of the mean is at most the mean of images
Holder's Inequality
For conjugate exponents p and q, the L^p and L^q norms bound the integral of a product
Minkowski's Inequality
The L^p norm satisfies the triangle inequality: ||f+g||_p <= ||f||_p + ||g||_p for p >= 1
Borel-Cantelli Lemma
If the sum of event probabilities is finite, almost surely only finitely many events occur
Law of Total Expectation
The expectation of X equals the expectation of its conditional expectation: E[X] = E[E[X|Y]]
Law of Total Variance
Var(X) decomposes as the sum of the expected conditional variance and the variance of the conditional mean
Linearity of Conditional Expectation
Conditional expectation is linear: E[aX+bY|G] = aE[X|G] + bE[Y|G] almost surely
Martingale Definition
A stochastic process is a martingale if future conditional expectations equal the current value
Strong Law of Large Numbers
The sample mean of i.i.d. integrable random variables converges almost surely to the population mean
Central Limit Theorem
Standardized sums of i.i.d. finite-variance random variables converge in distribution to the standard normal
Kolmogorov 0-1 Law
Every tail event of an independent sequence of random variables has probability exactly 0 or 1
Doob's Martingale Convergence
An L1-bounded submartingale converges almost surely to an integrable limit
Optional Stopping Theorem
The expected value of a stopped martingale equals its initial value under integrability conditions
Characteristic Function
The characteristic function φ_X(t) = E[exp(itX)] uniquely determines the distribution of a random variable
Chernoff Bound
The tail probability P(X ≥ ε) is bounded exponentially by the moment generating function
Weak Convergence (Distribution)
A sequence of probability measures converges weakly if integrals of bounded continuous functions converge