Lecture 16 — Gaussian Tail Bounds and Independence
This lecture has two main components:
- Tail bounds for random variables (Markov, Chebyshev, Chernoff/Exponential tail for the Gaussian).
- Independence of σ-algebras and random variables via the π–λ theorem.
1. Gaussian Tail Bounds
Let $Z \sim N(0,1)$.
1.1 Exact integral representation
\[P(Z > x) = \int_x^{\infty} \frac{1}{\sqrt{2\pi}} e^{-y^2/2}\, dy, \qquad x>0.\]The handwritten notes (page 1) perform a change of variable:
- Let $y = x + z$, so $dy = dz$.
- Then: \(\int_x^\infty e^{-y^2/2}\,dy = e^{-x^2/2} \int_0^\infty e^{-xz} e^{-z^2/2}\,dz \le e^{-x^2/2} \int_0^\infty e^{-xz}\,dz = e^{-x^2/2} \cdot \frac{1}{x}.\)
Thus:
\[P(Z > x) \le \frac{1}{\sqrt{2\pi}}\frac{e^{-x^2/2}}{x}.\]1.2 Lower bound
On page 1, the lecture notes show the inequality:
\[\int_x^\infty e^{-y^2/2}\,dy \ge \int_x^\infty \left(1 - \frac{3}{y^4}\right)e^{-y^2/2}\,dy = \left( \frac{1}{x} - \frac{1}{x^3} \right) e^{-x^2/2}.\]So overall:
\[\boxed{ \left(\frac{1}{x} - \frac{1}{x^3}\right) \frac{e^{-x^2/2}}{\sqrt{2\pi}} \;\le\; P(Z>x) \;\le\; \frac{1}{x} \frac{e^{-x^2/2}}{\sqrt{2\pi}}. }\]These are the classical Mills ratio bounds.
2. Markov and Chebyshev Inequalities
For any nonnegative $X$,
\[P(X > a) \le \frac{\mathbb{E}[X]}{a}.\]For any square-integrable $X$ with mean $m$ and variance $\sigma^2$,
\[P(\vert X - m\vert > a) \le \frac{\sigma^2}{a^2}.\]The handwritten notes then consider shifting to optimize:
Given $Y$ with $\mathbb{E}[Y]=m$, $\operatorname{Var}(Y)=\sigma^2$, the notes derive:
\[P(Y > a) \le P(\vert Y+b\vert > a+b) \le \frac{\mathbb{E}[(Y+b)^2]}{(a+b)^2} = \frac{\sigma^2 + b^2}{(a+b)^2}.\]Choose $b$ to minimize the right-hand side.
The notes compute:
Thus:
\[\boxed{ P(Y > a) \le \frac{\sigma^2}{\sigma^2 + a^2}. }\](The algebra is written step-by-step on page 1 of the handwritten notes.)
3. Independence (Durrett Ch. 2.1)
Let $ {\mathcal{F}\alpha}{\alpha\in I} \text{ be sub-σ-algebras of } \mathcal{F}$.
3.1 Definition
They are independent if for all finite choices $\alpha_1,\dots,\alpha_n$ and events $A_i \in \mathcal{F}_{\alpha_i}$,
\[P(A_1 \cap \cdots \cap A_n) = P(A_1)\cdots P(A_n).\]3.2 Independence of random variables
A family of random variables ${X_\alpha}$ is independent if
${\sigma(X_\alpha)}$ are independent as σ-algebras.
Measurability of $X_\alpha$
Each $X_\alpha: \Omega\to\mathbb{R}$ is $\mathcal{F}/\mathcal{B}(\mathbb{R})$-measurable.
So:
\[\sigma(X_\alpha) = \{ X_\alpha^{-1}(B): B\in\mathcal{B}(\mathbb{R})\}.\]4. Independence via π–λ Theorem (Key Theorem)
Theorem.
If the collections ${A_i}$ are independent on a π-system and each $A_i$ generates a σ-algebra, then the σ-algebras they generate are independent.
Outline of proof as in notes
Fix an index $1$. Define:
\[\mathcal{L} = \{A\in\mathcal{F} : P(A \cap A_2 \cap\cdots\cap A_n) = P(A)\,P(A_2)\cdots P(A_n) \quad \forall A_2\in \mathcal{A}_2,\dots,A_n\in\mathcal{A}_n\}.\]The handwritten notes show:
- $\mathcal{L}$ is a λ-system:
- Contains $\Omega$,
- Closed under difference,
- Closed under increasing unions.
- $\mathcal{A}_1$ is a π-system.
Thus by Dynkin’s π–λ theorem:
\[\sigma(\mathcal{A}_1) \subset \mathcal{L}.\]That is:
anything generated by the π-system keeps the independence property.
Repeat cyclically for all indices → independence of all σ-algebras.
5. Example from the Notes (page 2)
We have two random variables $X,Y$.
Given:
\[P(X\le x, Y\le y) = P(X\le x)\, P(Y\le y).\]Do they have to be independent?
Yes.
Explanation:
- Sets of the form $(-\infty,x]$ form a π-system (though not a σ-algebra).
- Independence on this π-system implies independence of the σ-algebras they generate, which are $\sigma(X)$ and $\sigma(Y)$.
Thus $X$ and $Y$ are independent.
6. Example (page 2, bottom)
If ${\mathcal{F}_1,\mathcal{F}_2,\mathcal{F}_3}$ are independent σ-algebras:
- Then
\(\sigma(\mathcal{F}_1,\mathcal{F}_2) \quad\text{and}\quad \mathcal{F}_3\) are also independent.
Reason:
$\mathcal{F}_1\cap \mathcal{F}_2$ as a π-system satisfies the required closure.
Apply π–λ again.
Application to random variables
If $X_1,X_2,X_3,X_4,X_5$ are independent random variables, then:
- $X_1+X_2$
- $e^{X_3} - \sin(X_1+X_3)$
are independent random variables.
Because the σ-algebras they generate depend only on disjoint subsets of the original independent family.
7. Summary (Lecture 16)
- Derived Gaussian tail bounds using exponential change-of-variable and Markov/Chernoff ideas.
- Showed optimized Chebyshev-type inequality.
- Defined independence of σ-algebras and random variables.
- Proved independence using π–λ Theorem.
- Applied independence properties to examples of functions of independent RVs.
Comments