Edit this page on GitHub

2018.Q1 — Gaussian Identities, Stein’s Lemma, and Moment Recursions”

2018 Probability Prelim Exam (PDF)

Let $Z \sim N(0,1)$ and denote by $f_Z(x)$, $-\infty < x < \infty$ the density of $Z$.

(a) Prove for each $a < b$
\(E(Z;\, a < Z < b) = f_Z(a) - f_Z(b).\)

(b) Let $f, f’: \mathbb{R} \to \mathbb{R}$ be two continuous functions where $f’$ is the derivative of $f$.
Assume that there exists a positive integer $k$ and a constant $0 \le C < \infty$ so that both
\(\limsup_{\vert x\vert \to\infty} \frac{\vert f(x)\vert }{\vert x\vert ^k} \le C, \qquad \limsup_{\vert x\vert \to\infty} \frac{\vert f'(x)\vert }{\vert x\vert ^k} \le C\) (In words: $f$ and $f’$ have at most polynomial growth.)

(i) Prove that $E(\vert f’(Z)\vert ) < \infty$ and $E(\vert Z f(Z)\vert ) < \infty$.

(ii) Prove that $E(f’(Z)) = E(Z f(Z))$.
Hint: Use integration by parts.

(c) Show how we can get the following two identities from part (b)(ii).

(i)
\(E(Z^{n+1}) = n E(Z^{n-1}), \qquad n \ge 1.\)

(ii) Let $f, f’$ be as in part (b) and let $g, g’$ be another pair of continuous functions,
$g’$ is the derivative of $g$ and both have at most polynomial growth. Then
\(E(f'(Z) g(Z)) = E(Z f(Z) g(Z)) - E(f(Z) g'(Z)).\)

(a) Compute $E(Z;\, a < Z < b)$

Assumptions / Notation

  • $Z \sim N(0,1)$ with density $\phi(x)$.
  • Gaussian derivative identity: \(\phi'(x) = -x\phi(x).\)

Solution

Claim.

For $a < b$,
\(E(Z;\, a < Z < b) = \phi(a) - \phi(b).\)

Proof.

Step 1. Expand the restricted expectation.
\(E(Z;\, a<Z<b) = \int_a^b z\,\phi(z)\,dz.\)

Step 2. Substitute the Gaussian derivative identity.
Since
\(z\phi(z) = -\phi'(z),\)
we rewrite: \(\int_a^b z\phi(z)\,dz = \int_a^b -\phi'(z)\,dz.\)

Step 3. Integrate the derivative.
\(\int_a^b -\phi'(z)\,dz = -\phi(z)\Big\vert _a^b = \phi(a) - \phi(b).\)

Conclusion.

\(E(Z;\, a < Z < b) = \phi(a) - \phi(b).\)

Key Takeaways

  • This is the canonical use of the identity $\phi’(x) = -x\phi(x)$.
  • Most Gaussian expectation manipulations reduce to boundary evaluations of $\phi$.
  • When you see $z\phi(z)$, instinct: convert to a derivative.

(b)(i) Show $E\vert f’(Z)\vert <\infty$ and $E\vert Z f(Z)\vert <\infty$

Assumptions

  • $f$ and $f’$ continuous.
  • Polynomial growth: \(\vert f(x)\vert \le C(1+\vert x\vert ^k),\quad \vert f'(x)\vert \le C(1+\vert x\vert ^k).\)
  • Gaussian moments finite: \(E\vert Z\vert ^m < \infty \quad \forall m\ge 0.\)

Solution

Claim.

Under the assumptions: \(E\vert f'(Z)\vert < \infty, \qquad E\vert Z f(Z)\vert < \infty.\)

Proof.

Step 1. Control the tails.
Polynomial growth gives: \(\vert f'(x)\vert \le A(1+\vert x\vert ^k) \quad \text{for large } \vert x\vert .\)

Similarly: \(\vert f(x)\vert \le B(1+\vert x\vert ^k).\)

Step 2. Control the middle.
Continuity on compact sets implies boundedness, so for $\vert x\vert \le R$: \(\vert f'(x)\vert \le M.\)

Step 3. Combine into a single uniform bound.
For all $x\in\mathbb{R}$, \(\vert f'(x)\vert \le A(1+\vert x\vert ^k).\)

Step 4. Take expectations.
\(E\vert f'(Z)\vert \le A(1 + E\vert Z\vert ^k) < \infty.\)

Step 5. Apply same reasoning to $E\vert Z f(Z)\vert $.
Using
\(\vert Z f(Z)\vert \le B(\vert Z\vert + \vert Z\vert ^{k+1}),\) we obtain: \(E\vert Z f(Z)\vert \le B(E\vert Z\vert + E\vert Z\vert ^{k+1}) < \infty.\)

Conclusion.

Both expectations are finite.

Key Takeaways

  • Gaussian tails dominate all polynomials.
  • Any function with polynomial growth is integrable against a Gaussian.
  • Tail-middle decomposition is a standard technique:
    • compact region → bounded
    • tail → controlled by growth condition

(b)(ii) Show $E[f’(Z)] = E[Z f(Z)]$

Assumptions

  • Same polynomial growth bounds as in part (i).
  • Integration by parts applies.
  • Boundary term $f(z)\phi(z)\to 0$ as $\vert z\vert \to\infty$.

Solution

Claim.

\(E[f'(Z)] = E[Z f(Z)].\)

Proof.

Step 1. Write $E[Z f(Z)]$ as an integral.
\(E[Z f(Z)] = \int_{-\infty}^\infty z f(z)\phi(z)\,dz.\)

Step 2. Replace $z\phi(z)$ using $\phi’(z) = -z\phi(z)$.
\(z f(z)\phi(z) = -f(z)\phi'(z).\)

Thus: \(E[Z f(Z)] = -\int_{-\infty}^\infty f(z)\phi'(z)\,dz.\)

Step 3. Integration by parts.
Take

  • $u=f(z)$
  • $dv=-\phi’(z)dz$ ⇒ $v=\phi(z)$

Then: \(-\int f(z)\phi'(z)\,dz = -f(z)\phi(z)\Big\vert _{-\infty}^{\infty} + \int f'(z)\phi(z)\,dz.\)

Step 4. Boundary term vanishes.
Polynomial × Gaussian ⇒
\(f(z)\phi(z)\to 0.\)

Thus: \(E[Z f(Z)] = E[f'(Z)].\)

Conclusion.

Stein’s identity holds: \(E[f'(Z)] = E[Z f(Z)].\)

Key Takeaways

  • This is the fundamental Gaussian Stein identity.
  • The Gaussian is the only distribution where IBP gives $E[f’] = E[Zf]$.
  • Growth conditions ensure the boundary term disappears.

(c)(i) Derive the Gaussian moment recursion

Solution

Claim.

For $n \ge 1$:
\(E(Z^{n+1}) = nE(Z^{n-1}).\)

Proof.

Let
\(f(z)=z^n, \qquad f'(z)=n z^{n-1}.\)

Apply Stein’s identity: \(E[f'(Z)] = E[Z f(Z)].\)

Compute:

  • LHS:
    $E[f’(Z)] = nE[Z^{n-1}].$
  • RHS:
    $E[Z f(Z)] = E[Z^{n+1}].$

Thus: \(E[Z^{n+1}] = nE[Z^{n-1}].\)

Conclusion.

Gaussian moments satisfy the classical two-step recursion.

Key Takeaways

  • Odd moments vanish; even moments follow $(2m-1)!!$.
  • The recursion emerges immediately from Stein’s identity.

(c)(ii) Prove product version

\(E[f'(Z) g(Z)] = E[Z f(Z) g(Z)] - E[f(Z) g'(Z)].\)

Solution

Claim.

The identity holds under the polynomial growth assumptions.

Proof.

Step 1. Define $h(z)=f(z)g(z)$.
Then by the product rule: \(h'(z)=f'(z)g(z)+f(z)g'(z).\)

Step 2. Apply Stein’s identity to $h$.
\(E[h'(Z)] = E[Z h(Z)].\)

Expand: \(E[f'(Z)g(Z)] + E[f(Z)g'(Z)] = E[Z f(Z) g(Z)].\)

Step 3. Rearrangement gives the result. \(E[f'(Z)g(Z)] = E[Z f(Z) g(Z)] - E[f(Z) g'(Z)].\)

Conclusion.

The product-rule version of Stein’s identity follows immediately.

Key Takeaways

  • This identity is the Gaussian analog of the product rule for integration by parts.
  • Extremely useful for computing covariances and deriving orthogonality relations (e.g., Hermite polynomials).
  • Appears repeatedly in advanced probability and stochastic processes.
Edit this page on GitHub

Comments