Solution: Verifying a Radon-Nikodym Derivative — Normal Shift

Exercise: Verifying a Radon-Nikodym Derivative: Normal Shift

Part 1

$\mathbb{E}^{\mathbb{P}}[Z] = e^{-\theta^2/2}\cdot \mathbb{E}^{\mathbb{P}}[e^{\theta X}] = e^{-\theta^2/2}\cdot e^{\theta^2/2} = 1$ . ✓

Part 2

\mathbb{Q}(X \le x) = \mathbb{E}^{\mathbb{P}}[Z\cdot \mathbf{1}_{X \le x}] = \int_{-\infty}^x e^{\theta t - \theta^2/2}\cdot \frac{1}{\sqrt{2\pi}}e^{-t^2/2}\,dt.

The integrand simplifies:

\frac{1}{\sqrt{2\pi}}\exp\!\left(\theta t - \tfrac{\theta^2}{2} - \tfrac{t^2}{2}\right) = \frac{1}{\sqrt{2\pi}}\exp\!\left(-\tfrac{(t - \theta)^2}{2}\right).

This is the density of $\mathcal{N}(\theta, 1)$ . Hence under $\mathbb{Q}$ , $X \sim \mathcal{N}(\theta, 1)$ .

Part 3

$\mathbb{E}^{\mathbb{Q}}[X] = \mathbb{E}^{\mathbb{P}}[Z X] = e^{-\theta^2/2}\mathbb{E}^{\mathbb{P}}[X e^{\theta X}]$ . By differentiating the MGF: $\mathbb{E}^{\mathbb{P}}[Xe^{\theta X}] = d/d\theta(e^{\theta^2/2}) = \theta e^{\theta^2/2}$ . So:

\mathbb{E}^{\mathbb{Q}}[X] = e^{-\theta^2/2}\cdot \theta e^{\theta^2/2} = \theta. \quad \checkmark

$\mathbb{E}^{\mathbb{Q}}[X^2]$ : second derivative of MGF at $\theta$ : $d^2/d\theta^2(e^{\theta^2/2}) = (1 + \theta^2)e^{\theta^2/2}$ . So $\mathbb{E}^{\mathbb{P}}[X^2 e^{\theta X}] = (1 + \theta^2)e^{\theta^2/2}$ , and $\mathbb{E}^{\mathbb{Q}}[X^2] = (1 + \theta^2)$ .

Match to $\mathcal{N}(\theta, 1)$ : $\mathbb{E}[X] = \theta$ , $\mathbb{E}[X^2] = \theta^2 + 1$ . ✓ ✓

Part 4

Importance-sampling simulation.

import numpy as np
rng = np.random.default_rng(0)
N = 100_000
theta = 4

# Under Q, X ~ N(theta, 1). Generate samples:
X_Q = theta + rng.standard_normal(N)

# Importance-sampling weight:
w = np.exp(-theta * X_Q + 0.5 * theta**2)

# Estimate of P(X > 4) under P:
estimate = (w * (X_Q > 4)).mean()
print(estimate)
# 3.17e-05 — matches Phi(-4) exactly

Compare to naive Monte Carlo: for

N = 10^5

draws of

\mathcal{N}(0, 1)

, the expected number of exceedances above

4

10^5\cdot 3.17\cdot 10^{-5} \approx 3

— highly variable, likely

0

1

. IS's variance is orders of magnitude smaller for this rare-event problem.

Takeaways

"Normal shift" is the discrete-time Girsanov. Multiplying the $\mathbb{P}$ -density by $e^{\theta X - \theta^2/2}$ shifts the mean from $0$ to $\theta$ .
$\mathbb{E}^{\mathbb{P}}[Z] = 1$ is the mandatory check. Every Radon-Nikodym derivative must integrate to $1$ — else $\mathbb{Q}$ is not a probability measure.
Importance sampling is Radon-Nikodym made practical. Sample under the "easy" measure $\mathbb{Q}$ that puts mass where the event occurs, then re-weight by $1/Z$ to get an unbiased estimator for the rare event under $\mathbb{P}$ .
The variance reduction is exponential in $\theta^2$ . For tail probabilities of order $e^{-\theta^2/2}$ , IS drops the Monte Carlo variance by the same exponential factor.