Exercise: Verifying a Radon-Nikodym Derivative — Normal Shift

Prerequisites: Radon-Nikodym Theorem, Moment Generating Functions

Problem

Under $\mathbb{P}$, let $X \sim \mathcal{N}(0, 1)$. Define:

1. Verify $\mathbb{E}^{\mathbb{P}}[Z] = 1$. (Hint: MGF of the standard normal at $\theta$ is $e^{\theta^2/2}$.)

2. Define $\mathbb{Q}$ by $d\mathbb{Q}/d\mathbb{P} = Z$. Compute $\mathbb{Q}(X \le x)$ by the change-of-measure formula and identify the distribution of $X$ under $\mathbb{Q}$.

3. Compute $\mathbb{E}^{\mathbb{Q}}[X]$ and $\mathbb{E}^{\mathbb{Q}}[X^2]$ using the change-of-measure formula $\mathbb{E}^{\mathbb{Q}}[g(X)] = \mathbb{E}^{\mathbb{P}}[Z\cdot g(X)]$, and verify they match the moments of $\mathcal{N}(\theta, 1)$.

4. Importance sampling application. Suppose you want to estimate $\mathbb{P}(X > 4)$ for $X \sim \mathcal{N}(0, 1)$ under $\mathbb{P}$. Direct Monte Carlo is terrible: most samples fall below $4$. Use the change of measure with $\theta = 4$ to write an importance-sampling estimator for $\mathbb{P}(X > 4)$: $\mathbb{P}(X > 4) = \mathbb{E}^{\mathbb{Q}}\!\left[\frac{1}{Z}\cdot \mathbf{1}_{X > 4}\right] = \mathbb{E}^{\mathbb{Q}}\!\left[e^{-\theta X + \theta^2/2}\mathbf{1}_{X > 4}\right],$ where under $\mathbb{Q}$, $X \sim \mathcal{N}(4, 1)$. Simulate $10^5$ samples under $\mathbb{Q}$ and compare the tail-probability estimate to the exact value $\Phi(-4) \approx 3.17\cdot 10^{-5}$.

Hint

For part 4, compare the IS estimator's variance to naive Monte Carlo. Naive MC with $10^5$ samples will typically produce zero hits; IS converges rapidly.