Exercise: Proving the Weak LLN from Chebyshev

Prerequisites: Law of Large Numbers, Expectation and Variance

Problem

Let $X_1, \ldots, X_n$ be pairwise uncorrelated random variables with common mean $\mu$ and common variance $\sigma^2 < \infty$ .

Compute $\mathbb{E}[\bar X_n]$ and $\operatorname{Var}(\bar X_n)$ in terms of $\mu$ , $\sigma^2$ , and $n$ . Explain carefully where the assumption "pairwise uncorrelated" is used.
State Chebyshev's inequality and apply it to $\bar X_n$ to derive the bound $\mathbb{P}(|\bar X_n - \mu| > \epsilon) \le \frac{\sigma^2}{n\epsilon^2}.$
Conclude the weak law of large numbers: $\bar X_n \xrightarrow{\mathbb{P}} \mu$ .
Conceptual. Suppose we drop the "pairwise uncorrelated" assumption and allow $\operatorname{Cov}(X_i, X_j) = \rho\sigma^2$ for $i \ne j$ with some fixed $\rho > 0$ . Compute $\operatorname{Var}(\bar X_n)$ in this case and show that the bound no longer drives to zero. Why does this matter for estimating the mean of an autocorrelated time series (e.g. a momentum signal)?

For part 1, use that

\operatorname{Var}(\sum a_i X_i) = \sum a_i^2 \operatorname{Var}(X_i)

for uncorrelated variables. For part 4, you need the full formula including covariances.

Jump to the solution when you're ready.