Exercise: Compute the First Adam Update by Hand

Prerequisites: Adam Optimizer

Problem

Let $g_1=4$, $m_0=v_0=0$, $\alpha=0.1$, $\beta_1=0.9$, $\beta_2=0.999$, and $\epsilon=0$. Compute $m_1$, $v_1$, $\hat m_1$, $\hat v_1$, and the parameter step.

Hint