# Law of Iterated Expectation

Let $$X$$ and $$Y$$ denote continuous, random, real-valued variables with joint probability density function $$f(X, Y)$$. The marginal density function of $$Y$$ is $$f_Y(y) := \int_{x \in \mathbb R} f(x, y) dx$$. The expectation $$E(Y)$$ of $$Y$$ can be recovered by integrating against the marginal density function. In particular,

$$$\label{eq:E(Y)} E(Y)=\int_{y \in \mathbb R} y f_Y(y) dy = \int_{y \in \mathbb R} \int_{x \in \mathbb R} y f(x, y) \ dx \ dy.$$$

The conditional probability density function of $$Y$$ given that $$X$$ is equal to some value $$x$$ is defined by

$$$\label{eq:cdf} f_{Y \mid X} (y \mid X = x) := f_{Y \mid X} (y \mid x) = \frac{f(x, y)}{f_X(x)} = \frac{f(x,y)}{\int_{y \in \mathbb R} f(x, y) \ dy}.$$$

The conditional expectation $$E(Y \mid X = x)$$ of $$Y$$ given that $$X$$ has value $$x$$ is given by

$$$\label{eq:cond exp} E(Y \mid X = x) = \int_{y \in \mathbb R} y f_{Y \mid X} (y \mid x) \ dy.$$$

But $$E(Y \mid X = x)$$ depends on $$X$$, so in turn is itself a random variable denoted $$E(Y \mid X)$$, whence we can compute its expectation. Now

\begin{align*} E (E (Y \mid X)) &= \int_{x \in \mathbb R} E(Y \mid x) f_X(x) \ dx \\ &= \int_{x \in \mathbb R} f_X(x) \left( \int_{y \in \mathbb R} y f_{Y \mid X}(x, y) \ dy \right) \ dx \quad \text{(by \ref{eq:cond exp} )} \\ & = \int_{x \in \mathbb R} \int_{y \in \mathbb R} f_X(x) \cdot y \frac{f(x,y)}{f_X(x)} \ dy \ dx \quad \text{(by \ref{eq:cdf})}\\ & = \int_{y \in \mathbb R} \int_{x \in \mathbb R} y f(x, y) \ dx \ dy \\ & = E(Y). \quad \text{(by \ref{eq:E(Y)})} \end{align*}

This result is sometimes called the law of the iterated expectation.