This is the technique that unifies most of the others in this module.
Conditional expectation is a random variable
The point people miss. E[X | Y = y] is a number for each y - but E[X | Y] is a function of Y, and since Y is random, so is that function. It has its own distribution and its own expectation.
Grasping this is what makes the next section make sense.
The tower property
E[X] = E[ E[X | Y] ]
In words: to find the average of X, first average X within each value of Y, then average those averages weighted by how likely each Y is.
This is the law of total expectation, and it is the workhorse.
Worked example
You roll a die, then flip that many coins. Expected number of heads?
Conditioning on the die value N:
E[heads | N] = N/2
That is a random variable - it depends on N. Taking expectations again:
E[heads] = E[N/2] = E[N]/2 = 3.5/2 = 1.75
Trying this directly, by summing over every combination of die value and head count, is a page of arithmetic for the same answer.
Choose the conditioning well
The art is picking Y so that E[X | Y] is easy. Good candidates:
- The first step of a process (gives you recursion).
- A hidden parameter, like the die value above.
- The number of trials in a compound experiment.
Conditional variance
The companion identity, which appears in more advanced interviews:
Var(X) = E[Var(X | Y)] + Var(E[X | Y])
Total variance splits into variation within groups plus variation between group means. This is exactly the decomposition behind analysis of variance and behind the bias-variance intuition in modelling.
Where it leads
A martingale is defined by a conditional expectation - the expected next value given everything so far equals the current value. Linear regression is a conditional expectation. Bayesian updating is a conditional expectation. The identity is not an interview trick; it is the spine of the subject.
Practise in conditional probability.