Skip to content
Get started

Training error versus fresh-noise error

You fit an OLS regression y=Xβ+εy = X\beta + \varepsilon with n=40n=40 observations. The design matrix XX is held fixed and has an intercept column plus 44 regressors (full column rank); the errors are i.i.d. with mean 00 and variance σ2=2\sigma^2 = 2.

After fitting, you draw a fresh response vector ynew=Xβ+εnewy^{\text{new}} = X\beta + \varepsilon^{\text{new}} at the same design points, with new noise independent of the old and of the same law. By how much does the expected error on the fresh responses exceed the expected training error? That is, compute E[∥ynew−Xβ^∥2]−E[∥y−Xβ^∥2].\mathbb{E}\big[\lVert y^{\text{new}} - X\hat\beta\rVert^2\big] - \mathbb{E}\big[\lVert y - X\hat\beta\rVert^2\big].

Show hints (3)+
  1. Write Xβ^=Xβ+HεX\hat\beta = X\beta + H\varepsilon with the hat matrix HH.
  2. For white noise, E[ε⊤Mε]=σ2tr⁡M\mathbb{E}[\varepsilon^\top M\varepsilon] = \sigma^2\operatorname{tr}M, and tr⁡H\operatorname{tr}H is the number of columns of XX - intercept included.
  3. The fresh error is εnew−Hε\varepsilon^{\text{new}} - H\varepsilon, two independent pieces.

Answer

Reveal answer →

20

Create a free account to see the full worked solution and track your progress.

Interview format: Hedge Fund Quant, Systematic & Data-Driven Trading

Related questions