Research-side interviews probe these because they are the difference between a backtest and a strategy.
Survivorship bias
Testing on today's index constituents excludes every company that went bankrupt, was delisted, or was acquired. The sample is conditioned on survival, and survival correlates with returns.
The effect is large - studies have found it inflates measured equity returns by a meaningful margin per year. Fund databases have the same problem: poor performers close and disappear from the record.
The fix: point-in-time constituent data that reflects what was actually in the index on each date, including entities that no longer exist.
Look-ahead bias
Using information that was not available when the decision would have been made.
Common instances:
- Restated financials. Reported earnings are revised; using the final revised figure to trade on the original announcement date is look-ahead.
- Reporting lags. Quarterly data is available weeks after quarter end, not on the last day of the quarter.
- Index membership applied before the change was announced.
- Full-sample normalisation - standardising a feature using the mean and standard deviation of the entire dataset leaks future information into every observation.
That last one is subtle and extremely common in practice.
Selection and reporting bias
The strategies you read about are the ones that worked. Every published anomaly survived a filter you cannot see, which connects directly to multiple testing.
Backfill bias
When a database adds a fund or a security, it often backfills history - and entities are added after they have performed well. The early history in a database is systematically favourable.
Overlooking costs
A backtest without transaction costs, market impact, borrow costs and slippage is not a backtest. High-turnover strategies routinely go from impressive to negative once costs are included.
In the interview
"Your backtest shows a Sharpe of 3. What do you check first?"
A strong answer runs through this list: point-in-time data, look-ahead in feature construction, realistic costs, how many variants were tried, and whether the result holds out of sample. Naming survivorship and look-ahead unprompted is exactly the instinct research desks hire for.
Practise in statistics.