Test 1,000 worthless signals at the 5% level and expect 50 to look significant.
Backtesting is multiple testing by another name, and this is the central problem in quantitative research. Every parameter tuned, variation tried and date range checked counts as a test, including the ones you did not write down.
Corrections: Bonferroni (divide alpha by the number of tests - conservative), false discovery rate (usually more appropriate), and above all genuine out-of-sample validation.
The interview question: "best of 100 backtests has t = 3 - is it real?" The expected maximum from pure noise is around 3, so no.