Research-side interviews frequently ask you to design a test rather than to compute something. This is what they are looking for.
Start with falsification
Before running anything, write down what result would make you abandon the idea. If no result would, you are not doing research.
This single habit prevents most of the failure modes below, and stating it first in an interview is a strong opening.
Hold out data properly
Split into training, validation and test. The test set is looked at once, at the end.
The discipline is harder than it sounds: every time you check test performance and then adjust, that set becomes part of your training data. Researchers who check the holdout repeatedly have no holdout.
Time series need care
Standard k-fold cross-validation is invalid for time series - it trains on future data to predict the past.
Use walk-forward validation: train on a window, test on the period immediately after, roll forward. Also leave a gap between train and test periods when features have look-ahead spans, to avoid leakage across the boundary.
Pre-register the decision rule
Decide the success criterion before seeing results. "Sharpe above 1 out of sample after costs, over at least three years" is a rule. "It looks promising" is not.
Without this, the goalposts move, and they always move in the direction that keeps the idea alive.
Sample size
Work out in advance how much data you need to detect the effect size you care about. Given that Sharpe estimation error is roughly 1/sqrt(years), many strategy ideas simply cannot be validated with available history - and knowing that before you start saves months.
Control for the obvious
A new signal should be tested against existing ones, not just against zero. A signal that is 90% correlated with a factor you already trade adds nothing, however significant it looks in isolation.
The interview question
"How would you test whether this signal works?"
Structure the answer: state the hypothesis, state what would falsify it, describe the data and its point-in-time integrity, describe the split and why it respects time, state the decision rule in advance, and say how you would control for existing signals and costs.
That structure matters more than any individual choice within it.
Practise in statistics.