Week 8: Test for Robustness and Common Errors

A good historical result does not automatically mean a strategy is reliable. Learn to identify the classic failure modes:

The five classic backtesting errors

• Overfitting — parameters are adjusted repeatedly until the strategy perfectly matches historical data. Testing hundreds of moving-average combinations and picking the single best result produces a misleading backtest.
• Look-ahead bias — the test uses information that was not available at trade time, such as a candle's final close before the candle ended.
• Survivorship bias — testing only companies that are successful and listed today, excluding those delisted or merged during the period.
• Data mining — hundreds of ideas are tested but only the best result is presented, without acknowledging the failures.
• Insufficient sample size — 12 profitable historical trades is not strong evidence of repeatability.

Practical robustness checks

• Change indicator periods slightly — a small parameter change should not collapse the system.
• Add higher slippage and re-run.
• Test different market phases — trending, range-bound, volatile and quiet.
• Test related instruments.
• Keep a portion of data out of development (out-of-sample).
• Check whether a few trades produced most of the profit.

Week 8 outcome

You should know whether your strategy shows reasonable stability or merely fits one historical period.

This lesson is for educational purposes only. It is not  investment advice, a recommendation or an assurance of returns. Trading and derivatives involve substantial risk, and backtested or paper-trading results do not guarantee future performance.