Home/Learn
Seven backtesting mistakes that make a strategy look better than it is
On this page
1. Look-ahead bias2. Survivorship bias3. Ignoring costs4. Overfitting5. Trying many ideas and keeping the winner6. No untouched test period7. Comparing with nothingAfter the backtestA backtest answers one question: how would these rules have done on past prices, under the assumptions you gave it? Most backtests that look great have a flaw in those assumptions. These are the most common ones.
1. Look-ahead bias
The rules use information that wasn't available yet, like trading at today's open on a signal computed from today's close. Check that every decision uses only data from before the moment of the trade.
2. Survivorship bias
Testing only on stocks that exist today leaves out the companies that went bankrupt or were delisted. The past looks safer than it was. Use data that includes the losers, or prefer broad ETFs whose history is complete.
3. Ignoring costs
Spreads, fees and slippage are small per trade and huge over hundreds of trades. A strategy that trades often can turn from profitable to losing once realistic costs are added. Test it again with costs doubled; if the edge disappears, it was thin to begin with.
4. Overfitting
Adjust enough settings and any strategy fits the past perfectly, and then fails on new data. Prefer simple rules with few settings, and be suspicious of a result that only works at one exact value.
5. Trying many ideas and keeping the winner
Test 200 random strategies and a few will look excellent by luck alone. The more variants you tried, the higher the bar the winner must clear. Write down how many you tested, and say so when you report the result.
6. No untouched test period
Set aside a stretch of history before you start, and don't look at it until your rules are final. If the strategy only works on the data you tuned it on, you've learned that it fits that data, nothing more.
7. Comparing with nothing
A strategy that earned 6% a year sounds fine until you see that simply holding an index fund, or even Treasury bills, did as well with less effort. Always compare with buy-and-hold and with a cash-like benchmark, and look at the worst drop, not just the average.
After the backtest
Even a clean backtest is only a hypothesis. The next step is paper trading: run it forward on live prices and check whether it behaves the way the backtest said. See what paper trading can and can't tell you.
General education, not investment advice. Trading involves risk of loss; backtests and paper results are hypothetical and do not predict future results.