Why Most Backtests Lie
The purpose of a backtest is to estimate how a strategy would have performed in the past, in order to predict how it might perform in the future. But a long list of methodological errors can make a losing strategy look like a winning one. Understanding these mistakes is not optional — it's the difference between real edge and self-deception.
Mistake 1: Look-Ahead Bias
Look-ahead bias occurs when your backtest inadvertently uses data that would not have been available at the time of the trade. It is the most damaging error in algorithmic backtesting.
Common sources
- Using the close price of a candle that is still forming to trigger an entry
- Calculating an indicator on the current candle and entering on the same candle's open
- Using daily OHLCV data where the high and low within the candle are unknown at the time of a candle-open entry
Fix
Always use confirmed candle data. Log the signal on candle close, enter on the next candle's open. This adds one candle of lag but eliminates the bias entirely.
Mistake 2: Overfitting (Curve-Fitting)
Overfitting is the optimisation of parameters to the historical test period until the strategy appears to perform perfectly. The result is parameters that are tuned to random noise in past data, not genuine market structure.
Warning signs
- You tried 50+ parameter combinations and accepted the best result
- The strategy has very high backtest returns but with parameters no one would have chosen intuitively (e.g., RSI(17), EMA(73))
- Adding more parameters always improved results
Fix
Use walk-forward optimisation: split your data into equal in-sample and out-of-sample windows, optimise on in-sample, verify on out-of-sample. Repeat by rolling the window forward. If results collapse on out-of-sample data, you have overfit.
Mistake 3: Ignoring Transaction Costs
Every trade incurs fees. Binance's standard maker/taker fee is 0.1%. For a strategy with 200 trades per year, this alone costs 20% annually before any gain or loss. Slippage (the difference between expected and achieved fill price) adds another 0.05–0.1% per trade in normal conditions — more during high volatility.
Fix
Configure realistic fees in your backtesting engine. Add at least 0.15% round-trip cost (0.1% fees + 0.05% slippage) to every trade. If the strategy is not profitable after this adjustment, it is not suitable for live trading.
Mistake 4: No Out-of-Sample Test
Testing and optimising over the same dataset produces results that describe the past perfectly but generalise poorly. Every strategy must be validated on data it has never "seen."
Fix
Reserve 30% of your historical data as a hold-out test set before you begin optimisation. Touch it only once, at the end, to produce a final performance estimate.
Mistake 5: Survivorship Bias in Asset Selection
If you backtest on the top 10 cryptocurrencies by current market cap, you are only testing on coins that survived and appreciated. The coins that failed or delisted are missing from the data.
Fix
Either test on major liquid pairs only (BTC, ETH) where survivorship is less distorting, or acknowledge that your return estimates are inflated and adjust expectations downward for live trading on smaller altcoins.
Mistake 6: Unrealistic Position Sizing
Many backtests use a fixed dollar amount per trade (e.g., $1,000) rather than a % of equity. This means that after a drawdown, the bot continues risking the same absolute dollar amount — which is a larger % of the diminished portfolio. The equity curve is therefore not accurately modelled.
Fix
Use percentage-of-equity position sizing in every backtest. Both your gains and your losses compound realistically, giving you an accurate representation of the maximum drawdown you would actually experience.
Mistake 7: Testing Only on Bull Markets
Crypto had extraordinary bull runs in 2017, 2020–2021, and 2023–2024. A strategy that simply buys every dip will show phenomenal results if backtested exclusively over these periods. Bear markets and choppy sideways markets are where naive strategies are destroyed.
Fix
Backtest across at least one full cycle: a bull market, a bear market, and a consolidation period. For BTC/USDT, testing from January 2018 through the end of 2019 covers a severe bear market and is an essential validation period. A strategy that only works in bull markets is not a strategy — it's a bull market.
The Gold Standard Backtest Checklist
- ✅ Signals fire on confirmed candle close, entries on next open
- ✅ 0.15%+ round-trip cost included
- ✅ 30% out-of-sample hold-out never touched until final validation
- ✅ Test period covers at least one bear market
- ✅ Position sizing is % of equity, not fixed dollar amount
- ✅ Parameter choices are justifiable without backtest results
- ✅ Walk-forward test confirms out-of-sample stability