Backtesting: Definition
Backtesting is the process of testing a trading strategy’s rules against historical price data to estimate how it would have performed, before risking real capital on it.
Backtesting cannot prove a strategy will work in the future — markets change regime, and history never repeats exactly — but it is the standard first filter for rejecting strategies with no historical edge, unacceptable drawdowns, or statistically insignificant sample sizes.
The backtesting process — 5 steps
- Define the rules precisely. Entry condition, exit condition (both profit-target and stop-loss), position size, and the instrument/timeframe — every rule must be specific enough that two people running the same test on the same data get the same trades.
- Choose a representative historical window. Include multiple market regimes (trending, ranging, high volatility, low volatility) — a strategy tested only during one regime will look artificially strong or weak.
- Run the strategy mechanically over the data, applying the rules exactly as defined, with no discretionary override — the point of backtesting is to remove hindsight bias, which discretionary adjustment reintroduces.
- Include realistic costs — commission, slippage, and (for options) the bid-ask spread — a backtest run at zero cost systematically overstates real-world performance.
- Evaluate the results against the statistics that matter: win rate, average win/loss, Profit Factor, Max Drawdown, and total trade count (see the Max Drawdown glossary entry for how that specific metric is read).
Worked example
A trader backtests a simple rule over 6 months of data: enter long when a 20-period moving average crosses above a 50-period moving average, exit on the reverse cross. The test produces:
- 42 total trades
- 24 winners, 18 losers → Win Rate = 24/42 = 57.1%
- Average win: $180, average loss: $140
- Gross profit = 24 × $180 = $4,320; Gross loss = 18 × $140 = $2,520
- Profit Factor = $4,320 / $2,520 = 1.71
- Net profit = $4,320 − $2,520 = $1,800
A Profit Factor of 1.71 with 42 trades is a reasonable starting point for further validation — the next step is testing the same rules on a separate, untouched date range (out-of-sample data) to check the result wasn’t a product of curve-fitting to this specific window.
Overfitting and lookahead bias
Two failure modes make a backtest look better than a strategy actually is:
- Overfitting (curve-fitting): repeatedly tweaking a strategy’s parameters until the backtest looks good on one specific dataset. The resulting “optimized” strategy is often fit to noise in that dataset rather than a real, repeatable edge, and performs worse on new data.
- Lookahead bias: accidentally using information in the backtest that would not have been available at the time of the actual trade — a common cause is a repainting indicator whose historical signal shifts after the bar closes, making the backtest show entries that could never have been taken live at that exact price.
In-sample vs. out-of-sample testing
The standard defense against overfitting is splitting historical data into two parts: in-sample data used to develop and tune the strategy, and out-of-sample data — reserved, untouched during development — used only afterward to check whether the strategy still performs. A strategy that performs well in-sample but poorly out-of-sample is a strong overfitting signal, regardless of how good the original backtest looked.
How Quantzee’s non-repainting design relates to backtesting
Lookahead bias caused by a repainting indicator is one of the most common and hardest-to-spot backtest-validity problems, because the chart can visually show a “perfect” historical signal that was never actually available in real time. Quantzee’s indicator suite is built to confirm signals only on the closed bar and never redraw historical markers — a design choice made specifically so that a backtest run against Quantzee signals reflects trades that were genuinely achievable at the time, not signals that only look clean in hindsight.
Common mistakes
- Optimizing parameters on the full dataset with no out-of-sample check. This is the single most common cause of a backtest that looks great and then fails live.
- Ignoring transaction costs. Especially damaging for higher-frequency strategies, where commission and slippage can consume most or all of a thin per-trade edge.
- Too small a sample. Fewer than ~30 trades makes win rate and Profit Factor statistically unreliable.
- Not checking for repainting. Refresh the chart and confirm historical signal markers haven’t shifted — if they have, the backtest is not trustworthy.