How to backtest a trading strategy without fooling yourself
An honest backtest replays your rules bar by bar using only the data available at that moment, holds one position at a time, is measured on tickers that played no part in choosing the rules, and is compared against simply buying and holding. Nearly every impressive backtest fails one of those four conditions. Below is the sequence we follow, and the specific mistakes that inflated our own early numbers.
Updated 2026-09-22 · Thomas Solutions LLC
Step 1 — write the rules down first
Entry condition, exit condition, holding period, and what counts as a win. Fixing this in advance is what stops a test from being quietly adjusted until it looks good. Our rules: enter at the closing price on a signal day, exit at the close a fixed number of trading days later, win means the exit close moved in the signalled direction.
Step 2 — replay walk-forward, with no look-ahead
On each historical bar the engine must see only prices up to that bar. This sounds obvious and is violated constantly — by indicators computed over the whole series at once, by survivorship-filtered ticker lists, and by split or dividend adjustments applied with hindsight. Use split-adjusted session opens, highs, lows and closes, and recompute every indicator inside the loop.
Step 3 — hold one position at a time
This is the error that produces the absurd numbers. If a strategy signals almost daily with a 20-day hold, twenty positions overlap; compounding them as though each were fully funded is arithmetically impossible. Before we fixed it, one ticker reported +19,812%. With one fully-allocated position at a time, and a count of how many signals were skipped because capital was already committed, the results became something a real account could have produced.
Step 4 — hold tickers back
Split your universe. We used 15 tickers to choose filters and held 15 back entirely, measuring those once, afterwards. The held-back numbers were lower — 57.3% hit rate and 0.38 Sharpe versus 58.8% and 0.59 — and they are the ones worth believing. Any change that helps in sample and hurts out of sample gets discarded, which is exactly what happened to a strategy re-weighting scheme that looked excellent in testing.
Step 5 — read the right metrics
- Hit rate — share of positions that moved the signalled way.
- Average return per position — the honest per-trade figure.
- Maximum drawdown — the worst peak-to-trough fall you would have sat through.
- Sharpe ratio — return relative to volatility; 0.5 is modest, above 1.0 is strong.
- Buy and hold — the benchmark that most strategies quietly lose to.
Step 6 — subtract the costs the test ignored
Commissions, bid-ask spread, slippage on thin names, taxes and missed dividends all sit outside a backtest. A 0.2% average gain per position rarely survives them. Treat a marginal result as a negative one.
Do this on your own tickers
The Backtesting page runs this exact procedure on any US-listed ticker over one, two or five years with holding periods of 5 to 30 trading days, reporting hit rate, average and compounded return, drawdown, Sharpe, per-strategy accuracy and the buy-and-hold comparison, plus how many signals were skipped and why. Our own results across 30 tickers are published on the measured performance page, losing names included.
Backtested results are hypothetical, exclude trading costs, and do not indicate future results. Cashora is a market probability signal predictor operated by Thomas Solutions LLC — not a broker, no trades are executed, and nothing here is investment advice.
