Free tool

Backtest evidence checklist

It is easy to make a backtest look excellent when the idea underneath it is worthless. These 22 checks are the ones that separate a test you can act on from one that flatters a bad idea. Tick what is true of your own strategy. Nothing is sent anywhere, and your answers stay in this browser.

Passing every check does not mean a strategy will make money. It means the test was honest, which is a different claim and the only one anybody can make for you.

0 / 22 Tick the checks that are true of your strategy.

Costs are in the test

A strategy that trades often is the one most flattered by leaving costs out. This is the cheapest way to turn a losing idea into a winning chart.

Commission or fee applied to every trade, entry and exitNot once per round trip. Both sides cost.The spread applied at the price you would actually getYou buy at the ask and sell at the bid, never at the mid.Slippage assumed, and larger for larger ordersThe price moves while you fill. Size makes it worse, and a backtest that fills instantly at the quoted price is describing a market that does not exist.Overnight funding or swap applied to positions held overnightOn leveraged instruments this can quietly exceed the edge.The test still works with costs doubledIf doubling costs kills it, the edge was the cost assumption.

The sample is big enough to mean anything

Thirty trades tell you almost nothing. The error bars around a win rate estimated from thirty samples are wide enough to contain both a real edge and pure luck.

At least a hundred closed trades, or a written reason why fewer is defensibleFewer can be legitimate for a slow strategy, but then say so and widen your doubt.The window covers a rise, a fall and a flat stretchA rule tested only through one direction has been tested against one market, once.Removing the best five trades does not destroy the resultIf it does, you have five lucky trades and a story, not a strategy.The price data covers the whole window with no silent gapsA missing week is a week your rule was never tested on and will not know it.

The parameters were not tuned to death

Try enough variations and one of them will look brilliant by chance alone. That is not a discovery, it is a search result.

You know how many parameters the rule hasMore knobs, more ways for the fit to be an accident.You know how many variations you tried in totalIncluding the ones you abandoned. Especially those.Neighbouring parameter values give similar resultsA stable plateau survives the real world. A lone spike is a coincidence with a number attached.The value you chose was not simply the single best of a large gridThe peak of a grid is where luck accumulates.

It was tested on data it never saw

Tuning and testing on the same data is memorising the answers and then sitting the same exam. This is the single most common way honest people produce dishonest backtests.

A slice of data was held back before you started, and never looked atHeld back afterwards is not held back.The out-of-sample result was accepted once, not re-run after adjustmentsEvery re-run after a change spends the evidence. The second look is no longer out of sample.A walk-forward or rolling test was run, not just one splitOne split can be a lucky split.

Risk was measured, not assumed

A result without its worst moment attached is half a result.

You know the deepest peak-to-trough fall, and you could sit through itThe number that ends most strategies is the one the trader could not stomach.Position size follows a written rule, not a feelingSizing decides the outcome more often than entries do.You know what happens on a gap, a halt, or a fill you do not getBacktests fill everything. Markets do not.

You wrote down what would make you stop

Deciding this in advance is the difference between a plan and a hope.

Invalidation criteria written down BEFORE any money is committedFor example: a fall past a stated depth, or a stated number of losing months.A forward test on live prices with no money at riskThe only test that includes live feeds, real spreads and your own operational mistakes.The full record is kept, losing periods includedA record with the bad months deleted is not a record.

What to do with the gaps

An unticked box is not a failure, it is the next piece of work. Fix the cheapest one first: costs and the held-back slice cost nothing but discipline, and between them they catch most of what goes wrong.

Sonar Sciences enforces this list by default rather than leaving it to discipline: costs are always on, the out-of-sample gate decides before you can commit money, and the record keeps its losing periods. The Studio is free. Open the Studio →