Sonar Sciences
Start free
Start freeSign in
Free tool

Strategy experiment brief

Write down what you are testing, and what would prove you wrong, before you run it. Then keep the brief and compare.

Backtesting rewards changing your mind after seeing the answer. You try a setting, the curve dips, you try another, and by the tenth attempt the strategy fits the history rather than the market. Nothing in the tooling stops that, because from the inside it feels like work.

A brief written first is the cheapest defence there is. It is borrowed from trial pre-registration in medicine, where the protocol is filed before the data is collected precisely so the hypothesis cannot be edited to match the result. It costs about twenty minutes and it makes the difference between a test and a search.

Nothing you type is sent anywhere. It is held in your browser and exported as a file you keep.

01 / HYPOTHESIS

One sentence, as cause and effect. "Volatility contracts before the London open and expands after it" is a hypothesis. "RSI below thirty" is a setting, and a setting is not a reason: it cannot be wrong, only re-tuned.

02 / MARKET AND WINDOW

Fix this before you look, or the scope becomes another parameter. A strategy that works on one instrument in one year, chosen after testing nine, has told you about the choosing rather than the strategy.

03 / ENTRY

Precise enough that somebody else could run it and get your numbers. If a condition needs a judgment call, it is not testable and the backtest will quietly resolve it in your favour.

04 / EXIT

Target, stop, time limit, and what happens when two fire on the same bar. Ambiguity here is where backtests and live accounts diverge most, because the tester resolves the tie one way and the broker resolves it the other.

05 / RISK AND COSTS

Size and worst acceptable loss decide whether a result is survivable, not just positive. State the spread, slippage and commission the test applies: an edge that only exists at zero cost is a chart.

06 / FALSIFICATION

The most important field, and the one everybody skips. Decide it now, while you have no result to defend. Without it every outcome becomes evidence for the idea and the test cannot fail.

07 / HELD BACK

Name it now and do not open it. A window you have never seen is the only honest test left once you have finished tuning, and it stops being one the moment you peek.

0 of 7 answered

Your brief

This is what gets exported. Keep it somewhere you will find it again, unchanged, and read it back when the test finishes.

Nothing filled in yet.
Then hold yourself to it. When the result arrives, the only questions are whether the falsification condition was met and whether anything in the brief changed while you were testing. A brief edited to match the outcome is worth less than no brief, because it makes a search look like an experiment.

What this deliberately does not do

It does not score your idea, rank it, forecast a return or tell you whether to trade it. Those would all be judgments about your money made by a web page that has seen one paragraph. It writes down what you decided, so that later you cannot quietly un-decide it.

Questions

Does anything I type get sent to Sonar Sciences?

No. The form saves to your browser's local storage so a refresh does not lose your work, and the export is generated in the page. There is no request, no account and no sign-up.

What if I do not know the answer to a field yet?

Leave it and come back. The gaps are informative on their own: an empty falsification field usually means the idea is not yet specific enough to be wrong about, which is worth discovering before you spend a weekend testing it.

Do I need this if I already backtest carefully?

The brief is not about care, it is about order. Writing the falsification condition after seeing the equity curve is a different act from writing it before, even when the same person writes the same sentence.

Next, when you have a result: the backtest evidence checklist works through whether the number you got means anything.