Validation is the ordered set of tests a strategy must survive before real money touches it. This is the complete standard: five gates, run in the order that catches lies fastest and wastes the least research effort.
Run no backtest without spreads, slippage, and funding included. Costs are not a refinement to add later; they are the first filter, because a large fraction of apparent edges are smaller than the cost of trading them, and every hour spent researching a pre-cost mirage is wasted. If the strategy cannot pay the spread in simulation, nothing downstream matters.
Before reading any metric, read the trade count and the window composition. The strategy needs enough independent trades across enough regimes for its statistics to be estimates rather than anecdotes. A thin sample does not fail this gate; it simply reclassifies the work as a hypothesis awaiting more evidence, which changes what you are allowed to conclude.
Perturb every free parameter and re-run. Performance should degrade gracefully in a neighborhood around the chosen settings; a strategy that only works at exactly one configuration has memorized history, which is the signature of overfitting. Prefer fewer parameters over better ones: every knob added is evidence diluted.
Judge the frozen rules on data research never touched: a chronological holdout window, or better, a full walk-forward analysis whose stitched unseen segments test the tuning process itself. This verdict is accepted, not negotiated. Returning to tune the rules after seeing a failed holdout converts the holdout into training data and the validation into theater. Correct the headline statistics for selection while you are here: the deflated Sharpe ratio asks whether the result survives the number of variants tried, which is the question every impressive backtest must answer.
For a strategy that passed the first four gates, quantify its risk honestly. Monte Carlo resampling converts the single equity curve into a distribution and reads drawdown at the tail percentiles, which is what position sizing should be built on. Double the assumed costs and confirm survival. Then paper trade: live market, real prices, zero capital, because the transition from historical data to a live feed surfaces its own class of failures, from data differences to execution divergence.
A validated strategy is one whose rules were explicit before testing, whose backtest paid full costs, whose sample supports its statistics, whose parameters sit on a plateau, whose out-of-sample verdict was positive and accepted once, and whose risk was sized on the distribution rather than the curve. That sentence is the entire evidence standard behind Sonar Sciences: the Studio enforces it during research, and nothing the platform reports has survived anything less.
Validation is the ordered testing that separates a real edge from a fitted backtest: costs included from the first run, a sample large and varied enough to support conclusions, parameters that survive perturbation, a positive out-of-sample or walk-forward verdict accepted exactly once, and risk sized on a stress-tested distribution rather than a single curve.
The mechanical tests run in minutes on a platform that automates them; the calendar time goes to forward confirmation. A reasonable rhythm is research and historical validation first, then a paper-trading period on live prices to surface execution divergence before any capital is committed.
Yes, always. Paper trading is the only test that includes live data feeds, real spreads as they happen, and the operational failure modes no historical simulation contains. It costs nothing but time, and it is the last cheap place to discover that the backtest and the market disagree.
Bring one strategy you already trust. The Studio validates it against four years of real data, costs included, for free.
Start building