Why pre-registering a test keeps you honest

Pre registering pass criteria means deciding the success rules before a backtest runs. It reduces bias by limiting the ability to change metrics or thresholds after results are visible, which helps contain post hoc selection and overfitting.

Pre registering a test means writing down the pass criteria before you run the backtest. The pass criteria are the conditions that determine whether the result is treated as strong enough to investigate further. In practice, this means deciding in advance which metric matters, what threshold counts as a pass, and how many trials or variants are part of the test.

This reduces bias because it removes the freedom to move the goalposts after seeing the results. Once a researcher has looked at a set of outcomes, it becomes easy to favor the metric, threshold, or subset that makes the strategy look better. That flexibility can turn random variation into an apparently convincing finding. Pre registration limits that flexibility by separating the design of the test from the observation of the outcome.

Sonar describes this workflow as moving from research to publishing through structured checkpoints that make methods and decisions explicit before results are presented. That framing matters because the main risk in backtesting is not only calculation error. It is also selective interpretation after the fact. A pre declared pass rule creates an audit trail for what counted as success before any performance numbers were known. That makes later evaluation easier to interpret and harder to reshape around favorable outcomes.

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.