Why public track records skew positive

Public track records are positively skewed because publication is a filtered subset of research, continued visibility depends on meeting publishing criteria, and multiple testing can inflate the metrics of the survivors. The backtest‑overfitting audit and deflated Sharpe ratio are tools to address these selection problems.

Publicly visible strategy histories can look better than the average research attempt because visibility itself is often conditional: strategies with disappointing results are more likely to stop being shown, while the survivors keep accumulating public observations.

Sonar’s research-to-publishing framework describes a gated process between research output and what becomes publicly documented. The framework separates research, validation, and publishing, which means publication is not a neutral mirror of all attempts. Instead, publication follows review criteria intended to screen for robustness and documentation quality before results are shared publicly. That structure is important for interpreting any public record set: once there is a selection layer between experimentation and publication, the published subset is not the same thing as the full tested universe. A reader looking only at published histories is therefore observing a filtered population rather than all research outcomes.

Publication is an ongoing process, not a one-time event. If continued publication depends on a strategy continuing to satisfy framework criteria, then strategies that degrade, fail validation, or no longer merit publication can disappear from the visible set while stronger‑looking strategies remain. That is the core logic of survivorship‑driven positivity in public track records.

Sonar’s backtest‑overfitting audit tool provides a second reason published scores can look too good: multiple testing can inflate the apparent quality of the strategies that survive selection. The tool is explicitly built to audit research for backtest overfitting, i.e., the risk that repeated trials, parameter searches, and specification choices produce an apparently strong backtest that does not reflect durable signal. This matters for public track records because the strategy that gets published is often selected from many tested variants. If enough variants are tried, the published one can inherit an exaggerated in‑sample score simply because it was the winner of a search process.

The deflated Sharpe ratio is presented as a way to adjust the interpretation of Sharpe‑like results for non‑normality and, crucially, for multiple testing or selection effects. In plain terms, it asks whether a reported Sharpe remains impressive once you account for the fact that many strategies or parameterizations may have been tried. This demonstrates that a public record can be positively skewed not only because weak strategies stop publishing, but also because the visible survivors may have been selected from a larger experimental pool in a way that mechanically boosts observed metrics.

Taken together, these points support a realistic interpretation of public track records:

1. Published strategies are a selected subset of researched strategies, not the full distribution of attempts. 2. Continued visibility can depend on passing a research‑to‑publishing process, so underperforming strategies may cease to be shown. 3. Selection after many tests can inflate apparent backtest quality, which is exactly what overfitting audits and deflated Sharpe‑style adjustments are designed to detect.

For quantitative traders and allocators, the practical takeaway from these sources is methodological rather than promotional: public track records should be interpreted as outputs of a selection and validation pipeline, not as an unbiased sample of all strategy research. To evaluate them realistically, ask what testing universe preceded publication, whether overfitting was audited, and whether a multiple‑testing‑aware metric such as the deflated Sharpe ratio was used when judging the apparent strength of the visible result.

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.