How to read a backtest report

Reading a backtest report in layers: prioritize out-of-sample and overfitting-aware evidence because in-sample quality can be misleading. Give early attention to the deflated Sharpe ratio, as it adjusts Sharpe interpretation for multiple testing and selection effects. Use core risk-adjusted measures such as Sharpe and Calmar to evaluate return efficiency under volatility and drawdown lenses. Treat descriptive statistics as context rather than primary proof of robustness.

A standard backtest report can contain dozens of statistics. The practical problem is not whether each number is valid in isolation, but which numbers deserve attention first when deciding whether a strategy’s historical results are likely to generalize. The most defensible reading order is: start with out-of-sample evidence, then review core risk-adjusted return measures, and only then use secondary descriptive statistics as context rather than as primary decision criteria.

1) Start with out-of-sample evidence

Statistics derived from data not used to fit the strategy should carry more weight than in-sample figures.

Sonar’s backtest-overfitting audit explicitly frames the central problem: a strategy can look strong in-sample simply because it has adapted to historical noise rather than persistent structure. The tool is built to assess the extent to which a backtest may be overfit, which makes out-of-sample behavior the first place to look when interpreting any report. In practical terms, if a report separates training and test results, the test or out-of-sample section should be reviewed before almost anything else because it is more relevant to the question a trader actually cares about: whether the observed edge survives beyond the data used to construct it.

Prioritizing out-of-sample evaluation helps detect when in-sample backtest quality is misleading.

2) Among headline performance statistics, give special weight to the deflated Sharpe ratio

Within the out-of-sample-first framework, the deflated Sharpe ratio deserves special attention because it addresses a specific weakness of ordinary backtest reading: impressive-looking Sharpe ratios can arise by chance when many variants, parameters, or ideas have been tested.

According to Sonar’s glossary, the deflated Sharpe ratio is intended to adjust the interpretation of a Sharpe ratio for multiple testing and non-normality effects. That makes it a more conservative measure than raw Sharpe when a researcher has searched across many candidate strategies. In other words, it does not simply ask whether returns were high relative to volatility; it asks whether the observed Sharpe is still statistically meaningful after accounting for the fact that many attempts may have been tried.

This is why the deflated Sharpe ratio is useful early in the reading process rather than late. A standard Sharpe ratio can summarize risk-adjusted return, but the deflated Sharpe ratio helps answer whether that summary may be inflated by selection bias. In a research workflow where many hypotheses are screened, that distinction matters. If a report includes both, the deflated Sharpe ratio is the more informative screening statistic because it is closer to the overfitting question introduced above.

3) Then review core risk-adjusted return measures such as Sharpe and Calmar

After checking whether the backtest has survived an out-of-sample or overfitting-aware lens, the next level is core risk-adjusted performance. Using normalized measures rather than raw return figures alone is recommended.

### Sharpe ratio

Sharpe ratio remains one of the standard ways to compare return relative to variability. Its value in a report is that it compresses two dimensions into one: reward and dispersion. Even without making any promise about future results, it helps an analyst ask whether the path of returns was efficient or merely high in absolute terms.

But Sharpe should be read with restraint. Sonar’s deflated Sharpe ratio glossary exists precisely because raw Sharpe can be overstated in research settings that involve repeated trials. So Sharpe matters, but less as a standalone verdict than as a baseline statistic that should be interpreted alongside overfitting-aware adjustments.

### Calmar ratio

Calmar ratio belongs in the next layer because it introduces a different and often highly practical risk lens: drawdown. Where Sharpe is volatility-based, Calmar compares return to maximum drawdown. That can matter when a strategy’s smoothness understates the severity of peak-to-trough losses. A report with an acceptable Sharpe but poor drawdown efficiency may look less robust once Calmar is considered.

For that reason, Sharpe and Calmar complement each other rather than compete. One emphasizes variability of returns; the other anchors interpretation to the worst historical loss path. Reading both after the out-of-sample section gives a more balanced picture of the trade-off a backtest is actually showing.

4) Treat descriptive statistics as secondary context

Many report fields are useful, but not equally probative. Counts of trades, percentage of winning periods or trades, average trade, turnover, exposure, and similar descriptors can help explain how a strategy behaves. What they should not do is override stronger evidence from out-of-sample and risk-adjusted metrics.

The reason is simple: descriptive statistics tell you what the backtest looked like, but they do not by themselves answer whether the result is robust. A strategy can have attractive-looking descriptive properties and still fail an overfitting-aware test. Conversely, a strategy may have unremarkable descriptive numbers while showing better evidence of statistical credibility.

Descriptive statistics belong later in the reading process because they are explanatory context, not primary evidence of validity.

A practical reading order

A defensible workflow for reading a standard backtest report is:

1. Check whether the report includes out-of-sample evaluation. If it does, read that first. 2. If available, review the deflated Sharpe ratio early. It directly addresses the risk that apparent quality comes from multiple testing or selection bias. 3. Review core risk-adjusted measures next, especially Sharpe and Calmar. Use them to judge efficiency of returns under both volatility and drawdown lenses. 4. Only then examine secondary descriptive statistics. Use them to understand implementation and behavior, not to establish credibility on their own.

The hierarchy is conceptual: prioritize out-of-sample and overfitting-aware evidence, then interpret core risk-adjusted measures, then use descriptive statistics as supporting context.

In short: the most reliable way to read a backtest report is not to start with the most flattering number. Start with the numbers least likely to be contaminated by fitting, then move to risk-adjusted summaries, and leave the descriptive decoration for last.

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.