How to bootstrap confidence intervals for a backtest

Bootstrap resampling provides a robustness-oriented way to turn a single backtest metric into a distribution and reason about uncertainty. It can be paired with overfitting controls and Deflated Sharpe Ratio concepts.

The core idea is simple: a backtest usually reports single values for metrics like Sharpe ratio or maximum drawdown, but a single realization does not show how sensitive those metrics are to the particular sequence of outcomes observed in sample. Bootstrap methods address that by resampling observed results to create many alternative realizations, producing a distribution for the metric rather than one point estimate.

A practical workflow is: 1. Start with the realized trade-level or return-level series from the backtest. 2. Resample that series many times with replacement. 3. Recompute the metric of interest on each resample, such as Sharpe ratio or maximum drawdown. 4. Use the resulting empirical distribution to form percentile bands or other confidence intervals.

This converts statements like “the backtest Sharpe is 1.2” into a more decision-useful statement like “under resampling of the observed outcomes, Sharpe falls in this range.” The same logic applies to path-dependent quantities such as drawdown, where the full resampled path can be used to recompute the worst peak-to-trough loss each time.

The Sonar backtest-overfitting audit tool supports the broader reason to do this. Its purpose is to assess whether apparent backtest quality is likely to be an artifact of selection, multiple testing, or luck. A single metric observed once is especially vulnerable to that problem. Building a bootstrap distribution makes it easier to distinguish a stable result from one that depends heavily on a particular historical sequence. In other words, bootstrap confidence intervals are not a replacement for an overfitting audit, but they complement it by showing the range of plausible metric values supported by the observed sample.

The glossary entry on the Deflated Sharpe Ratio adds an important caution. It explains that raw Sharpe ratios can be misleading when many trials, parameter choices, or strategy variants have been explored. Deflation adjusts the interpretation of Sharpe for selection bias and non-normality considerations. That means a bootstrap interval around a raw Sharpe estimate should not be interpreted in isolation if the strategy emerged from extensive searching. A narrow bootstrap interval can still coexist with an overstated Sharpe if the research process involved multiple testing. For rigorous validation, uncertainty from resampling and bias from selection should be considered together.

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.