Bootstrap resampling estimates the sampling uncertainty of a statistic by repeatedly resampling the observed data and recomputing the statistic on each resample.
Bootstrap resampling estimates the sampling uncertainty of a statistic by repeatedly resampling the observed data and recomputing the statistic on each resample. In strategy research, the observed data can be trades, daily returns, or another return series from a backtest. The result is an empirical distribution for the statistic of interest, which can be used to form confidence intervals and error bars without assuming the statistic is normally distributed.
The basic mechanism is simple. Start with an observed sample of returns or trades from a strategy. Draw a new sample of the same size by sampling from that observed sample with replacement. Compute the chosen metric on that resampled data, such as a Sharpe ratio or drawdown statistic. Repeat this many times. The collection of recomputed values approximates the sampling distribution implied by the observed data. Confidence intervals can then be constructed from the quantiles of that empirical distribution.
This is useful in strategy validation because trading returns often depart from the conditions that make normal approximation convenient. The Sonar Sciences strategy validation material explicitly emphasizes uncertainty estimation, confidence intervals, and Monte Carlo and bootstrap methods as part of backtest validation and robustness analysis. It also notes that market returns and strategy outcomes can be noisy and that robust evaluation should quantify uncertainty rather than rely on point estimates alone. That makes bootstrap resampling a natural fit when a researcher wants error bars around a reported performance statistic while avoiding a strict normality assumption.
The same strategy validation material also highlights distributional diagnostics such as skewness and kurtosis, along with confidence intervals for metrics including the Sharpe ratio and drawdowns. This matters because skewed or heavy tailed return distributions can make standard parametric intervals less reliable if they are built on a normal approximation. A bootstrap interval instead uses the empirical shape of the observed sample through repeated resampling. In plain terms, it lets the data generate the error bars.
A practical workflow is to pair the bootstrap with the metric that matters for the research question. If the goal is to quantify uncertainty around risk adjusted performance, resample returns and recompute the Sharpe ratio on each bootstrap draw. If the goal is downside risk estimation, recompute a drawdown measure on each draw. The output is not just a single estimate, but a range of plausible values consistent with the observed sample. This gives a direct way to report how sensitive the conclusion is to sampling variation.
Bootstrap resampling also connects naturally to overfitting control. The Sonar Sciences backtest overfitting audit is designed to test whether a strategy likely reflects genuine signal or selection bias from repeated research choices. The tool uses methods such as combinatorially symmetric cross validation and reports diagnostics including a probability of backtest overfitting and a probability of loss. In that setting, bootstrap based error bars can complement the audit by showing whether headline statistics are stable across resampled versions of the observed return path. Wide or unstable bootstrap intervals are consistent with a result that is fragile to sampling variation, which is one of the warning signs that should prompt deeper scrutiny.
The glossary entry on the Deflated Sharpe Ratio adds another layer to this idea. It explains that Sharpe ratios can be inflated by multiple testing, non normal returns, and short sample lengths, and that a corrected evaluation should adjust for those effects. Bootstrap resampling does not replace a deflation or multiple testing correction, but it is aligned with the same principle. Both approaches move the researcher away from taking a single backtest statistic at face value. Instead, they treat reported performance as uncertain and potentially distorted by the data generating process and the research process.
Covered in depth in the Strategy validation & overfitting pillar hub.