How does backtesting work

Backtesting works by replaying a strategy’s rules over historical data under a specific simulation design. The result is highly sensitive to the chosen dataset, transaction cost model, and event sequence because each setting changes the simulated trade path and portfolio statistics. Sonar’s materials present these choices as core parts of research design and use the backtest overfitting audit tool and deflated Sharpe ratio to help interpret results after multiple testing and specification search.

Backtesting is the process of applying a strategy’s rules to historical market data to see how those rules would have behaved under a specific simulation setup. In practice, a backtest is not a single objective replay. It is a model. The output depends on what data is used, how orders are simulated, what trading costs are assumed, and the exact sequence in which information, signals, and fills are processed.

Sonar describes a backtest as a workflow that turns a research hypothesis into a fully specified simulation. The strategy logic is evaluated on historical bars, positions are updated according to the rules, and the resulting path of trades and portfolio values is summarized with risk and return statistics. That means the backtest result is produced by a chain of assumptions, not by the strategy idea alone. Changing any link in that chain changes the answer. [1]

Historical data is the first major source of sensitivity. A strategy can be tested on different instruments, universes, date ranges, bar sizes, and data fields. Each choice changes the set of opportunities the strategy is allowed to act on. Even when the trading rule is unchanged, a backtest on one sample period can differ materially from a backtest on another sample period because the underlying market conditions are different. Sonar’s research fundamentals emphasize that the research dataset and sample definition are central parts of a valid test design, because a strategy is always being evaluated on the market history that was chosen for it. [1]

Data handling choices also matter inside the sample. Bar aggregation can alter when a signal appears. Corporate actions and survivorship treatment can affect price histories and universe membership. Missing data rules can change whether a signal is generated or skipped. If the backtest uses daily bars, the strategy only sees the information encoded in those bars. If it uses intraday bars, the same rule may trigger at different times and at different prices. These are not cosmetic differences. They change the simulated trade list and therefore every downstream metric. Sonar’s framework presents these settings as part of the specification that defines what exactly is being tested. [1]

Transaction costs are another direct driver of results. A backtest can include commissions, fees, slippage, financing assumptions, and other implementation frictions, or it can omit them. If costs are omitted, the simulation measures gross behavior. If costs are included, the simulation measures net behavior under the chosen cost model. Sonar’s fundamentals material treats costs as an explicit parameter of the test rather than an afterthought, because even small per trade assumptions can compound materially when turnover is high. [1]

The effect is mechanical. A strategy that trades frequently can look very different under low slippage versus high slippage. A rule that appears viable before commissions may weaken once commissions are applied. A portfolio that rebalances often may be especially sensitive to spread and impact assumptions. The key point is not that one cost model is universally correct. The key point is that the cost model is part of the experiment. Change the model and the measured outcome changes with it. [1]

The order of simulated events is equally important. A backtest must decide when data becomes visible, when a signal is computed, when an order is sent, and when that order is assumed to fill. If those steps are sequenced incorrectly, the test can accidentally use information that would not have been available at the time of decision. Sonar’s fundamentals stress that a valid simulation needs a clear event timeline so that rule evaluation and execution occur in the proper order. [1]

This sequencing issue is easiest to see bar by bar. Suppose a strategy uses the close of a bar to generate a signal. If the simulation lets that same signal trade at that same close without a justified execution rule, the test may be giving itself access to information and execution conditions that are not aligned. If instead the model computes the signal after the close and executes at the next bar’s open, the trade path changes. Entry prices change, holding periods can change, and all summary statistics change with them. The strategy rule may be identical in wording, but the event ordering produces a different backtest. [1]

For the same reason, stop logic, limit logic, and intrabar assumptions can alter results. When both a target and a stop are reachable inside one bar, the backtest needs a rule for which event is processed first. On coarse data, that rule can determine whether a trade is recorded as favorable or unfavorable. Sonar’s methodology treats such implementation details as part of the simulation design because the engine must resolve them somehow, and different resolutions can lead to different outputs. [1]

Sonar’s backtest overfitting audit tool is built around the idea that a strong backtest can reflect extensive specification search rather than a durable effect. The tool is designed to audit whether repeated trial and error over parameter combinations, features, or rule variants may have inflated the apparent quality of the selected strategy. In other words, the more choices a researcher makes while searching for a good historical result, the more important it becomes to adjust how that result is interpreted. [2]

That is where the deflated Sharpe ratio comes in. Sonar’s glossary defines the deflated Sharpe ratio as an adjusted form of the Sharpe ratio that accounts for non normal returns, limited sample size, and multiple testing. Its purpose is to reduce the chance of treating an apparently strong Sharpe ratio as meaningful when it may be partly explained by luck or by searching across many alternatives. This is directly relevant to backtesting, because backtest sensitivity is not only about simulation mechanics. It is also about how many opportunities the researcher had to tune data filters, cost assumptions, parameter values, and rule structures before selecting the final specification. [3]

Taken together, these materials support a simple interpretation of backtesting. A backtest replays a strategy bar by bar under a chosen set of assumptions. Historical data choices determine what market history the strategy sees. Cost assumptions determine how much friction is imposed on each simulated trade. Event sequencing determines whether signals and fills are processed in a realistic order. Because each of these settings directly enters the simulation, each of them directly influences the computed results. Sonar’s methodology and audit tools frame this sensitivity as a core part of quantitative research rather than as a minor implementation detail. [1][2][3]

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.