Why backtest results differ across platforms

Identical strategy rules can yield different backtest results across platforms because each platform embeds its own assumptions about fills, costs, data construction, and bar timing. Those differences change the simulated trade path, which changes both raw performance statistics and derived metrics such as the deflated Sharpe ratio.

Backtest results can differ across platforms even when the trading rules are identical. The main reason is that a backtest is not just a strategy definition. It is also a set of assumptions about how orders are filled, what trading costs are applied, which market data are used, and how bars are timestamped and processed. When those assumptions differ, the same rules can produce different performance numbers.

A backtest converts trading rules into simulated trades. That conversion requires a fill model. A fill model determines when an order is assumed to execute and at what price. If one platform assumes execution at the next bar open while another allows an intrabar fill using the bar high, low, or midpoint, the entry and exit prices can change. Small differences in price placement can accumulate across many trades and change return, drawdown, turnover, and risk-adjusted metrics.

Cost assumptions create another source of variation. Commissions, fees, spread, and slippage affect realized trade prices and net results. If one platform applies zero commission by default and another applies a nonzero commission schedule, the same gross trade path will produce different net performance. The effect is usually larger for strategies with higher turnover, because more trades create more opportunities for costs to compound.

Data source differences also matter. Platforms may use different vendors, different adjustment methods, different session definitions, or different levels of granularity. A strategy driven by daily bars can still change materially if the underlying daily bars were built from different intraday records, corporate action adjustments, or session cutoffs. When a signal depends on a threshold, even a small difference in a high, low, close, or volume value can change whether a trade is triggered.

Bar timing is another common cause of disagreement. A platform must decide whether a bar timestamp represents the start of the interval or the end of the interval, and whether signals generated on that bar are eligible for execution on the same bar or only on the next one. If two systems handle timestamps differently, they may shift entries and exits by one bar. A one-bar shift can materially alter both path-dependent risk and summary statistics.

Sonar Sciences describes these implementation differences in its platform comparison research. That research shows that the same strategy can produce different outputs across platforms because assumptions differ beneath the surface of the rules. The comparison emphasizes that discrepancies are not necessarily evidence that one engine is correct and another is incorrect. They often reflect different defaults in execution logic, costs, and data handling that users must inspect and normalize before comparing results across systems.

Sonar's backtest overfitting audit tool provides a useful way to think about the consequences of these differences. The tool is designed to examine whether a backtest result is likely to be overstated because of repeated testing and selection. In that context, platform-specific assumptions matter because any change in fills, costs, data, or timing can change the measured Sharpe ratio and related statistics that feed an audit. A strategy that appears stronger under one platform configuration may look weaker after cost or execution assumptions are made more conservative.

This is closely related to the deflated Sharpe ratio. Sonar's glossary defines the deflated Sharpe ratio as a version of the Sharpe ratio adjusted for multiple testing and non-normal returns. The point is not only that a raw Sharpe ratio can be optimistic after many trials. It is also that the metric itself is sensitive to the underlying return stream. If two platforms produce different fills, costs, or timestamps, they produce different return series. Once the return series changes, the Sharpe ratio changes, and the deflated Sharpe ratio can change as well.

A platform comparison is only meaningful when the hidden assumptions are made explicit, such as explicitly recording the fill model (e.g., next‑bar open versus intrabar high/low fill), the commission and slippage parameters, the data vendor and granularity, any price‑adjustment rules, session start/end definitions, and the timestamp convention used. Traders and strategy developers need to document the fill model, commission and slippage settings, data source and granularity, adjustment rules, session boundaries, and timestamp conventions. Without that normalization step, two backtests with the same strategy logic are not actually the same experiment.

Drafted with AI assistance from cited sources. Reviewed and approved by Sonar Sciences Quant & Research Team.