Curve fitting occurs when trading rules are repeatedly adjusted until they align with the quirks of one historical sample rather than with patterns that persist beyond it.

Curve fitting occurs when trading rules are repeatedly adjusted until they align with the quirks of one historical sample rather than with patterns that persist beyond it. In strategy development, that means a model can look strong in-sample while failing to generalize to unseen periods. Sonar’s strategy-validation research frames this as a core validation problem: a backtest is not just a measurement of a rule set, but also a record of all the choices made while building it. As the number of tweaks, filters, and parameter searches increases, the chance rises that the final specification reflects historical accidents rather than durable structure.[1]

Sonar’s backtest overfitting audit tool is presented specifically to detect this failure mode. The tool evaluates whether repeated specification changes and selection among alternatives are likely to have produced an overfit result. According to the tool description, strategies that are heavily tuned can show attractive backtest statistics in the development sample and then degrade materially when examined on out-of-sample or otherwise untouched data.[2] That pattern is the practical signature of curve fitting: optimization improves the fit to the sample that was used for design, but not the strategy’s ability to predict behavior in new market periods.

This is why incremental rule tweaking is dangerous. A developer may add one more threshold, alter one lookback, or narrow one entry condition and see in-sample metrics improve after each step. But Sonar’s validation framework emphasizes that each such decision is part of the testing burden, because the strategy was effectively selected from many candidates, even if only one final version is shown in the report.[1] In that setting, apparent improvement can reflect search luck rather than information. The backtest overfitting audit is designed to make that hidden search process visible and to test whether the selected strategy still holds up once judged outside the data that shaped it.[2]

The same issue appears in risk-adjusted performance metrics. Sonar’s glossary entry on the deflated Sharpe ratio explains that the ordinary Sharpe ratio can be overstated when many trials, variations, or specifications have been explored before choosing the best-looking result.[3] A standard Sharpe ratio summarizes observed excess return per unit of volatility, but it does not by itself account for selection bias from multiple testing. The deflated Sharpe ratio adjusts for that problem by considering the number of trials and the non-normal features that can distort naive performance interpretation.[3] In suspected overfitting cases, this creates an important contrast: an in-sample Sharpe ratio may appear compelling, while the deflated Sharpe ratio can indicate that the result is not statistically persuasive once the full search process is acknowledged.[3]

That comparison matters because curve fitting often hides behind polished summary metrics. A strategy that has been tuned through many parameter combinations can produce a high conventional Sharpe ratio simply because the search found a favorable historical path. Sonar’s materials argue that the correct question is not whether one selected backtest looks good, but whether it still looks unusual after adjusting for how many opportunities there were to find such a result.[1][3] If the deflated Sharpe ratio weakens substantially relative to the standard Sharpe ratio, that is consistent with a strategy whose apparent edge depends on data-mined luck rather than robust signal.[3]

For practitioners, the lesson in Sonar’s validation material is straightforward: curve fitting is not merely “using too many parameters.” It is the broader act of letting historical noise influence design decisions until the final rule set reflects the sample’s accidents. Robust validation therefore requires separating development from confirmation, accounting for the full strategy search process, and interpreting performance metrics with tools such as the deflated Sharpe ratio rather than taking headline in-sample statistics at face value.[1][2][3]

Covered in depth in the Strategy validation & overfitting pillar hub.