Parameter sensitivity analysis asks a simple but important question: if you move a strategy parameter a little, do the results change a little, or do they collapse?

Parameter sensitivity analysis asks a simple but important question: if you move a strategy parameter a little, do the results change a little, or do they collapse? For quantitative traders, that distinction helps separate a robust region of the parameter space from a narrow setting that may look good only because it happened to fit the historical sample.

In practice, the analysis is a systematic sweep over one or more key parameters, recording a performance metric at each setting and then comparing how sharply that metric varies across the range. The Sonar Sciences strategy validation framework explicitly includes sensitivity analysis as part of parameter stability checks, alongside walk-forward validation and overfitting diagnostics. In that framework, the goal is not to find a single best-looking point, but to examine whether neighboring parameter values produce similar outcomes. A broad, relatively flat region is the pattern associated with robustness; a jagged peak is the pattern associated with fragility and possible overfitting.[1]

A useful way to structure the exercise is: 1. Choose the parameters that materially define the strategy logic. 2. Specify a reasonable range for each parameter. 3. Run the backtest across that grid or sweep. 4. Record consistent metrics for each run. 5. Inspect whether performance is stable across adjacent settings or concentrated in a small pocket.

Parameter sensitivity is assessed by varying parameters and checking whether performance remains stable across nearby values, rather than by relying on an isolated optimum.[1]

Metric choice matters. The brief mentions returns, Sharpe ratio, and deflated Sharpe ratio. Of these, the deflated Sharpe ratio is an adjustment intended to account for non-normality and multiple testing, helping assess whether an observed Sharpe ratio is statistically meaningful rather than a byproduct of selection effects.[3] In a sensitivity study, that makes it a useful companion metric: if a parameter region appears attractive on raw Sharpe but loses strength once deflated, that is evidence against treating the region as robust.

This links directly to the plateau-versus-spike interpretation. A plateau means performance remains comparatively consistent as parameters move within a neighborhood. A spike means the favorable result is confined to a narrow setting. The strategy validation source frames this as a robustness question: stable performance across nearby configurations is more credible than dependence on a precise calibration.[1] The backtest overfitting audit tool extends that reasoning by testing whether apparently strong outcomes may be artefacts of trying many variants. Its methodology is designed to estimate the probability that a selected backtest result is overfit, rather than genuinely persistent.[2] That matters because a sharp spike can arise not only from genuine model sensitivity, but also from repeated search over the parameter space until one combination happens to score well in-sample.

So parameter sensitivity analysis and overfitting audit serve different but complementary roles: - Sensitivity analysis shows how the strategy behaves as parameters move.[1] - A backtest overfitting audit evaluates whether the apparent success of the chosen configuration is likely to be an artefact of the research process.[2] - The deflated Sharpe ratio helps judge whether risk-adjusted performance remains statistically credible after accounting for multiple testing and distributional issues.[3]

Taken together, parameter sensitivity analysis quantifies how small adjustments to strategy parameters affect backtest metrics, and the shape of that response helps distinguish a stable plateau from a narrow spike. Where sensitivity appears sharp, further robustness checks and overfitting diagnostics are warranted before treating the result as reliable.[1][2][3]

Covered in depth in the Strategy validation & overfitting pillar hub.