Research
Method notes, validation checklists, and platform comparisons for people who test a strategy before they trade it. Organised by what you are trying to settle, not by publication date.
Start with the question you have
01Strategy validation & overfittingWhether a result is evidence or an artefact of how you searched.61 pieces02Cross-venue market data & signalsWhat your sample actually contains, and what a single feed hides.55 pieces03From research to publishingVersioned streams, review, payouts, and how money moves.35 pieces04Platform comparisonsWhere each tool sits in a workflow, including what we do not do.31 pieces05Strategy research fundamentalsDefinitions and checklists. Start here if a term is unfamiliar.60 pieces
Most useful right now
ToolBacktest Overfitting AuditEight checks against your own results, with a written verdict at the end.no signup · 4 min
ArticleWhy arithmetic and geometric returns differArithmetic and geometric averages differ because returns compound multiplicatively. A sequence like a five percent gain then a negative five percent loss has a zero arithmetic average but still turns 100 into 99.75, so realized growth is below the simple average. For multi period portfolio growth, the geometric mean is the relevant measure because volatility creates a gap between average period returns and compounded wealth growth.3 min read
ArticleBar replay or automated backtestingManual bar replay is suited to observation, context-building, and hypothesis generation, but not to statistical validation. Automated backtesting, especially when paired with out-of-sample checks, a deflated Sharpe ratio, and a backtest overfitting audit, is the proper tool for evaluating whether historical results are credible.3 min read
Pillar
Everything in Research
242 of 242ArticleWhat is a deflated Sharpe ratio?Strategy validation & overfitting9 minArticleHow many trades does a backtest needStrategy validation & overfitting4 minArticleHow to audit someone else's backtestStrategy validation & overfitting6 minArticleHow to bootstrap confidence intervals for a backtestStrategy validation & overfitting3 minArticleHow to choose a benchmark for a strategyStrategy validation & overfitting4 minArticleHow to choose an out-of-sample splitStrategy validation & overfitting5 minArticleHow to detect data leakage in strategy featuresStrategy validation & overfitting7 minArticleHow to detect look-ahead bias in a backtestStrategy validation & overfitting3 minArticleHow to detect regime dependence in resultsStrategy validation & overfitting4 minArticleHow to detect survivorship bias in a datasetStrategy validation & overfitting5 minArticleHow to measure strategy capacityStrategy validation & overfitting6 minArticleHow to model transaction costs in a backtestStrategy validation & overfitting4 minArticleHow to run a permutation test on a strategyStrategy validation & overfitting5 minArticleHow to set an embargo period in validationStrategy validation & overfitting3 minArticleHow to stress test a strategy across market regimesStrategy validation & overfitting3 minArticleHow to test parameter stabilityStrategy validation & overfitting4 minArticleHow to validate exit rules separatelyStrategy validation & overfitting4 minArticleWhy a backtest needs a cost modelStrategy validation & overfitting5 minArticleWhy a smooth equity curve is suspiciousStrategy validation & overfitting4 minArticleWhy adding rules usually adds overfittingStrategy validation & overfitting3 minArticleWhy backtest start dates matterStrategy validation & overfitting4 minArticleWhy drawdown statistics need long samplesStrategy validation & overfitting3 minArticleWhy one out-of-sample pass is not enoughStrategy validation & overfitting4 minArticleWhy optimized parameters fail out of sampleStrategy validation & overfitting4 minArticleWhy optimizing for return alone overfitsStrategy validation & overfitting3 minArticleWhy random entries can look profitable in backtestsStrategy validation & overfitting4 minArticleWhy Sharpe ratios shrink out of sampleStrategy validation & overfitting4 minArticleWhy stationarity matters for backtestsStrategy validation & overfitting3 minArticleWhy trade frequency changes what results meanStrategy validation & overfitting7 minArticleWhy you must count every backtest you runStrategy validation & overfitting4 minToolBacktest Overfitting AuditStrategy validation & overfitting8 checksReferenceAlpha decayStrategy validation & overfittingdefinitionReferenceAutocorrelationStrategy validation & overfittingdefinitionReferenceBlock bootstrapStrategy validation & overfittingdefinitionReferenceBonferroni correctionStrategy validation & overfittingdefinitionReferenceBootstrap resamplingStrategy validation & overfittingdefinitionReferenceConfidence intervalStrategy validation & overfittingdefinitionReferenceCurve fittingStrategy validation & overfittingdefinitionReferenceData snooping biasStrategy validation & overfittingdefinitionReferenceEmbargo periodStrategy validation & overfittingdefinitionReferenceFalse discovery rateStrategy validation & overfittingdefinitionReferenceHeteroskedasticityStrategy validation & overfittingdefinitionReferenceHoldout setStrategy validation & overfittingdefinitionReferenceIn-sample periodStrategy validation & overfittingdefinitionReferenceLook-ahead biasStrategy validation & overfittingdefinitionReferenceMarket regimeStrategy validation & overfittingdefinitionReferenceMonte Carlo permutation testStrategy validation & overfittingdefinitionReferenceMultiple testing problemStrategy validation & overfittingdefinitionReferenceNull hypothesisStrategy validation & overfittingdefinitionReferenceP-hackingStrategy validation & overfittingdefinitionReferenceP-valueStrategy validation & overfittingdefinitionReferenceParameter sensitivityStrategy validation & overfittingdefinitionReferenceParameter stabilityStrategy validation & overfittingdefinitionReferenceProbability of backtest overfittingStrategy validation & overfittingdefinitionReferencePurged cross-validationStrategy validation & overfittingdefinitionReferenceRegime filterStrategy validation & overfittingdefinitionReferenceSelection biasStrategy validation & overfittingdefinitionReferenceStationarityStrategy validation & overfittingdefinitionReferenceSurvivorship biasStrategy validation & overfittingdefinitionReferenceT-statisticStrategy validation & overfittingdefinitionReferenceWhite's reality checkStrategy validation & overfittingdefinitionArticleHow to align timestamps across data sourcesCross-venue market data & signals5 minArticleHow to build a point-in-time datasetCross-venue market data & signals6 minArticleHow to choose candle timeframes for researchCross-venue market data & signals4 minArticleHow to clean tick dataCross-venue market data & signals3 minArticleHow to detect a regime change in dataCross-venue market data & signals3 minArticleHow to detect bad prints and outliersCross-venue market data & signals5 minArticleHow to detect interpolated candlesCross-venue market data & signals4 minArticleHow to detect stale quotes in a feedCross-venue market data & signals3 minArticleHow to estimate slippage from dataCross-venue market data & signals4 minArticleHow to find gaps in historical dataCross-venue market data & signals5 minArticleHow to handle exchange outages in a datasetCross-venue market data & signals3 minArticleHow to reconcile data from multiple venuesCross-venue market data & signals3 minArticleHow to store market data for researchCross-venue market data & signals4 minArticleHow to verify a data vendor's historyCross-venue market data & signals5 minArticleWhy a single data feed biases a backtestCross-venue market data & signals4 minArticleWhy broker feeds differ from exchange feedsCross-venue market data & signals5 minArticleWhy the same candle differs across venuesCross-venue market data & signals4 minArticleWhy consolidated prices hide venue behaviourCross-venue market data & signals3 minArticleWhy execution research needs tick dataCross-venue market data & signals4 minArticleWhy funding rates matter in perpetual backtestsCross-venue market data & signals3 minArticleWhy liquidity varies by session and venueCross-venue market data & signals3 minArticleWhy perpetual and spot prices divergeCross-venue market data & signals6 minArticleWhy market regimes belong in your sampleCross-venue market data & signals4 minArticleWhy reported volume can misleadCross-venue market data & signals3 minArticleWhy timezone errors corrupt daily candlesCross-venue market data & signals4 minReferenceBackwardationCross-venue market data & signalsdefinitionReferenceBid-ask spreadCross-venue market data & signalsdefinitionReferenceCandle interpolationCross-venue market data & signalsdefinitionReferenceContangoCross-venue market data & signalsdefinitionReferenceFunding rateCross-venue market data & signalsdefinitionReferenceFutures basisCross-venue market data & signalsdefinitionReferenceImplementation shortfallCross-venue market data & signalsdefinitionReferenceIndex priceCross-venue market data & signalsdefinitionReferenceLatencyCross-venue market data & signalsdefinitionReferenceLimit orderCross-venue market data & signalsdefinitionReferenceLiquidityCross-venue market data & signalsdefinitionReferenceMark priceCross-venue market data & signalsdefinitionReferenceMarket fragmentationCross-venue market data & signalsdefinitionReferenceMarket impactCross-venue market data & signalsdefinitionReferenceMarket orderCross-venue market data & signalsdefinitionReferenceOHLCVCross-venue market data & signalsdefinitionReferenceOpen interestCross-venue market data & signalsdefinitionReferenceOrder bookCross-venue market data & signalsdefinitionReferencePerpetual futuresCross-venue market data & signalsdefinitionReferencePoint-in-time dataCross-venue market data & signalsdefinitionReferenceSlippageCross-venue market data & signalsdefinitionReferenceStale quoteCross-venue market data & signalsdefinitionReferenceStop orderCross-venue market data & signalsdefinitionReferenceTick dataCross-venue market data & signalsdefinitionReferenceTick sizeCross-venue market data & signalsdefinitionReferenceTimestamp alignmentCross-venue market data & signalsdefinitionReferenceTransaction costsCross-venue market data & signalsdefinitionReferenceTWAPCross-venue market data & signalsdefinitionReferenceVWAPCross-venue market data & signalsdefinitionReferenceWash tradingCross-venue market data & signalsdefinitionArticleHow to archive a retired strategyFrom research to publishing4 minArticleHow to document a trading strategyFrom research to publishing4 minArticleHow to keep a strategy research logFrom research to publishing5 minArticleHow to make a backtest reproducibleFrom research to publishing4 minArticleHow to prove when a result was producedFrom research to publishing4 minArticleHow to report a backtest honestlyFrom research to publishing4 minArticleHow to run an independent strategy reviewFrom research to publishing5 minArticleHow to separate research code from execution codeFrom research to publishing4 minArticleHow to version a datasetFrom research to publishing4 minArticleWhat belongs in a performance disclosureFrom research to publishing4 minArticleWhat counts as investment adviceFrom research to publishing3 minArticleWhy a track record needs an audit trailFrom research to publishing4 minArticleWhy every public claim needs a sourceFrom research to publishing4 minArticleWhy financial promotion rules matter to quantsFrom research to publishing3 minArticleWhy identity checks exist in financeFrom research to publishing3 minArticleWhy pre-registering a test keeps you honestFrom research to publishing3 minArticleWhy public track records skew positiveFrom research to publishing4 minArticleWhy publication needs a review gateFrom research to publishing3 minArticleWhy reproducibility matters in trading researchFrom research to publishing4 minArticleWhy screenshots are not a track recordFrom research to publishing3 minArticleWhy strategy changes need version historyFrom research to publishing5 minReferenceAudit trailFrom research to publishingdefinitionReferenceClaim registerFrom research to publishingdefinitionReferenceDouble-entry ledgerFrom research to publishingdefinitionReferenceFinancial promotionFrom research to publishingdefinitionReferenceImmutable recordFrom research to publishingdefinitionReferenceIncubation periodFrom research to publishingdefinitionReferenceInvestment adviceFrom research to publishingdefinitionReferenceKnow your customerFrom research to publishingdefinitionReferencePeer reviewFrom research to publishingdefinitionReferencePre-registrationFrom research to publishingdefinitionReferenceReproducibilityFrom research to publishingdefinitionReferenceStrategy versioningFrom research to publishingdefinitionReferenceTrack recordFrom research to publishingdefinitionReferenceTrading signalFrom research to publishingdefinitionArticleBar replay or automated backtestingPlatform comparisons3 minArticleCan you backtest in a spreadsheetPlatform comparisons5 minArticleCloud or local compute for backtestingPlatform comparisons4 minArticleDatabases or flat files for market dataPlatform comparisons4 minArticleWhat paid market data buys over free feedsPlatform comparisons6 minArticleHow to benchmark a backtesting engine for correctnessPlatform comparisons3 minArticleHow to choose a backtesting platformPlatform comparisons6 minArticleHow to compare backtesting engines fairlyPlatform comparisons3 minArticleHow to compare execution quality across brokersPlatform comparisons3 minArticleHow to compare charting tools for researchPlatform comparisons4 minArticleHow to compare crypto data providersPlatform comparisons5 minArticleHow to evaluate a market data vendorPlatform comparisons4 minArticleHow to read a strategy tester report criticallyPlatform comparisons5 minArticleHow to trial a research platform in a weekPlatform comparisons6 minArticleNotebooks or scripts for strategy researchPlatform comparisons3 minArticleOpen source or commercial backtesting toolsPlatform comparisons5 minArticlePaper trading or backtestingPlatform comparisons5 minArticlePython or Pine Script for strategy researchPlatform comparisons4 minArticleREST or websocket market dataPlatform comparisons5 minArticleRetail and institutional data feeds comparedPlatform comparisons4 minArticleVectorized or event-driven backtestingPlatform comparisons3 minArticleWhat the MT5 strategy tester does and does not testPlatform comparisons3 minArticleWhy feature checklists mislead platform choicesPlatform comparisons4 minArticleWhy backtest results differ across platformsPlatform comparisons4 minArticleWhy the same strategy fills differently at two brokersPlatform comparisons3 minReferenceBacktesting enginePlatform comparisonsdefinitionReferenceEvent-driven backtestingPlatform comparisonsdefinitionReferenceFeature matrixPlatform comparisonsdefinitionReferencePine ScriptPlatform comparisonsdefinitionReferenceVectorized backtestingPlatform comparisonsdefinitionReferenceVendor lock-inPlatform comparisonsdefinitionArticleWhy arithmetic and geometric returns differStrategy research fundamentals3 minArticleHow trading costs compound over timeStrategy research fundamentals3 minArticleHow does backtesting workStrategy research fundamentals5 minArticleHow is drawdown calculatedStrategy research fundamentals4 minArticleHow leverage changes risk of ruinStrategy research fundamentals5 minArticleHow long should you paper trade a strategyStrategy research fundamentals5 minArticleHow to annualize returns and volatilityStrategy research fundamentals3 minArticleHow to calculate position sizeStrategy research fundamentals4 minArticleHow to compare two equity curvesStrategy research fundamentals6 minArticleHow to read a backtest reportStrategy research fundamentals6 minArticleHow to read an equity curveStrategy research fundamentals6 minArticleCan you separate skill from luck in trading resultsStrategy research fundamentals6 minArticleWhat counts as a trading strategyStrategy research fundamentals3 minArticleWhat is a good Sharpe ratioStrategy research fundamentals4 minArticleWhat is risk per tradeStrategy research fundamentals3 minArticleWhy losses hurt more than gains helpStrategy research fundamentals3 minReferenceAlphaStrategy research fundamentalsdefinitionReferenceBacktestStrategy research fundamentalsdefinitionReferenceBenchmarkStrategy research fundamentalsdefinitionReferenceBetaStrategy research fundamentalsdefinitionReferenceCalmar ratioStrategy research fundamentalsdefinitionReferenceCompound annual growth rateStrategy research fundamentalsdefinitionReferenceCorrelationStrategy research fundamentalsdefinitionReferenceDeflated Sharpe ratioStrategy research fundamentalsdefinitionReferenceDrawdown durationStrategy research fundamentalsdefinitionReferenceEquity curveStrategy research fundamentalsdefinitionReferenceExpected shortfallStrategy research fundamentalsdefinitionReferenceExposureStrategy research fundamentalsdefinitionReferenceFat tailsStrategy research fundamentalsdefinitionReferenceForward testingStrategy research fundamentalsdefinitionReferenceKelly criterionStrategy research fundamentalsdefinitionReferenceKurtosisStrategy research fundamentalsdefinitionReferenceLeverageStrategy research fundamentalsdefinitionReferenceLog returnsStrategy research fundamentalsdefinitionReferenceLookback periodStrategy research fundamentalsdefinitionReferenceMarginStrategy research fundamentalsdefinitionReferenceMaximum drawdownStrategy research fundamentalsdefinitionReferenceNotional valueStrategy research fundamentalsdefinitionReferenceOut-of-sample testingStrategy research fundamentalsdefinitionReferenceOverfittingStrategy research fundamentalsdefinitionReferencePaper tradingStrategy research fundamentalsdefinitionReferencePayoff ratioStrategy research fundamentalsdefinitionReferencePosition sizingStrategy research fundamentalsdefinitionReferenceProfit factorStrategy research fundamentalsdefinitionReferenceRandom walkStrategy research fundamentalsdefinitionReferenceRebalancingStrategy research fundamentalsdefinitionReferenceRisk of ruinStrategy research fundamentalsdefinitionReferenceRisk-reward ratioStrategy research fundamentalsdefinitionReferenceRolling windowStrategy research fundamentalsdefinitionReferenceSharpe ratioStrategy research fundamentalsdefinitionReferenceSkewnessStrategy research fundamentalsdefinitionReferenceSortino ratioStrategy research fundamentalsdefinitionReferenceTail riskStrategy research fundamentalsdefinitionReferenceTrade expectancyStrategy research fundamentalsdefinitionReferenceUnderwater curveStrategy research fundamentalsdefinitionReferenceValue at riskStrategy research fundamentalsdefinitionReferenceVolatility targetingStrategy research fundamentalsdefinitionReferenceVolatilityStrategy research fundamentalsdefinitionReferenceWalk-forward validationStrategy research fundamentalsdefinitionReferenceWin rateStrategy research fundamentalsdefinition
Nothing is written for that combination yet.
Get the next utility when it ships
The audit is live. When the next tool is finished you get the link and a note on what it checks. Nothing else, and no digest.