The statistics behind the verdict are versioned like a published standard. The rules change for exactly two reasons (a strictly better published technique, or a discovered error) and every change is dated and disclosed here in plain language. One rule never changes: the method never gets easier to PASS.
Method v1.2 CURRENT
June 15, 2026
Added a seasonal naive to the benchmark line, prompted by an independent review. Net effect: the bar got harder, never easier. A model can only PASS now if it beats the toughest of three naive baselines.
Seasonal naive added. For data with a real seasonal rhythm (e.g. a weekly business cycle), the benchmark now includes “same period one cycle back”, e.g. same day last week. The verdict is scored against whichever baseline is hardest to beat (persistence, expanding climatology, or seasonal naive), so a strong weekly/seasonal pattern can no longer inflate Forecast Value Added. A model that merely re-discovers the weekly shape now correctly does not pass.
Trend-safe detection. The season is detected from the autocorrelation of the series’ first differences, so a pure upward or downward trend cannot masquerade as seasonality. When no seasonality is present, the method is identical to v1.1.
Common-window comparison. When a season is detected, every baseline and the forecast are scored on the same out-of-sample window, so the comparison stays apples-to-apples.
Why this change: an independent reviewer observed that a strongly weekly series could beat persistence and climatology while only re-stating its weekly pattern. The seasonal naive closes that gap, credit to the reviewer. Verdicts on non-seasonal data are unchanged from v1.1.
Method v1.1
June 11, 2026
An adversarial audit of the scoring engine (independent review passes plus a fifteen-dataset verification battery, with every verdict cross-checked against a separate implementation) found and fixed the following. Net effect: verdicts are the same or fairer to your model than v1.0. No change made PASS easier to obtain undeservedly.
The PASS gate was biased against you. Fixed. The 95% confidence band that decides PASS was computed against a subtly different benchmark than the headline FVA number it sits next to, which pushed the band downward. A genuinely good model could be wrongly demoted to INCONCLUSIVE; the flaw could never produce an undeserved PASS. The band now brackets the same fixed benchmark as the displayed FVA, and degenerate resamples are dropped rather than counted as zero.
Flat series can no longer earn a verdict from noise. The "nothing to beat" guard is now relative to the series' own scale, so a series that barely moves returns INCONCLUSIVE instead of a confident score built on rounding dust.
The test never guesses your columns. If the Actual or Forecast column can't be identified, scoring is blocked until you choose. Previously a column could be guessed and scored. Ambiguous number formats (such as European decimal commas), non-numeric cells, and out-of-range values are refused or skipped with a visible note, never silently mangled.
Timelines that can't be trusted aren't sorted. A date column with repeating values (a month number that restarts each year, for example) keeps your file's order and says so, instead of silently shuffling the history.
Why we publish this: a verification tool that hides its own corrections has no business verifying anything. The full QA record is available on request.
Method v1.0
June 2026
Initial published method.
Benchmark: the tougher of persistence ("assume nothing changed") and an expanding climatology mean, zero parameters, no fitting, computed strictly from values before each point (no look-ahead).
Scoring: walk-forward, out-of-sample; headline metric is Forecast Value Added (FVA) versus that benchmark, alongside the error ratio and bias.
Verdict rules: PASS requires at least twelve scored periods, FVA above zero, and the lower bound of a deterministic (data-seeded) bootstrap 95% confidence interval above zero. FAIL means FVA at or below zero. Everything else is INCONCLUSIVE, including "positive but within noise." Same input, same verdict, every time.
Probabilistic extras: CRPS and interval coverage are reported when an uncertainty column is provided, and never influence the verdict.