How the model has changed
every dated change to the method, newest first, with its evidence
The methodology pages describe what the forecast does now. This page is their history: every dated change, test result, correction and review note, newest first, with the wording it had where it first appeared and the evidence it cites. Evidence files are in the project's repository; the ones a reader can open are in the review folder.
October 28, 2026
- Alaska's ranked-choice races: the shape of the error (adjustment register, test record). Promotion case registered before the measurement (experiments/rcv_student_t_promotion_preregistration.md) on the rule of experiments/audit7_preregistration.md item 4 (passed 2026-10-27): the ranked-choice races take the shape every other statewide race is drawn with. Same-data dry run: the four Alaska rows move 0.24 to 0.43 points (largest the governor's election-day chance, 88.94% -> 89.38%), centres unchanged. ranked_choice_final_round 1.2.0, published 2026-10-28; three contests cannot test the shape on their own.
Evidence:
experiments/audit7_preregistration.md (item 4),experiments/audit7_result.md,experiments/rcv_student_t_promotion_preregistration.md,experiments/rcv_student_t_promotion_result.md,experiments/validation_2026/ranked_choice_2026_preregistration.md (scored on 2026, descriptively) - House and statewide estimate: one religion census for every election (adjustment register, test record). Registered as a correction judged by non-inferiority on 2024, the only cycle the defect touches (experiments/religion_vintage_preregistration.md): House Brier and margin MAE, statewide blend Brier and margin MAE, each not worse than its limit. Rerun on the rebuilt House panel: House Brier 0.01982 -> 0.01946, MAE 3.565 -> 3.589 (limit +0.05), blend Brier 0.02554 -> 0.02562 (limit +0.0005), blend MAE 3.124 -> 3.106; all four pass, and the rule lets the consistent input win the tie. house_ground_up 2.12.0, published 2026-10-28.
Evidence:
experiments/religion_vintage_preregistration.md,experiments/religion_vintage_result.md,experiments/religion_vintage_rebuilt_panel.md (rerun on the rebuilt House panel),docs/FAILURE_MODES.md FM-257 - House: a shared adjustment from district polls (not used) (adjustment register, test record). Registered (experiments/district_poll_shared_preregistration.md) and passed its four conditions on 2020-2024: pooled Brier 0.02699 -> 0.02566, better in each cycle; 2024 margin MAE worse. Held off before the election (on copies the House majority chance moved 87.8% -> 82.3%) and registered as a prospective test (experiments/district_poll_shared_2026_prospective_preregistration.md): PUB against SH, both rebuilt from one run's sealed bundle, frozen 2026-10-28 and at the final cutoff (job h86-shadow); used from 2028 only if the pooled 2020-2026 Brier is lower, the pooled log loss is not worse by more than 0.002, the 2026 Brier is not worse by more than 0.002 and the four-cycle seat error is not worse by more than one seat.
Evidence:
experiments/district_poll_shared_preregistration.md,experiments/district_poll_shared_result.md,experiments/district_poll_shared_2026_prospective_preregistration.md,experiments/district_poll_shared_2026_frozen_20261028.json - House: district polls discounted by age (adjustment register, challenge). 2026-10-28 (experiments/house_poll_age_measured_preregistration.md, registered before any 2026 result): from 2,296 whole-sample district polls of 2004-2024 the miss grows with age, E[miss^2] = 52.17 + 0.1149 x days (95% for the slope 0.023 to 0.239), a weight relative to an eve poll of about 0.79 at 120 days and 0.68 at 210 against the schedule's 0.80 and 0.50; arms DECAY (published), LIVE (equal weights) and DRIFT (measured), rebuilt from the sealed bundles of the last House run on or before 2026-10-27 and the last before 2026-11-03; scored on 2026 after certification.
Evidence:
experiments/house_poll_rules_v2_preregistration.md,experiments/house_poll_rules_v2_result.json,experiments/house_poll_age_measured_preregistration.md (the measured loss of accuracy with age, arm DRIFT, against the published schedule; scored on 2026) - House: rater blend (adjustment register, challenge). 2026-10-28 (experiments/house_rater_blend_v2_preregistration.md, registered before any run): the live blend B against no blend C0 and a margin-form blend M (each rating read as a margin in its own year, minus that year's national estimate); 2012-2014 ratings added for a walk-forward 2016; decided on the district scores with seat-distribution scores and the mechanism check as guards; the 2026 arms written once at the final cutoff by job rater-v2-shadow and scored after certification. No setting changes before 2026-11-03.
Evidence:
experiments/house_rater_blend_preregistration.md,experiments/house_rater_blend_result.json,experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json),experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification) - House: rater blend (adjustment register, parameters). weight w 0.149 in the live fit of 2026-10-28 (data/forecasts/ground_up_2026.json, rater_blend.w, written by scripts/forecast_house_ground_up.py on each House run; shown as 15%); category offsets from 2018-2024
Evidence:
experiments/house_rater_blend_preregistration.md,experiments/house_rater_blend_result.json,experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json),experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification) - House: spread of the seats forecast from their own polls (adjustment register, test record). Registered shape test (experiments/own_polls_shape_preregistration.md, disclosed as a second look at the rows of the first measured-spread test): on the polled district-cycles of 2018-2024, 90% coverage in [0.86, 0.94], 80% coverage in [0.75, 0.85], log loss not worse than the published spread by more than 0.002, Brier not worse by more than 0.0005, every cycle's 90% coverage at least 0.80. Rerun on the rebuilt House panel, 406 rows: 0.901, 0.768, log loss 0.4491 -> 0.4175, Brier 0.1468 -> 0.1362, cycles 0.867-0.947; all five pass (the published full error covered 0.983 at 90%). In house_ground_up 2.11.0, published as 2.12.0 on 2026-10-28.
Evidence:
experiments/own_polls_band_result.md (the measured spread with a normal shape: missed its 80% limit by 0.004),experiments/own_polls_shape_preregistration.md,experiments/own_polls_shape_result.md,experiments/own_polls_shape_rebuilt_panel.md (rerun on the rebuilt House panel),docs/FAILURE_MODES.md FM-247,experiments/validation_2026/house_components_2026_preregistration.md (scored on 2026) - Measured error of a sparse race's poll average (adjustment register, challenge). 2026-10-28 (experiments/sparse_error_accounting_preregistration.md, registered with section 3a): the published shared-once arm B against the exact posterior with its own residual (Ck) and, because the prior's and the polls' errors overlap (rho 0.41, 95% 0.23 to 0.64, from the thirty-three sparse races of 2020-2024; k_rho^2 25.40), the arm that allows for the overlap (Crhok), to which the decision rule applies; scored on 2026 after certification.
Evidence:
experiments/sparse_blend_poll_weight.md,docs/FAILURE_MODES.md FM-104,experiments/sparse_variance.md (the first registration, rejected on its metric),experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment),experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026) - Sparse-race blend (adjustment register, challenge). 2026-10-28 (experiments/sparse_prior_once_result.md, FM-253): a race below four usable polls takes the seat prior inside the sparse blend and the ground-up prior again in apply_statewide_prior.py. Counting it once (ELAB_SPARSE_PRIOR_ONCE) passed its registered rule on the 256-race record (log loss 0.16415 -> 0.16400, Brier 0.05162 -> 0.05163), but the sparse races' 90% coverage fell 0.843 -> 0.784; the flag stays off, to be fixed together with the sparse races' range width (experiments/sparse_error_accounting_preregistration.md).
Evidence:
experiments/preregistration_sparse_blend.md,experiments/prior_only_fallback_ignores_sparse_polls.md,experiments/sparse_prior_once_preregistration.md,experiments/sparse_prior_once_result.md (the prior counted once: passed its rule, held: the sparse races' 90% coverage fell; FM-253) - Training money dated by when each filing was received (adjustment register, test record). Registered before any run (experiments/h128_money_and_history_preregistration.md), measured on copies of the live database: House gate selection Brier 0.03681 -> 0.03618, seat error 12.20 -> 11.87, wrong winners 20.0 -> 18.25, backtest M2 MAE 6.160 -> 5.964, selection rule and 2024 confirmation passed; statewide gate passed; blend 0.8.7 record (256 races) log loss 0.1634 -> 0.1630, Brier 0.0514 -> 0.0513, 80%/90% coverage 0.832/0.930 -> 0.828/0.930. Every condition passed; applied live on 2026-10-28 and the House gate rerun on the live rows (PASSED).
Evidence:
experiments/h128_money_and_history_preregistration.md,experiments/h128_money_and_history_result.md
October 27, 2026
- Fundamentals prior on the race's starting level (adjustment register, description). The latent path's starting level gets an informative prior from the state's previous results (same seat where possible) with a width from how much such margins move, instead of a vague Normal(0, 15) centred on a tie. Since 27 October 2026 a gap length's width is measured only from at least eight different states, not merely eight pairs of results: the governor gaps of two, six, ten and fourteen years had come from two to five states (Vermont, New Hampshire and the odd-year states), and now take the nearest gap measured from eight or more. It passed its registered test with no measurable change in accuracy; single_race_latent 0.7.2 (the poll model) and 0.8.6 (the published blend); ELAB_VOL_MIN_STATES=0 restores 0.7.1.
Evidence:
docs/DECISIONS.md ADR-0007,docs/FAILURE_MODES.md FM-84,experiments/volatility_min_states_preregistration.md,experiments/volatility_min_states_result.md,docs/FAILURE_MODES.md FM-237 - Fundamentals prior on the race's starting level (adjustment register, test record). Shipped as a fix (ADR-0017), not gated; 284 races: CRPS 3.928 -> 3.871, MAE 5.46 -> 5.40, better in 5 of 5 cycles. The volatility minimum of states (M-19) registered as a consistency correction, A/B over every champion election-eve backtest of 2016-2024 (256 races): published blend log loss 0.1641 and 0.1642, Brier 0.0516 both, coverage 0.469/0.820/0.934 both, MAE 4.57 both; adopted 2026-10-27.
Evidence:
docs/DECISIONS.md ADR-0007,docs/FAILURE_MODES.md FM-84,experiments/volatility_min_states_preregistration.md,experiments/volatility_min_states_result.md,docs/FAILURE_MODES.md FM-237 - House: shared state error (adjustment register, description). Districts in the same state share an error, measured from walk-forward residuals about each cycle's mean; the cycle's own lean bias is drawn with the national shock every district shares (house_ground_up 2.10.0 on, the 'shared' arm of ELAB_DISTRICT_STATE_VAR).
Evidence:
experiments/district_gate_preregistration.md,experiments/district_state_var_preregistration.md,experiments/district_state_var_result.md,docs/FAILURE_MODES.md FM-236 - House: shared state error (adjustment register, test record). Registered three-arm test on the district gate (experiments/district_state_var_preregistration.md): arm 'shared' adopted by its consistency rule (competitive Brier 0.1415 = 0.1415, coverage 4 = 4; experiments/district_state_var_result.md).
Evidence:
experiments/district_gate_preregistration.md,experiments/district_state_var_preregistration.md,experiments/district_state_var_result.md,docs/FAILURE_MODES.md FM-236 - Methodology pages (/methodology). The methodology pages were rewritten as a present-tense description. Every dated note, version change, audit and review note they carried moved to this change log, with its wording and evidence; the repository's failure log, decisions, adversarial review, adjustment register, data-source register and requirements moved from the methodology section to this record area.
Evidence: /methodology, /record
October 26, 2026
- Drift to Election Day (adjustment register, description). A forecast made before election day is widened by how far races have historically moved between that horizon and the vote: variance a*h^b, h in days (1.650*h^0.669 for 2026, about 3.9 points of sd at four weeks, in margin points; 1.477*h^0.691 before the 26 October 2026 refit).
Evidence:
experiments/uncertainty_terms.md,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Drift to Election Day (adjustment register, parameters). experiments/uncertainty_terms_pre<cycle>.json drift a, b (the fixed-selection refit since 2026-10-26)
Evidence:
experiments/uncertainty_terms.md,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Drift to Election Day (adjustment register, test record). Estimated from historical trajectories. Refitted with the fixed result selection on 2026-10-26 (FM-232) as part of the adopted terms refit; its test scored election eve only, where the drift term is near zero, so the refitted law is untested at the horizons it acts on.
Evidence:
experiments/uncertainty_terms.md,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Generic ballot: each pollster's lean removed before averaging (adjustment register, test record). Registered walk-forward test 2004-2024 (arms A plain, B centred house effects, C anchored), decision rule: B replaces A if lower MSE and better in at least 7 of 11 cycles at 31+1 days; B passed (8 of 11; bootstrap over cycles -0.98, 95% -1.78 to -0.44).
Evidence:
experiments/generic_house_effects_preregistration.md,experiments/generic_house_effects_result.json - Generic ballot: pollster house effects (docs/STATISTICAL_MODEL.md §11.1.3). 11.1.3 Pollster house effects removed before averaging (adopted 2026-10-26). [...] Registered test experiments/generic_house_effects_preregistration.md: walk-forward 2004-2024 MSE 7.84 against 8.82 for the plain average, better in 8 of 11 cycles at 31+1 days (the rule needed 7); bootstrap difference -0.98 (95% -1.78 to -0.44). Not yet refitted on the adjusted path: the drift law and the firm-jackknife sd (TODO H105); an arm anchored to media and academic firms did better still and is to be registered as its own test (TODO H100).
Evidence:
experiments/generic_house_effects_preregistration.md,experiments/generic_house_effects_result.json - Generic ballot: pollster house effects (docs/STATISTICAL_MODEL.md §11.1.3). Not yet refitted on the adjusted path: the drift law and the firm-jackknife sd (TODO H105). [Removed from the methodology page: the review gate found the drift-law clause unsupported by the evidence given; TODO H105 records the drift law as fitted on the plain generic path and moot for 2026, and the firm jackknife as already seeing adjusted polls.]
Evidence:
TODO H105 - House district poll age discount (docs/STATISTICAL_MODEL.md §11.1). The clean split was registered and tested and was not adopted (no improvement; the shared component is small): experiments/house_poll_variance_result.md (outside statistical review 2026-10-25, F7). Since House 2.9.0 (2026-10-26) each district poll is also discounted by its age on a fixed schedule (elab.models.district.polls.age_factor in shift_v2, for every district and the seats forecast from their own polls; adjustment house_poll_age_decay): the owner's amendment to the registered test (experiments/house_poll_rules_v2_preregistration.md, amendment 2159486e), adopted because the walk-forward rerun on 2016-2024 showed no harm (-0.58, 95% -2.70 to +1.10, better in 4 of 5 years).
Evidence:
experiments/house_poll_variance_result.md (outside statistical review 2026-10-25, F7),experiments/house_poll_rules_v2_preregistration.md (amendment 2159486e),experiments/house_poll_rules_v2_result.json - House record (/accuracy, The House model, election by election). The House model (2.6.0) fitted only on earlier elections and scored on each of 2016–2024. This record is version 2.6.0's. The published version adds rules it does not include: the scandal rule (2.7.0), CA-6 forecast from its polls (2.8.0) and district polls discounted by their age (2.9.0). Each was tested on its own, and the whole published model has not been scored on these past elections.
Evidence:
experiments/house_spread_result.json,experiments/house_scandal/result.md,experiments/house_poll_rules_v2_result.json - House: district polls discounted by age (adjustment register, description). Each district poll counts for less the further it was taken from election day: full weight within 60 days, falling steadily to half at 210 days and a quarter beyond, and never more for an older poll. This is the owner's judgement that equal weighting of old and new polls is contrary to what is known about polling, adopted by an amendment written before its backtest; the walk-forward backtest on 2016-2024 found it not harmful (squared error -0.58 against the previous rule, better at 120, 90 and 60 days, neutral in the final month).
Evidence:
experiments/house_poll_rules_v2_preregistration.md,experiments/house_poll_rules_v2_result.json,experiments/house_poll_age_measured_preregistration.md (the measured loss of accuracy with age, arm DRIFT, against the published schedule; scored on 2026) - House: district polls discounted by age (adjustment register, name). House: district polls discounted by their age (kept under an amended rule)
Evidence:
experiments/house_poll_rules_v2_preregistration.md,experiments/house_poll_rules_v2_result.json,experiments/house_poll_age_measured_preregistration.md (the measured loss of accuracy with age, arm DRIFT, against the published schedule; scored on 2026) - House: district polls discounted by age (adjustment register, test record). Owner's amendment to experiments/house_poll_rules_v2_preregistration.md (written before the rerun): adopted unless clearly harmful (difference above 0 with p < 0.05); result -0.58, 95% -2.70 to +1.10, adopted.
Evidence:
experiments/house_poll_rules_v2_preregistration.md,experiments/house_poll_rules_v2_result.json,experiments/house_poll_age_measured_preregistration.md (the measured loss of accuracy with age, arm DRIFT, against the published schedule; scored on 2026) - Race-specific error (adjustment register, description). Every race's spread includes a race-specific error beyond its polls (4.93 points for 2026; 4.97 before the 26 October 2026 refit), the residual of the same variance-components fit.
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-79,docs/FAILURE_MODES.md FM-89,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Race-specific error (adjustment register, parameters). experiments/uncertainty_terms_pre<cycle>.json (since 2026-10-26 the fixed-selection refit for 2016-2026; single_race_latent 0.7.1)
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-79,docs/FAILURE_MODES.md FM-89,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Race-specific error (adjustment register, test record). Restored as a fix (FM-79): 247 races, 90% coverage 0.680 -> 0.895 walk-forward. Refitted 2026-10-26 with the fixed result selection (the round the polls measured, live rows, every party spelling; FM-232) and adopted by its registered consistency test (experiments/uncertainty_terms_refit_result.md): over every election-eve backtest of 2016-2024 the published forecast's log loss 0.1662 -> 0.1641, Brier 0.0517 -> 0.0516, 80%/90% coverage 0.836/0.945 -> 0.824/0.938, MAE 4.58 -> 4.57. The 2026-09-29 files are kept as experiments/uncertainty_terms_pre<cycle>_2026-09-29.json.
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-79,docs/FAILURE_MODES.md FM-89,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Shared national polling error (adjustment register, description). Every race's election-day spread includes an error shared by all races in a cycle, the size of past cycles' average polling misses (2.63 points for 2026; 2.44 before the 26 October 2026 refit, which fitted it against the November round the polls measured instead of runoff or superseded result rows).
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-82,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Shared national polling error (adjustment register, parameters). experiments/uncertainty_terms_pre<cycle>.json (since 2026-10-26 the fixed-selection refit for 2016-2026; single_race_latent 0.7.1)
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-82,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Shared national polling error (adjustment register, test record). Estimated (variance components over earlier cycles). Refitted 2026-10-26 with the fixed result selection (the round the polls measured, live rows, every party spelling; FM-232) and adopted by its registered consistency test (experiments/uncertainty_terms_refit_result.md): over every election-eve backtest of 2016-2024 the published forecast's log loss 0.1662 -> 0.1641, Brier 0.0517 -> 0.0516, 80%/90% coverage 0.836/0.945 -> 0.824/0.938, MAE 4.58 -> 4.57. The 2026-09-29 files are kept as experiments/uncertainty_terms_pre<cycle>_2026-09-29.json.
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-82,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Statewide prior: national tide taken out (docs/STATISTICAL_MODEL.md §7.1). This was written on 2026-09-28 and, through an omission in the callers, reached production only on
2026-10-26 (FM-231). Its 2026-09-28 measurement (14.26 points of mean absolute error to 14.03) read
the target cycle's realised House margin and is not evidence for it. The registered test is
experiments/detide_prior_result.md: over every election-eve backtest of 2016-2024 (256 races) the
published forecast's log loss was 0.1662 with and without it, Brier 0.0517 against 0.0518, 80%/90%
coverage 0.840/0.945 against 0.836/0.945, mean absolute error 4.58 both: no measurable change in
accuracy, adopted by its rule as a consistency correction. The poll model alone was worse with it
(log loss 0.1781 against 0.1744); the blend with the ground-up prior absorbs that. On the 2026
forecasts it moved the published margins by 0.14 points on average over 71 races, the largest
Oklahoma governor (Democratic chance 15.8% to 17.0%), with no rating or chamber control change.
Adopted together with the refitted uncertainty terms (§11, FM-232) as single_race_latent 0.7.1
(poll model) and 0.8.5 (the published blend); `ELAB_PRIOR_DETIDE=0` with the 2026-09-29 terms
files is 0.7.0/0.8.4, and one change without the other is refused. For a redistricted House seat, the
district's own presidential margin under the new lines replaces the old margin entirely
(`BaselineRelation`, MAE 3.33 against 10.39).
Evidence:
docs/FAILURE_MODES.md FM-231,docs/FAILURE_MODES.md FM-232,experiments/detide_prior_result.md,experiments/detide_prior_preregistration.md - Statewide prior: the previous result net of its year's national mood (adjustment register, description). The starting prior of a Senate or governor race is the seat's previous result, and that result carries the national mood of the year it was won in: a 2026 Senate prior taken from 2020 describes the seat in 2020's environment. Since 26 October 2026 that year's national level is taken out and this year's estimated level put in, both read from one fit of earlier cycles' Senate (or governor) results on the national House vote, built only from elections before the one forecast. It moves every 2026 Senate and governor prior toward the Democrats (Senate by 3.45 points on average, governors by 6.07) before the polls and the ground-up blend; the House is not changed. The de-tided prior passed its registered test with no measurable change in accuracy, and moved the published 2026 forecasts by 0.14 points on average over seventy-one races, with no rating or chamber change. In the Oklahoma governor's race, where the Republican is favored, the Republican's chance went from 84.2% to 83.0% and the Democrat's from 15.8% to 17.0%. Adopted together with the uncertainty-terms refit as single_race_latent 0.7.1 (the poll model) and 0.8.5 (the published blend); the 0.14-point figure is the de-tiding alone. ELAB_PRIOR_DETIDE=0 with the 2026-09-29 terms files restores 0.7.0.
Evidence:
experiments/detide_prior_preregistration.md,experiments/detide_prior_result.md,experiments/detide_ab_control.score.txt,experiments/detide_ab_treat.score.txt,docs/FAILURE_MODES.md FM-231 - Statewide prior: the previous result net of its year's national mood (adjustment register, test record). Registered as a consistency correction (amendment 1: the tide alone, the seat set cancelling), A/B over every champion election-eve backtest of 2016-2024 (256 races): adopted unless log loss worse by more than 0.002, Brier by more than 0.0005, or 80%/90% coverage further from nominal by more than 0.02. Published blend, control against treatment: log loss 0.1662 and 0.1662, Brier 0.0517 and 0.0518, coverage 0.836/0.945 and 0.840/0.945, MAE 4.58 and 4.58; adopted. The poll model alone was worse with it (log loss 0.1744 -> 0.1781, Brier 0.0554 -> 0.0571); the ground-up blend absorbs it.
Evidence:
experiments/detide_prior_preregistration.md,experiments/detide_prior_result.md,experiments/detide_ab_control.score.txt,experiments/detide_ab_treat.score.txt,docs/FAILURE_MODES.md FM-231 - Student-t conventions (docs/STATISTICAL_MODEL.md §0). Two conventions for a Student-t, one word each (audit 2026-10-26, M-12 and 7.2.4).
Evidence:
audit 2026-10-26, M-12 and 7.2.4 - Uncertainty terms refit (sigma_nat, sigma_race, drift) (docs/STATISTICAL_MODEL.md §11). **The files in force (refitted 2026-10-26, single_race_latent 0.7.1).** σ_nat, σ_race and the
drift law come from `experiments/uncertainty_terms_pre<T>.json`, fitted by
`scripts/estimate_uncertainty_terms.py --before T` on cycles before T only. Until 2026-10-26 they
were the 2026-09-29 fit, which scored each race's final polling average against the highest-share
Democratic and Republican rows of any round (a runoff's or a ranked-choice count's final round,
not the November round the polls measured), read superseded result rows, and dropped races whose
parties were stored under another spelling (FM-232). The fit now scores the round the polls were
taken for, live rows only, every party spelling. Refitted for T = 2016-2026 and tested as a
consistency correction over every election-eve backtest of 2016-2024
(experiments/uncertainty_terms_refit_result.md): the published forecast's log loss 0.1662 ->
0.1641, Brier 0.0517 -> 0.0516, 80%/90% coverage 0.836/0.945 -> 0.824/0.938, MAE 4.58 -> 4.57;
adopted. For 2026: σ_nat 2.44 -> 2.63, σ_race 4.97 -> 4.93, drift a = 1.477 -> 1.650 and b = 0.691
-> 0.669 (1,226 races over 25 cycles, against 587 over 18). The record covers election eve only,
where the drift term is near zero, so the refitted drift law reaches the forecasts weeks out
untested. The files for 2004-2014 (used only by the 2006-2014 backtests of 0.7.0) were not
refitted. The 2026-09-29 files are kept as `uncertainty_terms_pre<T>_2026-09-29.json`.
Evidence:
docs/FAILURE_MODES.md FM-232,experiments/uncertainty_terms_refit_result.md,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_pre<T>_2026-09-29.json
October 25, 2026
- Chamber simulations keep each race's published forecast (adjustment register, test record). Registered rules: every race within 1 point of its published chance (largest gap 0.0), the chamber gate passed, log score not worse by more than 0.10 (slightly better), in both chambers. Adopted.
Evidence:
experiments/chamber_marginals_preregistration.md,experiments/chamber_marginals_result.md - Changes kept after failing their rule (/methodology, Does a language model write any of these numbers?). Two changes have been adopted after failing their rule, by the project owner's decision: the House rater blend (disclosed on the accuracy page) and the statewide scandal rule, whose registered fading version failed three of its criteria and whose persistent 3-point version, registered for information only, was adopted instead (Flash audit 2026-10-25, F5).
Evidence: /accuracy,
experiments/shock_shift_result.md,experiments/house_rater_blend_result.md - Design effect (docs/STATISTICAL_MODEL.md §0 table). **implemented**, measured walk-forward from the dispersion law (2026: 0.757; 1.6 where nothing is measured), adopted 2026-10-25 |
Evidence:
data/model/adjustments.json design_effect,experiments/design_effect_measured_result.md - Design effect on poll sampling error (adjustment register, description). Every poll's sampling variance is multiplied by a design effect. Until 25 October 2026 it was a fixed 1.6; since then it is the value the project's own dispersion law measures from earlier cycles (0.76 for 2026, from 2018-2024), adopted on its registered test. Earlier cycles with nothing measured before them keep 1.6.
Evidence:
docs/STATISTICAL_MODEL.md section 4.1,experiments/poll_dispersion*.json (the measured values),experiments/design_effect_measured_preregistration.md,experiments/design_effect_measured_result.md - Design effect on poll sampling error (adjustment register, name). Design effect on poll sampling error: measured, walk-forward (was 1.6)
Evidence:
docs/STATISTICAL_MODEL.md section 4.1,experiments/poll_dispersion*.json (the measured values),experiments/design_effect_measured_preregistration.md,experiments/design_effect_measured_result.md - Design effect on poll sampling error (adjustment register, test record). Registered test 2026-10-25 (experiments/design_effect_measured_preregistration.md), every champion election-eve backtest of 2016-2024 against the 1.6 control: log loss 0.1641 against 0.1644, Brier 0.0507 both, 80%/90% coverage 0.863/0.945 against 0.867/0.949, no cycle worse by more than 0.0002. Adopted; the gain is small (experiments/design_effect_measured_result.md).
Evidence:
docs/STATISTICAL_MODEL.md section 4.1,experiments/poll_dispersion*.json (the measured values),experiments/design_effect_measured_preregistration.md,experiments/design_effect_measured_result.md - Ground-up prior on the all-vote scale (adjustment register, test record). Consistency correction (log loss worse by no more than 0.002, Brier by no more than 0.0005, coverage no further from nominal by more than 0.02): adopted; rerun with the corrected factor, MAE 0.05 points worse.
Evidence:
experiments/prior_all_vote_preregistration.md,experiments/prior_all_vote_result.md - Measured error of a sparse race's poll average (adjustment register, challenge). 2026-10-25 (experiments/sparse_shared_once_preregistration.md): shared error counted once. Log loss 0.1630 against 0.1644, Brier 0.0504 against 0.0507, 80% coverage 0.847 against 0.867, 90% 0.949 both (the rule's condition not met on the tie). Adopted by the owner's amendment; both versions scored after November 2026.
Evidence:
experiments/sparse_blend_poll_weight.md,docs/FAILURE_MODES.md FM-104,experiments/sparse_variance.md (the first registration, rejected on its metric),experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment),experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026) - Measured error of a sparse race's poll average (adjustment register, description). In the sparse blend a handful of polls is weighted by how far such averages have actually landed from results in earlier cycles (about 9-10 points), not by their sampling error (about 4). Since 25 October 2026 the national and race-specific parts of that error are taken out of it before the blend, because the forecast adds both again afterwards: they had been counted twice (adopted by the owner's decision after its registered test tied on one condition; experiments/sparse_shared_once_result.md).
Evidence:
experiments/sparse_blend_poll_weight.md,docs/FAILURE_MODES.md FM-104,experiments/sparse_variance.md (the first registration, rejected on its metric),experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment),experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026) - Senate and governor: scandal rule (adjustment register, description). After a verified scandal about the candidate themselves (a criminal charge or conviction, a specific misconduct allegation or official finding, or a documented false claim about their own record, reported by two outlets doing their own reporting), the race's margin moves 3.0 points against that candidate from the event's first date, only while the race has fewer than three polls taken after the event. Polls taken after the event already show most of its effect, so with three or more of them the margin is the polls' and the blend's alone. The spread is unchanged. A running mate's, relative's or staffer's trouble does not count. Events are listed by hand in config/statewide_scandal_events_2026.yaml with their sources; races forecast without polls take the shift too. One event is listed: Colorado governor (Victor Marx, from 2026-07-24). Borderline cases (ME, AK, TX, MN and KS Senate) are listed with the evidence that would make them count and are re-checked at each daily review.
Evidence:
experiments/shock_shift_result.md (17 Senate and governor races, registered as information),experiments/house_scandal/result.md (the House test, which met its registered criterion),config/statewide_scandal_events_2026.yaml (the verified events),experiments/shock_shift_prospective_preregistration.md (scored after certification),experiments/scandal_investigation/ (why the shift stops at three post-event polls),experiments/scandal_rule_v2_preregistration.md (the second test; arm A1 is this rule),experiments/scandal_polled_limit_result.md
October 24, 2026
- Chamber simulation: correlated errors (adjustment register, description). Senate and governor control probabilities draw one shared national error (and regional and office terms) per simulated election across all races; the model's measured lean is shown only as a sensitivity beside the headline, never applied to it. The shared national error is drawn wider there than on a race page, on purpose (FM-14): its size is itself estimated from past cycles, so it is a Student-t with scale sigma_nat times sqrt(1 + 1/n) and n - 1 degrees of freedom; for 2026 (17 cycles, 2026-10-24 run) that is about 10% wider in sd. Its tail weight is estimated from the past cycles (nu between 4 and 20; 20 for 2026, close to a normal), while each race's own error is a Student-t with 5 degrees of freedom on both the race page and in the chamber.
Evidence:
experiments/chamber_gate_preregistration.md,experiments/chamber_gate_result.md - Economic data vintages (/accuracy, introduction). With the figures as first published (ALFRED vintages of October 15), its fundamentals estimate misses by 0.14 points more on average, and the blended national estimate it feeds by slightly less (experiments/economic_vintages_2026-10-24.md).
Evidence:
experiments/economic_vintages_2026-10-24.md - Ground-up prior: spread by size of predicted margin (adjustment register, test record). Registered rule: lower pooled CRPS of the prior alone (6.075 against 6.087) and 90% coverage in every class within 0.05 of nominal or closer than the control's: met. Adopted, on the prior alone and in the blend's whole-record test.
Evidence:
experiments/prior_sd_by_margin_preregistration.md,experiments/prior_sd_by_margin_result.md - Promotion gate: significance test (docs/BACKTESTING.md §7 and §7.1). Out-of-sample score improves under an exact one-sided sign-flip test of the per-cycle differences (races within a cycle are not independent; since 2026-10-24, replacing a percentile bootstrap that overstated the evidence with few cycles: under a complete null, 41% of batches promoted something against a 10% target). The superseded description in §7.1 read: "The p-value comes from resampling cycles, with a +1 correction so that a p-value of exactly zero is unattainable — with seven cycles it is not evidence that strong, and letting the arithmetic say otherwise invites the ratchet."
Evidence:
src/elab/registry/promotion.py sign_flip_p_value,docs/DECISIONS.md ADR-0014 - Promotion test (/accuracy, introduction). The same past elections have also been used to test many proposed changes (most written down before they were run; the list is below), and every test run on them weakens what they can show; the model-promotion test was corrected on 2026-10-24 after an outside review found it overstated the evidence.
Evidence: /record/tests
- Promotion test (/methodology, How does a new model get adopted?). Statewide race-model challengers are compared with the champion on a cycle-level sign-flip test (exact, treating each election as one unit and assuming the per-election differences are symmetric about zero when there is no improvement; it used a bootstrap until October 24, 2026) with Benjamini-Hochberg control of false discoveries.
Evidence:
docs/BACKTESTING.md - Promotion test (/methodology, How does a new model get adopted?). An outside review on October 24, 2026 found the statistical test behind the promotion gate overstated the evidence with so few elections (under pure noise, 41% of batches promoted a change against a 10% target); it was replaced by an exact sign-flip test, and the two changes it had promoted still pass, at the weakest evidence five elections allow.
Evidence:
docs/BACKTESTING.md - Results read at their November round in nominee runoffs (adjustment register, test record). Two corrections of stored data, each by its registered rule: the panel by the statewide gate (better on every pooled measure), the start prior adopted unless Brier worse by more than 0.0005 or MAE by more than 0.05 (Brier +0.0001, MAE equal).
Evidence:
experiments/november_round_preregistration.md,experiments/november_round_result.md,experiments/november_round_prior_preregistration.md,experiments/november_round_prior_result.md - Senate and governor: scandal rule (adjustment register, parameters). 3.0 points, persistent while the race has fewer than three polls whose fieldwork started after the event's first date (counted per contender, as of each forecast); none after that. PROVISIONAL: the size rests on 17 statewide and 12 House cases, the statewide evidence registered as information rather than a confirmatory test; on election eve the polls taken after the event already carried most of the effect (experiments/scandal_investigation/), which is why the shift stops at three of them; widening the range was tested and rejected (shock_uncertainty_result.md: 90% ranges already held 0.92 of shocked races); scored prospectively after certification (scandal_rule_v2_preregistration.md, arm A1)
Evidence:
experiments/shock_shift_result.md (17 Senate and governor races, registered as information),experiments/house_scandal/result.md (the House test, which met its registered criterion),config/statewide_scandal_events_2026.yaml (the verified events),experiments/shock_shift_prospective_preregistration.md (scored after certification),experiments/scandal_investigation/ (why the shift stops at three post-event polls),experiments/scandal_rule_v2_preregistration.md (the second test; arm A1 is this rule),experiments/scandal_polled_limit_result.md - Student-t scale of the poll likelihood (docs/STATISTICAL_MODEL.md §4.1). What Σ_i is, exactly (settled 2026-10-24). [...] Correcting the parameterisation so that Σ_i is the variance was tested over 2016-2024 and scored the same within noise (log loss 0.1629 against 0.1626; experiments/student_t_scale_result.md), and its registered rule did not adopt it.
Evidence:
experiments/student_t_scale_result.md - Width of the statewide prior, net of the national swing (adjustment register, test record). Consistency correction: adopted unless log loss worse by more than 0.002, Brier by more than 0.0005 or 80%/90% coverage further from nominal by more than 0.02; log loss 0.1642 -> 0.1643, Brier and coverage equal. Adopted.
Evidence:
experiments/prior_width_net_preregistration.md,experiments/prior_width_net_result.md
October 23, 2026
- Campaign- and party-sponsored polls (adjustment register, description). Sponsored polls (a campaign, party committee, PAC or interest group) are kept; the sponsor's lean is left to the pollster's house effect. Roughly half of live statewide polls are flagged sponsored (327 of 676 when counted on 2026-10-05). Tested against dropping them, statewide and for the House (2026-10-23); kept by the owner's decision on that result. The in-house polling operations of campaigns and party committees (nine firms) are excluded by name (config/excluded_pollsters.yaml).
Evidence:
experiments/preregistration_polled_nonpartisan.md (the registration of the arm that drops sponsored polls, refused in house_polled_nonpartisan.md),experiments/house_polled_nonpartisan.md (dropping sponsored polls: refused),experiments/partisan_polls_backtest_2026_10_23_preregistration.md,experiments/partisan_polls_backtest_2026_10_23_result.md - Campaign- and party-sponsored polls (adjustment register, test record). Statewide: the champion's 2016-2024 election-eve record rerun with and without 1,194 partisan-sponsored polls (log loss 0.1640 vs 0.1620 pooled, 2016 worse by 0.011, four of five cycles worse; removal not supported). House: the deployed poll layer with and without sponsored polls (31-day lean MAE 4.15 vs 4.96; removal not supported). Earlier: House seat MAE 2018-2024, dropping refused (8.1 vs 6.0).
Evidence:
experiments/preregistration_polled_nonpartisan.md (the registration of the arm that drops sponsored polls, refused in house_polled_nonpartisan.md),experiments/house_polled_nonpartisan.md (dropping sponsored polls: refused),experiments/partisan_polls_backtest_2026_10_23_preregistration.md,experiments/partisan_polls_backtest_2026_10_23_result.md
October 22, 2026
- House: scandal rule (adjustment register, description). After a verified scandal about a House nominee (a criminal charge or conviction, a specific misconduct allegation, or a documented false claim about their own record, reported by two outlets doing their own reporting), the district's margin moves 3.0 points against that nominee from the event's first date; the spread is unchanged. The events are listed by hand in config/house_scandal_events_2026.yaml with their sources; party splits are not part of the rule. House model 2.7.0, adopted by the owner on 2026-10-22.
Evidence:
experiments/house_scandal_preregistration.md,experiments/house_scandal/result.md - House: scandal rule (adjustment register, test record). Pre-registered House test 2018-2024: 12 scandal candidates, all finished below the forecast (mean 5.4 points; 3.3 without Collins); the 3.0-point shift improved CRPS -0.77 [-1.31, -0.18], Holm p 0.045. Also scored prospectively on 2026 after certification.
Evidence:
experiments/house_scandal_preregistration.md,experiments/house_scandal/result.md - Senate and governor: scandal rule (adjustment register, test record). Senate and governor 17 shocked races (shock_shift_result.md, registered as information): the persistent shift at 3.0 scored better than the champion on every metric (leave-one-out CRPS 3.37 to 3.19, Brier 0.134 to 0.096, winner calls 76% to 87%), but its CRPS interval [-0.71, +0.38] includes zero and the gain rests largely on Akin. The House test of the same size (12 candidates, CRPS -0.77 [-1.31, -0.18], Holm 0.045) met its registered criterion. Adopted by the owner on both; scored prospectively on 2026 after certification.
Evidence:
experiments/shock_shift_result.md (17 Senate and governor races, registered as information),experiments/house_scandal/result.md (the House test, which met its registered criterion),config/statewide_scandal_events_2026.yaml (the verified events),experiments/shock_shift_prospective_preregistration.md (scored after certification),experiments/scandal_investigation/ (why the shift stops at three post-event polls),experiments/scandal_rule_v2_preregistration.md (the second test; arm A1 is this rule),experiments/scandal_polled_limit_result.md
October 20, 2026
- House rater blend (/accuracy, House districts and the raters). A version with a much larger share for close races was tested on 2026-10-20 with the test written first, and failed: following the raters that far also follows them when they miss together.
Evidence:
experiments/house_tiered_blend_result.json - Polls left out (/methodology, How the forecast works). Some firms' polls are never used: a firm that admitted or was shown on original reporting to have fabricated its data (Wedgewood Polls, 2026-10-20), a firm with no record as a pollster that can be checked, and a campaign's or party committee's own in-house polling operation.
Evidence: /pollsters
October 12, 2026
- Campaign- and party-sponsored polls (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were worse on average and better in 6 of 8 past election years, but it worked the other way in presidential years, governor races, lopsided races; four weeks out, forecasts with it were better on average and better in 6 of 8 past election years, but it worked the other way in midterm years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Chamber simulation: correlated errors (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Acts on chamber totals, not race forecasts: one chamber outcome per cycle gives five to ten scored points, too few for the per-cycle rule. Race-level numbers are unaffected.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Combined sponsor and firm records (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Not in the published path (rejected).
Evidence:
experiments/adjustment_audit_2026-10-12.md - Design effect on poll sampling error (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, the ranges missed their nominal coverage by 4.4 points with it and 4.4 without; four weeks out, the ranges missed their nominal coverage by 6.2 points with it and 5.6 without. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Drift to Election Day (adjustment register, challenge). A walk-forward drift multiplier was tested 2026-10-12 (experiments/statewide_spread_result.md) and not adopted: coverage error 0.083 -> 0.080 four weeks out, short of the 0.01 required.
Evidence:
experiments/uncertainty_terms.md,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Drift to Election Day (standard test of every adjustment). Flagged for review (2026-10-12): four weeks out, the ranges missed their nominal coverage by 6.5 points with it and 2.4 without. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Four polls to fit a trend (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): A threshold with no 'off' state: below three polls the latent model cannot be fitted at all. Comparing 3 against 4 is a specification change under policy (b).
Evidence:
experiments/adjustment_audit_2026-10-12.md - Fundamentals prior on the race's starting level (standard test of every adjustment). Passed the standard test as registered (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 8 of 10 past election years; four weeks out, forecasts with it were better on average and better in 8 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Generic-ballot bias correction (adjustment register, description). Subtracts the generic ballot's average past miss (2.62 points toward the Democrat, from 1998-2024) before it enters the national term. The same national term is in the ground-up prior of the Senate and governor forecasts (scripts/gate_statewide.py), so it reaches them too. It is kept while the statewide polls' own intercept was removed on 12 October 2026 because the two rest on different evidence: that intercept was fitted on race polls against race results and hurt in midterms; this correction is fitted on the generic ballot against the national House vote, and its midterm-only miss (2.70 points over seven midterms) is the same size as its pooled one, so the midterm evidence that removed the one does not apply to the other. It has not had the standard test on its own.
Evidence:
experiments/national_environment.md,experiments/house_generic_ballot_bias.md (refused: the same kind of correction, tested in the retired house_prior/1.7.0 and its corpus, not in N3),experiments/preregistration_generic_ballot_bias.md,experiments/mean_bias_challengers_2026-10-05.md (the challengers to it, no correction and a same-kind one: not promoted) - Ground-up blend (the published Senate and governor forecast) (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 5 of 5 past election years, but it worked the other way in governor races, lopsided races; four weeks out, forecasts with it were better on average and better in 4 of 5 past election years, but it worked the other way in governor races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Heavy-tailed election-day errors (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 5 of 10 past election years, but it worked the other way in lopsided races; four weeks out, forecasts with it were better on average and better in 7 of 10 past election years, but it worked the other way in presidential years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - House state and district shocks (docs/STATISTICAL_MODEL.md §0). The error model's national draw is normal; its state and district shocks are Student-t with 4 degrees of freedom whose scale is 0.5 times the walk-forward sd (since 2.6.0, experiments/house_spread_result.json: held-out district residuals have kurtosis near 14), not normal.
Evidence:
experiments/house_spread_result.json - House: district error by race type (adjustment register, challenge). 2.6.0 (2026-10-12): state and district shocks drawn from Student-t (4 df) at 0.5 x the walk-forward sd, chosen by a pre-registered coverage test (experiments/house_spread_result.md): 80%/90% coverage 0.925/0.976 -> 0.836/0.935, Brier and seat intervals no worse.
Evidence:
experiments/district_gate_preregistration.md - House: district poll update (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Tested by its own pre-registered test (scripts/test_district_polls_v2.py); the adjustment-audit harness has no House district-poll arm.
Evidence:
experiments/adjustment_audit_2026-10-12.md - House: district polls weighed against the blended lean (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Tested by its own pre-registered test; the adjustment-audit harness has no House arm for it.
Evidence:
experiments/adjustment_audit_2026-10-12.md - House: rater blend (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Tested by its own pre-registered test; the adjustment-audit harness has no House rater arm.
Evidence:
experiments/adjustment_audit_2026-10-12.md - House: seats without a two-party contest (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): These seats get no district forecast; the walk-forward gate assigned them from actual winners (an oracle), so there is no historical 'on' arm to score. Correctness under policy (a): the old rule made a seat with no nominee on record a Republican seat (LA-2 before FM-155 repaired Louisiana's rosters); fixed by FM-173, no effect on today's House (no such seat), +0.99 seats on the pre-FM-155 data.
Evidence:
experiments/adjustment_audit_2026-10-12.md - House: shared state error (adjustment register, challenge). 2.6.0 (2026-10-12): state and district shocks drawn from Student-t (4 df) at 0.5 x the walk-forward sd, chosen by a pre-registered coverage test (experiments/house_spread_result.md): 80%/90% coverage 0.925/0.976 -> 0.836/0.935, Brier and seat intervals no worse.
Evidence:
experiments/district_gate_preregistration.md,experiments/district_state_var_preregistration.md,experiments/district_state_var_result.md,docs/FAILURE_MODES.md FM-236 - House: the three core raters (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Tested by its own pre-registered test; the adjustment-audit harness has no House rater arm.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Likely-voter adjustment (adjustment register, description). Nothing is applied: likely-voter, registered-voter and adult samples enter alike; the population column is stored and read by nothing in the model. When a release is listed in several populations, one version is kept: likely voters, else registered voters, else adults, whatever order the source lists them in (since 2026-10-10, audit M02; before, whichever was stored first). A firm's typical lean, including any from its population, is absorbed by its house effect. A likely-voter adjustment (registered-voter and adult polls moved toward the Republican by the gap measured where 538 lists a poll both ways: -1.18 points for 2026) was pre-registered and tested on 2026-10-12 and refused: slightly worse log loss, better in 1 of 3 cycles; today it would move races by 0.26 points in the median.
Evidence:
docs/STATISTICAL_MODEL.md section 4 (describes an LV/RV term g_pop that is not implemented),docs/FAILURE_MODES.md FM-79 (the sigma_lv placeholder retired into sigma_race),experiments/preregistration_lv_rv_adjustment.md,experiments/lv_rv_adjustment_result.md (the adjustment tested and failed, so none is applied) - Likely-voter adjustment (adjustment register, test record). Pre-registered addition (experiments/preregistration_lv_rv_adjustment.md), standard test 2020-2024, eve and h28: refused (log loss gain -0.0009 eve, -0.0004 h28; 1 of 3 cycles improved; margin MAE 4.736 with vs 4.759 without).
Evidence:
docs/STATISTICAL_MODEL.md section 4 (describes an LV/RV term g_pop that is not implemented),docs/FAILURE_MODES.md FM-79 (the sigma_lv placeholder retired into sigma_race),experiments/preregistration_lv_rv_adjustment.md,experiments/lv_rv_adjustment_result.md (the adjustment tested and failed, so none is applied) - Likely-voter adjustment (adjustment register, why it could not be tested). Nothing is applied, so there is nothing to switch off. The one alternative, a likely-voter adjustment, was tested separately on 2026-10-12 and refused (populations are recorded from 2020 on, so three cycles were scored).
Evidence:
docs/STATISTICAL_MODEL.md section 4 (describes an LV/RV term g_pop that is not implemented),docs/FAILURE_MODES.md FM-79 (the sigma_lv placeholder retired into sigma_race),experiments/preregistration_lv_rv_adjustment.md,experiments/lv_rv_adjustment_result.md (the adjustment tested and failed, so none is applied) - Likely-voter adjustment (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Nothing is applied, so there is nothing to switch off; 7,242 of 11,258 historical statewide polls record no population at all, so an LV/RV term cannot be fitted walk-forward from this corpus. Adding one would be a new specification under policy (b).
Evidence:
experiments/adjustment_audit_2026-10-12.md - Mean polling-bias correction (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Not in the published path (rejected); its question is answered by scale_intercept.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Measured error of a sparse race's poll average (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, the ranges missed their nominal coverage by 4.4 points with it and 2.0 without; four weeks out, the ranges missed their nominal coverage by 6.5 points with it and 4.6 without. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Overlapping tracker waves (docs/STATISTICAL_MODEL.md §0 table). Overlapping tracker samples, ρ_overlap (§4.2.1): canonical non-overlapping waves (since 2026-10-12, audit M08).
Evidence:
model audit M08 - Pollster house effects (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 6 of 10 past election years, but it worked the other way in presidential years; four weeks out, forecasts with it were worse on average and better in 5 of 10 past election years, but it worked the other way in midterm years, governor races, lopsided races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Race-specific error (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, the ranges missed their nominal coverage by 4.4 points with it and 52.7 without; four weeks out, the ranges missed their nominal coverage by 6.5 points with it and 15.3 without. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Range widening and fading shifts after major events (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Not in the published path (switched off after failing their own pre-registered tests).
Evidence:
experiments/adjustment_audit_2026-10-12.md - Scale law (/methodology/parameters). The law is fitted as intercept + slope × estimate; since October 12, 2026 (0.8.3) only the slope is applied, and the intercept, a uniform shift, is not.
Evidence:
experiments/scale_intercept_removal_2026-10-12.md - Scale law (reference entry) (adjustment register, description). The whole forecast distribution was translated so its centre became a + b x centre, with a and b fitted on earlier cycles' final polling averages against results (2026: -1.744 + 1.070 x centre). Since 12 October 2026 (0.8.3) the intercept a is taken back out after the law, so what is applied is the stretch b x centre alone. Reported for reference; the two parts are judged separately below.
Evidence:
experiments/preregistration_scale.md,experiments/scale_compression.md,experiments/scale_calibration_2016.md,experiments/scale_intercept_check_2026-10-05.md - Scale law (reference entry) (adjustment register, name). Scale law c15 as a whole (reference entry: the slope is applied, as scale_slope; the intercept is removed since 2026-10-12)
Evidence:
experiments/preregistration_scale.md,experiments/scale_compression.md,experiments/scale_calibration_2016.md,experiments/scale_intercept_check_2026-10-05.md - Scale law (reference entry) (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years, lopsided races; four weeks out, forecasts with it were better on average and better in 5 of 10 past election years, but it worked the other way in midterm years, close races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Scale law: intercept (adjustment register, challenge). Removed 2026-10-12 (owner's decision on audit M11). On the stored 2016-2024 record, paired on the same source runs (234 races, election eve), 0.8.3 against 0.8.2: log loss 0.1733 -> 0.1691; midterms 0.1857 -> 0.1691 (121 races), presidential years 0.1600 -> 0.1690 (113); nominal 80%/90% ranges cover 0.876/0.957 overall and 0.926/0.984 in midterms (experiments/scale_intercept_removal_2026-10-12.md).
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (adjustment register, description). Moved every forecast by the fitted intercept, -1.744 points toward the Republican for 2026: a mean polling-bias correction learned from all past cycles together. Removed on 12 October 2026 by the owner's decision on audit M11 (single_race_latent/0.8.3): on the record it helped in presidential years and hurt in midterms, its pooled log loss was worse, and 2026 is a midterm. Until then it moved the published Senate and governor forecasts 1.4 points toward the Republican in the median race (at most 1.7) and win probabilities by 2.6 points (at most 11.7). The slope stays.
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (adjustment register, name). Scale law: intercept (a uniform shift; removed 2026-10-12)
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (adjustment register, test record). Standard test 2026-10-12 on the total-vote target (adjustment_audit_2026-10-12): pooled log loss 0.1604 with it, 0.1529 without (375 races, election eve); midterms worse with it (-0.020 log loss gain, 217 races), presidential years better (+0.010, 158); same direction 28 days out. Flagged for review. The 2026-10-05 check found the same split.
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (adjustment register, why it could not be tested). Not in the published path since 0.8.3 (removed 2026-10-12); listed for its evidence.
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years; four weeks out, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Scale law: slope (adjustment register, description). Moves the centre of every forecast away from a tie by the fitted slope (1.070 for 2026: a 10-point lead becomes 10.7), because final polling averages have been more compressed than results; the spread is not changed. Since 0.8.3 (12 October 2026) it is the only part of the scale law applied: the slope is fitted together with an intercept that is no longer used (a slope fitted without one would be 1.048).
Evidence:
experiments/preregistration_scale.md,experiments/scale_compression.md,experiments/compression_mechanism.md - Scale law: slope (adjustment register, test record). Promoted jointly with the intercept (c15, 2026-09-26); applied without it, tested as arm a0 (experiments/scale_intercept_check_2026-10-05.md) and in the adjustment audit of 2026-10-12 (KEEP).
Evidence:
experiments/preregistration_scale.md,experiments/scale_compression.md,experiments/compression_mechanism.md - Scale law: slope (standard test of every adjustment). Passed the standard test as registered (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 8 of 10 past election years; four weeks out, forecasts with it were better on average and better in 8 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Shared national polling error (adjustment register, challenge). Re-estimated on today's data and tested 2026-10-12 (experiments/statewide_spread_result.md): the wider refreshed terms (2026: 3.45, or 3.00 on federal cycles) moved coverage further from nominal; the 2026-09-29 files stay.
Evidence:
experiments/uncertainty_terms.md,docs/FAILURE_MODES.md FM-82,experiments/uncertainty_terms_refit_preregistration.md,experiments/uncertainty_terms_refit_result.md,docs/FAILURE_MODES.md FM-232 - Shared national polling error (standard test of every adjustment). Flagged for review (2026-10-12): on the eve of the election, the ranges missed their nominal coverage by 4.4 points with it and 0.9 without; four weeks out, the ranges missed their nominal coverage by 6.5 points with it and 4.6 without. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Sparse-race blend (standard test of every adjustment). Passed the standard test as registered (2026-10-12): on the eve of the election, forecasts with it were better on average and better in 9 of 9 past election years; four weeks out, forecasts with it were better on average and better in 10 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Statewide ground-up blend (race pages, From polls to forecast). Four weeks out the evidence is weaker: better overall on 209 races, but the standard audit found it better in only 3 of 5 elections and worse for governor races.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Statewide polling lean (/methodology, /accuracy and /chambers: the paragraph on correcting the historical lean). Three pre-registered evaluations on September 25, 2026 tested a correction that centered the forecast on the historical lean, using earlier versions of the statewide model (0.1.0 to 0.3.0) on 2018–2024 races. All three were rejected on the pre-registered log-loss gate; the correction over-corrected 2022, the cycle with the smallest measured bias. A separate analysis of subtracting the average historical polling error from race polls also found no support; the correction that reaches the forecasts instead rests on the generic ballot against the national House vote. Until October 12, 2026 the published Senate and governor forecasts applied the scale correction's shift toward Republicans, a median of 1.4 points across the 2026 races and at most 1.74 points in any of them. An audit on October 12, 2026 found that the shift helped in presidential years and hurt in midterms. The removal analysis shows that removing the shift improved log loss and Brier scores but worsened mean absolute error and mean error, and the shift was removed that day (version 0.8.3). The scale correction's stretch away from a tie remains; that stretch favors neither party. A correction toward Republicans does reach the Senate and governor forecasts indirectly: the national term in their ground-up estimate subtracts the generic ballot's average past miss toward Democrats, as on the House forecast. The lean above was measured on the current model with that generic-ballot correction in place.
Evidence:
experiments/scale_intercept_removal_2026-10-12.md,experiments/mean_bias_challengers_2026-10-05.md,experiments/bias_centre.md - Turnout and enthusiasm (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Not in the published path (rejected); no turnout model is published.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Unpolled races published from the ground-up prior (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): Switching it off leaves the race unpublished, not forecast differently: there is no 'off' forecast to score. The prior's own accuracy is the evidence.
Evidence:
experiments/adjustment_audit_2026-10-12.md - What a statewide margin measures (adjustment register, test record). Raw poll margins with a walk-forward scale law against undecideds allocated in proportion before averaging, 748 races, total-vote target: the allocation is worse overall (MAE +0.34, 90% CI +0.20 to +0.50) and in every undecided and third-party band.
Evidence:
experiments/audit_m07_result.md,experiments/audit_m07_allocation.json,experiments/compression_mechanism.md - What a statewide margin measures (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): The definition of the quantity forecast and scored, not a step that can be switched off; its one alternative treatment (allocating undecideds) was tested separately, as above.
Evidence:
experiments/adjustment_audit_2026-10-12.md - Which polls a forecast reads (adjustment register, parameters). ELAB_RECON_RELEASE_BASIS=released_at, ELAB_RECON_RELEASE_LAG_DAYS=2 (since 2026-10-10, audit M13; before: field_end, 0); ELAB_POLL_WAVES=per_question (since 2026-10-12, audit M08: of a firm's overlapping waves of one question in a race, the newest and those overlapping no kept wave; before: all). Each run records its rule.
Evidence:
docs/BACKTESTING.md,tests/leakage/test_asof_canary.py,tests/leakage/test_release_basis.py - Which polls a forecast reads (standard test of every adjustment). Cannot be tested on past elections (2026-10-12): A knowability rule, not an adjustment: switching it off is leakage, not an alternative forecast. Guarded by the leakage tests.
Evidence:
experiments/adjustment_audit_2026-10-12.md
October 11, 2026
- House: national uncertainty (adjustment register, description). Every simulated House election draws one national environment around N3. Its spread is the national estimator's walk-forward error at the end of the campaign (2.75 points), plus the movement still to come before then and the extra measurement error of today's generic-ballot average beyond what past end-of-campaign averages had (3.48 points in all on 2026-10-11; house_ground_up 2.5.0, audit M12).
Evidence:
experiments/district_gate_preregistration.md,experiments/audit_m12_result.md (the live spread adds the movement still to come and today's measurement excess, house_ground_up 2.5.0; FM-195) - House: the corrected national environment (adjustment register, parameters). data/national/backtest.json; 2026 (2026-10-11): N0 9.12 -> N3 7.28, from today's average
Evidence:
experiments/national_environment.md,experiments/district_gate_result.md - Measured error of a sparse race's poll average (adjustment register, parameters). Realised sd of a 1-3-poll average around the result, pooled over earlier cycles: 9.37 pts for 2026 (measured on the total-vote target since 2026-10-11, audit M07; 9.98 against the two-candidate renormalised result before); ELAB_SPARSE_SHARED_ONCE on by default (the national and race-specific parts taken out once), 0 restores the double count
Evidence:
experiments/sparse_blend_poll_weight.md,docs/FAILURE_MODES.md FM-104,experiments/sparse_variance.md (the first registration, rejected on its metric),experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment),experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026) - Statewide forecast target (docs/STATISTICAL_MODEL.md §0). **The target (audit M07, 2026-10-11).** Every statewide product forecasts, and is scored on, the
margin between the two contenders as shares of **all votes cast** (D% - R%, third parties and
write-ins in the denominator) -- the way results are reported, and the scale the fundamentals prior,
the scale law and the uncertainty terms were already fitted on. Until 2026-10-11 the scoring alone
renormalised the outcome to the two candidates' combined vote (FM-87); the scored record was
recomputed on the target (FM-197).
Evidence:
model audit M07,docs/FAILURE_MODES.md FM-87,docs/FAILURE_MODES.md FM-197 - Undecided allocation (docs/STATISTICAL_MODEL.md §0 table). Allocating undecideds and third parties in proportion to the two candidates before averaging was compared and is worse in every undecided and third-party band (MAE +0.34, 90% CI +0.20 to +0.50; experiments/audit_m07_result.md).
Evidence:
experiments/audit_m07_result.md - What a statewide margin measures (adjustment register, parameters). Since 2026-10-11 (audit M07): scoring moved from the two-candidate renormalised result to the total-vote margin the models were already fitted on.
Evidence:
experiments/audit_m07_result.md,experiments/audit_m07_allocation.json,experiments/compression_mechanism.md
October 10, 2026
- Combined sponsor and firm records (adjustment register, test record). Pre-registered standard test (eve and four weeks out, 2006-2024): log loss -0.0002 and -0.0011 with pooling (worse), better in 1 of 6 cycles four weeks out, sign reversal for the Senate; House check identical to the published arm. Not adopted.
Evidence:
experiments/sponsor_pooling_preregistration.md,experiments/sponsor_pooling_result.md,experiments/sponsor_pooling_result.json,experiments/sponsor_pooling_house_result.json - Implemented terms versus specification (docs/STATISTICAL_MODEL.md §0). ## 0. What is implemented, and what below is a proposal (model audit M15, 2026-10-10)
Several sections below were written as the specification to build towards, and read as if every
term were fitted. They are not. The published statewide fit (`src/elab/models/latent/single_race.py`,
`single_race_latent/0.7.0`, blended into 0.8.2) reads four things per poll
Evidence:
model audit M15 - Turnout and enthusiasm (adjustment register, description). No published forecast estimates who will turn out in November or how enthusiastic each side is. That enters only indirectly: through the polls (likely-voter screens are the pollsters' own turnout models), past results, demographics, the national environment, and an error band measured on past elections whose turnouts varied. Two bottom-up turnout models were tested for the House and not adopted: one built from each district's demographic groups made the seat count and the probabilities worse; one built from a large voter survey was no better than noise and worse in 2024. Both counted Hispanic voters inside other groups too, and the first used a 2021 estimate of how groups voted for its 2018 and 2020 tests, so they rule out those two designs, not turnout modelling in general (model audit 2026-10-10, M01, M04-M06).
Evidence:
experiments/electorate_backtest_result.md,experiments/ces_bottom_up_result.md - Turnout and enthusiasm (adjustment register, test record). Ecological: 2022 Brier 0.0363 -> 0.0422, expected-seat error 7.5 -> 12.5 (lean MAE 5.93 -> 5.25). Survey-based: selection Brier 0.03621 -> 0.03560 with a bootstrap interval including zero; 2024 Brier 0.0218 -> 0.0228. Neither adopted.
Evidence:
experiments/electorate_backtest_result.md,experiments/ces_bottom_up_result.md
October 9, 2026
- House: district polls weighed against the blended lean (adjustment register, description). When a district has polls, they are combined with the rater-blended lean using that blended lean's own measured error, not the unblended model's. Passed its pre-registered test (arm B): mean seat error 12.16 against 13.47 for the blend alone, Brier 0.112 against 0.121.
Evidence:
experiments/blend_plus_polls_preregistration.md,experiments/blend_plus_polls_result.json - House: district polls weighed against the blended lean (adjustment register, test record). Pre-registered (experiments/blend_plus_polls_preregistration.md): arm B passed as registered and replaced arm A.
Evidence:
experiments/blend_plus_polls_preregistration.md,experiments/blend_plus_polls_result.json - House: rater blend (adjustment register, description). Moves each rated district's lean part of the way toward the average category of the race raters, by a weight fitted on past cycles (about 15% today). It was pre-registered to be kept only if it lowered both the Brier score and the seat error; it lowered the Brier score (0.0331 against 0.0353) but raised the seat error (11.76 against 11.18, all of it from 2022). The project owner approved keeping it on 2026-10-08 and re-confirmed it on 2026-10-09 after the test was re-run on the election-eve rating history; the cost is stated on /accuracy and in ADJUSTMENTS §2.5b.
Evidence:
experiments/house_rater_blend_preregistration.md,experiments/house_rater_blend_result.json,experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json),experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification) - House: rater blend (adjustment register, name). House: rater blend (kept under an amended rule)
Evidence:
experiments/house_rater_blend_preregistration.md,experiments/house_rater_blend_result.json,experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json),experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification) - House: rater blend (adjustment register, test record). Pre-registered (experiments/house_rater_blend_preregistration.md, committed before running): keep only if lower Brier AND lower seat error than the model alone, 2018-2024 walk-forward. Result: lower Brier, higher seat error, so the rule as registered refused it. Kept by the project owner's amendment (2026-10-08, re-confirmed 2026-10-09), recorded in the result file as adopted_by_amendment; review point: the November 2026 House result, scored by the same rule.
Evidence:
experiments/house_rater_blend_preregistration.md,experiments/house_rater_blend_result.json,experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json),experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification) - House: seats without a two-party contest (adjustment register, description). A district without both a Democrat and a Republican on the ballot is given, unsimulated, to the one major party with a nominee on record (qual_D or qual_R present); a district where neither side, or both, has a nominee on record has an unknown field and is simulated from its lean (FM-173; before, it went to the Republican). A seat declared as forecast from its own polls (AK-AL, CA-6) is never given away this way (2026-10-09: CA-6 was briefly counted as a certain Democratic seat).
Evidence:
scripts/forecast_house_ground_up.py,docs/FAILURE_MODES.md FM-155, FM-173 - House: the three core raters (adjustment register, description). The rater average uses Cook, Sabato and Inside Elections only. Passed its pre-registered test against averaging every rater (Brier 0.0329 against 0.0331, seat error 11.66 against 11.76).
Evidence:
experiments/house_rater_core_preregistration.md,experiments/house_rater_core_result.json - House: the three core raters (adjustment register, test record). Pre-registered (experiments/house_rater_core_preregistration.md): passed as registered.
Evidence:
experiments/house_rater_core_preregistration.md,experiments/house_rater_core_result.json
October 8, 2026
- House rater blend (/accuracy and the district pages). Since House model 2.3.0, each rated district's lean is moved part of the way toward what that rating has meant in past elections. (District pages: "a step the House model has taken since its version 2.3.0 (see /accuracy)".) The test written before it was run required better seat totals as well, so it failed that rule. It was adopted anyway, by the project owner's decision after seeing the results: the district probabilities are what readers act on. Details: experiments/house_rater_blend_result.json.
Evidence:
experiments/house_rater_blend_result.json,docs/ADJUSTMENTS.md §2.5b - House rater blend (/methodology, How does a new model get adopted?). Two changes were adopted after failing their tests: on October 8, 2026 the House model's blend toward the raters improved the district probabilities but not the seat total, and the project owner adopted it anyway (/accuracy says so); and the statewide scandal rule, whose registered fading version failed, was adopted in its persistent form, which had been registered for information only (its accuracy interval includes zero; /methodology/adjustments).
Evidence: /accuracy,
experiments/house_rater_blend_result.md,experiments/shock_shift_result.md
October 6, 2026
- House: district poll update (adjustment register, test record). Pre-registered (scripts/test_district_polls_v2.py, committed before running): better lean MAE, Brier and seat error than the model alone in 2020, 2022 and 2024, polls as of four weeks out. A review by a separate AI model session on 2026-10-06 found the district poll update sound but flagged the age of the polls, calibration over the final thirty days and uncertainty in the sponsor correction; the review's 2026 variants give 235.7 to 241.5 Democratic seats, against 237.7 published.
Evidence:
scripts/test_district_polls_v2.py,experiments/district_polls_v2_result.json,experiments/competitive_brier_2026-10-06.json,experiments/review_house_changes_2026-10-06.md,experiments/district_polls_review.md
October 5, 2026
- Campaign- and party-sponsored polls (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 6 of 8 past election years, but it worked the other way in presidential years, lopsided races; four weeks out, forecasts with it were better on average and better in 6 of 8 past election years, but it worked the other way in midterm years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Chamber simulation: correlated errors (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Acts on chamber totals, not race forecasts: one chamber outcome per cycle gives five to ten scored points, too few for the per-cycle rule. Race-level numbers are unaffected.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Design effect on poll sampling error (standard test of every adjustment). Kept under an amended rule (2026-10-05). It changes how wide the ranges are, not the central forecast, and on 2026-10-05, after these results were seen, the rule for such terms was changed to judge them by whether their ranges hold the right share of results: on the eve of the election its 90% ranges held 90.3% of results (90.6% without it); four weeks out its 90% ranges held 89.8% of results (90.1% without it). Under the original rule, which scores win probabilities, it would have been flagged: on the eve of the election, forecasts with it were worse on average and better in 3 of 10 past election years, but it worked the other way in midterm years; four weeks out, forecasts with it were worse on average and better in 3 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Drift to Election Day (standard test of every adjustment). Kept under an amended rule (2026-10-05). It changes how wide the ranges are, not the central forecast, and on 2026-10-05, after these results were seen, the rule for such terms was changed to judge them by whether their ranges hold the right share of results: four weeks out its 90% ranges held 90.0% of results (86.4% without it). Under the original rule, which scores win probabilities, it would have been flagged: four weeks out, forecasts with it were worse on average and better in 3 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Four polls to fit a trend (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): A threshold with no 'off' state: below three polls the latent model cannot be fitted at all. Comparing 3 against 4 is a specification change under policy (b).
Evidence:
experiments/adjustment_audit_2026-10-05.md - Fundamentals prior on the race's starting level (standard test of every adjustment). Passed the standard test as registered (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 8 of 10 past election years; four weeks out, forecasts with it were better on average and better in 8 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Generic-ballot bias correction (adjustment register, challenge). Challenged 2026-10-05 (pre-registered at b5a423d; `experiments/mean_bias_challengers_2026-10-05.md`): no correction (H1) and a same-kind correction (H2) **not promoted**. The ledger leaves only 2016 citable (2018-2024 are burned), so neither could be promoted, and on 2016 both have worse seat error. The midterm-only generic-ballot miss (2.70, seven midterms) is the same size as the pooled one (2.62), so a same-kind correction leaves 2026 where it is.
Evidence:
experiments/national_environment.md,experiments/house_generic_ballot_bias.md (refused: the same kind of correction, tested in the retired house_prior/1.7.0 and its corpus, not in N3),experiments/preregistration_generic_ballot_bias.md,experiments/mean_bias_challengers_2026-10-05.md (the challengers to it, no correction and a same-kind one: not promoted) - Generic-ballot bias correction (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 2 of 5 past election years, but it worked the other way in close races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Ground-up blend (the published Senate and governor forecast) (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 5 of 5 past election years; four weeks out, forecasts with it were better on average and better in 3 of 5 past election years, but it worked the other way in governor races, lopsided races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Heavy-tailed election-day errors (standard test of every adjustment). Passed the standard test as registered (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 8 of 10 past election years; four weeks out, forecasts with it were better on average and better in 8 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: district error by race type (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 2 of 5 past election years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: district poll update (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Not in the published path.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: district response to the national swing (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 2 of 3 past election years, but it worked the other way in close races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: fundamentals in the national environment (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 2 of 5 past election years, but it worked the other way in presidential years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: national uncertainty (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 0 of 5 past election years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: seats without a two-party contest (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): These seats get no district forecast; the walk-forward gate assigned them from actual winners (an oracle), so there is no historical 'on' arm to score. Note: the live rule gives LA-2, a Democratic seat, to the Republican because neither party's candidate quality is known (a correctness check under policy (a)).
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: shared state error (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 0 of 5 past election years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - House: the corrected national environment (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 2 of 5 past election years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Likely-voter adjustment (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Nothing is applied, so there is nothing to switch off; 7,242 of 11,258 historical statewide polls record no population at all, so an LV/RV term cannot be fitted walk-forward from this corpus. Adding one would be a new specification under policy (b).
Evidence:
experiments/adjustment_audit_2026-10-05.md - Mean polling-bias correction (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Not in the published path (rejected); its question is answered by scale_intercept.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Measured error of a sparse race's poll average (standard test of every adjustment). Kept under an amended rule (2026-10-05). It changes how wide the ranges are, not the central forecast, and on 2026-10-05, after these results were seen, the rule for such terms was changed to judge them by whether their ranges hold the right share of results: on the eve of the election its 90% ranges held 90.5% of results (89.1% without it); four weeks out its 90% ranges held 90.0% of results (88.9% without it). Under the original rule, which scores win probabilities, it would have been flagged: on the eve of the election, forecasts with it were worse on average and better in 0 of 3 past election years; four weeks out, forecasts with it were worse on average and better in 0 of 3 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Pollster house effects (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 4 of 10 past election years; four weeks out, forecasts with it were better on average and better in 5 of 10 past election years, but it worked the other way in presidential years, governor races, close races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Race-specific error (standard test of every adjustment). Kept under an amended rule (2026-10-05). It changes how wide the ranges are, not the central forecast, and on 2026-10-05, after these results were seen, the rule for such terms was changed to judge them by whether their ranges hold the right share of results: on the eve of the election its 90% ranges held 90.5% of results (60.7% without it); four weeks out its 90% ranges held 90.0% of results (80.2% without it). Under the original rule, which scores win probabilities, it would have been flagged: on the eve of the election, forecasts with it were better on average and better in 5 of 10 past election years, but it worked the other way in lopsided races; four weeks out, forecasts with it were worse on average and better in 3 of 10 past election years, but it worked the other way in presidential years, close races.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Range widening and fading shifts after major events (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Not in the published path (switched off after failing their own pre-registered tests).
Evidence:
experiments/adjustment_audit_2026-10-05.md - Scale law (reference entry) (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years, lopsided races; four weeks out, forecasts with it were better on average and better in 6 of 10 past election years, but it worked the other way in midterm years. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Scale law: intercept (adjustment register, challenge). Challenged 2026-10-05 (pre-registered at b5a423d; `experiments/mean_bias_challengers_2026-10-05.md`): slope only (S1) and a same-kind intercept (S2) **both not promoted** on the citable folds 2006-2016. The champion's intercept stays.
Evidence:
experiments/preregistration_scale.md,experiments/scale_intercept_check_2026-10-05.md,experiments/mean_bias_challengers_2026-10-05.md - Scale law: intercept (standard test of every adjustment). Flagged for review (2026-10-05): on the eve of the election, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years; four weeks out, forecasts with it were worse on average and better in 6 of 10 past election years, but it worked the other way in presidential years, Senate races. It stays in the model unless a pre-registered test decides otherwise.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Scale law: slope (standard test of every adjustment). Passed the standard test as registered (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 8 of 10 past election years; four weeks out, forecasts with it were better on average and better in 7 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Shared national polling error (standard test of every adjustment). Kept under an amended rule (2026-10-05). It changes how wide the ranges are, not the central forecast, and on 2026-10-05, after these results were seen, the rule for such terms was changed to judge them by whether their ranges hold the right share of results: on the eve of the election its 90% ranges held 90.5% of results (89.1% without it); four weeks out its 90% ranges held 90.0% of results (89.4% without it). Under the original rule, which scores win probabilities, it would have been flagged: on the eve of the election, forecasts with it were worse on average and better in 2 of 10 past election years; four weeks out, forecasts with it were worse on average and better in 3 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Sparse-race blend (standard test of every adjustment). Passed the standard test as registered (2026-10-05): on the eve of the election, forecasts with it were better on average and better in 8 of 8 past election years; four weeks out, forecasts with it were better on average and better in 10 of 10 past election years.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Unpolled races published from the ground-up prior (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): Switching it off leaves the race unpublished, not forecast differently: there is no 'off' forecast to score. The prior's own accuracy is the evidence.
Evidence:
experiments/adjustment_audit_2026-10-05.md - Which polls a forecast reads (adjustment register, test record). Leakage tests (canary poll after the as_of must change nothing); a data rule, not a model adjustment.
Evidence:
docs/BACKTESTING.md,tests/leakage/test_asof_canary.py,tests/leakage/test_release_basis.py - Which polls a forecast reads (standard test of every adjustment). Cannot be tested on past elections (2026-10-05): A knowability rule, not an adjustment: switching it off is leakage, not an alternative forecast. Guarded by the leakage tests.
Evidence:
experiments/adjustment_audit_2026-10-05.md
October 4, 2026
- Older statewide record (/accuracy, Measured on an earlier model's ten-cycle record). The items below come from the record of an earlier version of the statewide model, single_race_latent/0.7.0, scored over 435 races in ten cycles (2006–2024). They have not been re-measured on the current model. The partisan gradient correction stays uncorrected and marked suggestive_only until 2026, its declared confirmatory cycle.
Evidence:
docs/STATISTICAL_MODEL.md - Range widening and fading shifts after major events (adjustment register, description). Widening a race's range after a confirmed severe event, and a centre shift that fades as new polls arrive. Built, registered, failed their tests, switched off. The scandal shift that is live for Senate and governor races is a separate entry (statewide_scandal_rule).
Evidence:
experiments/shock_uncertainty_result.md,experiments/shock_shift_result.md - Range widening and fading shifts after major events (adjustment register, test record). Pre-registered; both failed.
Evidence:
experiments/shock_uncertainty_result.md,experiments/shock_shift_result.md
October 2, 2026
- Unpolled races published from the ground-up prior (adjustment register, test record). No gate; justified by the prior's own error (MAE 8.3 on 2016-2022, 6.7 in 2024) and a tail check (0 of 87 seats favoured by 20+ flipped).
Evidence:
docs/FAILURE_MODES.md FM-140,experiments/unpolled_seat_tails.md
October 1, 2026
- Chamber simulation: correlated errors (adjustment register, test record). Chamber gate (pre-registered).
Evidence:
experiments/chamber_gate_preregistration.md,experiments/chamber_gate_result.md - Generic-ballot bias correction (adjustment register, test record). Never isolated: national_environment.md (N2 MAE 2.44 vs 3.01) ran the same day house_generic_ballot_bias.md refused the same correction for the old House model (environment MAE 2.72 vs 2.73).
Evidence:
experiments/national_environment.md,experiments/house_generic_ballot_bias.md (refused: the same kind of correction, tested in the retired house_prior/1.7.0 and its corpus, not in N3),experiments/preregistration_generic_ballot_bias.md,experiments/mean_bias_challengers_2026-10-05.md (the challengers to it, no correction and a same-kind one: not promoted) - Ground-up blend (the published Senate and governor forecast) (adjustment register, description). The election-day centre of every Democrat-vs-other race is averaged with a ground-up prediction (a gradient-boosted model of the state's lean from its results, demographics, economy, candidates and money, plus the national House environment N3), weighted by each one's variance; the spread is kept and the win probability is read from that same moved distribution (0.8.2 onward; 0.8.1 recomputed it from a normal curve the band did not follow, audit M09). Since 0.8.3 the blend first takes the scale law's intercept back out of the poll model's forecast.
Evidence:
experiments/statewide_gate_preregistration.md,experiments/statewide_gate_result.md - Ground-up blend (the published Senate and governor forecast) (adjustment register, test record). Pre-registered gate, selection 2016-2022, confirmation 2024; Brier and MAE (not log loss): Brier 0.0677 -> 0.0585, MAE 5.60 -> 5.38 on 175 races; most of the gain is 2020.
Evidence:
experiments/statewide_gate_preregistration.md,experiments/statewide_gate_result.md - House model (district pages, How was this number made?). By the ground-up House model, promoted to the published forecast by a pre-registered test; its tested record is on the Accuracy page. (The page also said "with an independent review"; that claim is withdrawn as unsupported.)
Evidence:
experiments/district_gate_preregistration.md,experiments/district_gate_result.md - House: district error by race type (adjustment register, test record). Part of the district gate package; not isolated.
Evidence:
experiments/district_gate_preregistration.md - House: district response to the national swing (adjustment register, test record). Part of the district gate package; not isolated.
Evidence:
scripts/backtest_district_model.py (pre-registered docstring),experiments/district_gate_result.md - House: fundamentals in the national environment (adjustment register, test record). Walk-forward national MAE 2004-2024 (N2 2.44, N3 2.16); not isolated at the district level.
Evidence:
experiments/national_environment.md - House: national uncertainty (adjustment register, test record). Part of the district gate package; not isolated.
Evidence:
experiments/district_gate_preregistration.md,experiments/audit_m12_result.md (the live spread adds the movement still to come and today's measurement excess, house_ground_up 2.5.0; FM-195) - House: the corrected national environment (adjustment register, test record). National: walk-forward MAE 2004-2024, N0 3.01, N3 2.16. District gate tested N3 only as part of a package (lean model, national term and errors changed together).
Evidence:
experiments/national_environment.md,experiments/district_gate_result.md
September 29, 2026
- Mean polling-bias correction (adjustment register, test record). mean_bias_correction.md (2026-09-23): out-of-sample |error| 3.22 -> 3.09, judged noise. c8_bias_centre (2026-09-25): rejected three times on log loss. c19 recency centre (2026-09-29): not promoted.
Evidence:
experiments/mean_bias_correction.md,experiments/bias_centre.md,experiments/preregistration_bias_centre.md,experiments/preregistration_recency_centre.md (c19, a later re-registration of a recency mean: suggestive only on earlier cycles, not adopted; 2026 is its confirmatory fold)
September 27, 2026
- Inverse-variance poll weights in the national environment (docs/STATISTICAL_MODEL.md §11.1.2). times — was registered as `c11_inverse_variance_weights` and is **applied as of 2026-09-27**
(`house_prior/0.9.0`). Promoted on its pre-registered rule: a firm-clustered 95% interval of
[+0.063, +0.329] pts² on 3,765 held-out polls over 127 firms, excluding zero, reproducing the
[+0.059, +0.304] measured at registration. The metric is leave-one-poll-out against *other firms'*
polls in a 21-day window, so it reads no election outcome and consumed no holdout under ADR-0010.
`experiments/preregistration_inverse_variance.md` has the decision; the environment moved by about a
third of a House seat, which is what the registration predicted and is not the argument for it.
Evidence:
experiments/preregistration_inverse_variance.md,docs/DECISIONS.md ADR-0010 - Model status and scored record (docs/STATISTICAL_MODEL.md introduction). Status, 2026-09-27: **fitted, and validated where it says so.** This line read "design
specification, not yet fitted. Nothing here is validated" until today, which was written before
anything was fitted and left standing for months afterwards — so the project's primary statistical
document asserted the opposite of the truth. The model has been fitted, walk-forward backtested and
scored on **435 races over ten cycles, 2006-2024** (`single_race_latent/0.7.0`: MAE 4.89, lean +2.08,
realised coverage 0.471 / 0.798 / 0.903 against a nominal 0.50 / 0.80 / 0.90 -- all three inside the
ADR-0018 floor), and two challengers have been promoted through a pre-registered gate. On those same
races a recency- and size-weighted polling average scores MAE 5.51 with a lean of +2.86, so the model
beats its baseline and removes 0.77 points of the polls' lean rather than adding to it
(F-11, `experiments/calibration_decomposition.md`).
The five-cycle figures this paragraph used to quote (295 races, MAE 4.68, coverage
0.475 / 0.810 / 0.919) were for `0.6.0` before the evidence base was extended to 2006 and before
FM-104's fix; they are not comparable to the ten-cycle ones and are not the same population.
What remains genuinely unvalidated is named where it arises, not hedged here. The cross-race posterior
predictive check now passes on all six cycles, 2014 at tail p = 0.089 where it failed at 0.037
(`experiments/phase5_validation_2026-09-29.json`, `experiments/phase5_exit_met.md`) -- a pass, and the
tightest of the six against a next-closest 0.110. `sigma_region` fits at 0.00, and that is now a bounded claim rather than a shrug:
Evidence:
docs/DECISIONS.md ADR-0018,experiments/calibration_decomposition.md (F-11),docs/FAILURE_MODES.md FM-104,experiments/phase5_validation_2026-09-29.json,experiments/phase5_exit_met.md
September 26, 2026
- Race-specific residual error (sigma_race) (docs/STATISTICAL_MODEL.md §11). **Both halves of the variance-components model, and this is where the model's largest defect lived.**
§10 decomposes the error of a polling average into `mu_c ~ N(0, σ²_nat)`, one draw per cycle shared by
every race in it, and `eps_{r,c} ~ N(0, σ²_race)`, the race-specific residual. On thirteen cycles
σ_nat is 2.48 points and **σ_race is 5.03** — the residual is the larger term. Until 2026-09-26 the
race-level budget carried σ²_nat and, where σ²_race belonged, three unfitted placeholders totalling
1.89 points² against the 25.3 the fitted term carries (`σ²_lv` 1.00, `σ²_und` 0.64, `σ²_model` 0.25).
The chamber simulator had been using σ_race all along.
Restoring it, walk-forward, took realised coverage on 247 scored races from 0.259 / 0.518 / 0.680 to
**0.474 / 0.777 / 0.895** against nominal 0.50 / 0.80 / 0.90, with nothing tuned (FM-79). The three
placeholders are **replaced** rather than joined, because a residual already contains the likely-voter,
undecided and specification error they stood for; carrying both over-widens. The budget therefore
contains no unfitted term, which is why `allow_guesses` no longer appears on the publishing path — the
refusal in §11's own implementation now guards the published forecast instead of being waived by every
caller.
The independent check that σ_race is the right term rather than a well-sized fudge: realised error with
each cycle's mean removed — which is what `eps` is — has an rms of 5.42 against a fitted 5.03.
Evidence:
docs/FAILURE_MODES.md FM-79 - Scale law (reference entry) (adjustment register, test record). Pre-registered, walk-forward 2016-2024, CRPS: -0.530 against threshold 0.073, 4 of 5 folds, log loss -0.0088. Promoted as 0.6.0.
Evidence:
experiments/preregistration_scale.md,experiments/scale_compression.md,experiments/scale_calibration_2016.md,experiments/scale_intercept_check_2026-10-05.md
September 25, 2026
- Chamber drift term (docs/STATISTICAL_MODEL.md §11.1). A chamber forecast built from priors rather than from race-level polling has a different structure and, until 2026-09-25, was missing this term entirely.
- House national drift (docs/STATISTICAL_MODEL.md §11.1). What was absent was any term for the movement of that environment between the forecast date and Election Day: the district spread is an election-to-election quantity, and σ_nat is the cycle-level polling error, so a forecast made 41 days out carried the same national uncertainty as one made on the morning of the election. That is why P(Democratic majority) printed as 0.995.
- Measured error of a sparse race's poll average (adjustment register, test record). Shipped as a defect fix: 67 sparse races, MAE 8.01 -> 6.34, 90% coverage 0.791 -> 0.925. As challenger c9 (sparse_variance.md, 2026-09-25) the same change was refused on log loss (+0.0150, 0 of 3 folds).
Evidence:
experiments/sparse_blend_poll_weight.md,docs/FAILURE_MODES.md FM-104,experiments/sparse_variance.md (the first registration, rejected on its metric),experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment),experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026) - Precision of today's national environment (docs/STATISTICAL_MODEL.md §11.1). come; it says nothing about the precision of the starting point. Until 2026-09-25 the budget treated
a window of 9 polls from 5 firms as exactly as precise as one of 165 polls from 43 — the estimate's
own error appeared nowhere. It is now added in quadrature as `max` of two figures, neither of which
bounds the other (FM-74, FM-75, FM-76):
Evidence:
docs/FAILURE_MODES.md FM-74,docs/FAILURE_MODES.md FM-75,docs/FAILURE_MODES.md FM-76 - Precision of today's national environment (docs/STATISTICAL_MODEL.md §11.1). The term is what makes an early, thinly polled or concentrated environment print a wider chamber interval than a late, broad one: for 2026 it took the House seat sd from 16.5 to 18.0 and P(D majority) from 0.994 to 0.991.
September 24, 2026
- Pollster house effects (adjustment register, test record). Never gated itself; the pooled cross-race alternative was tested (log loss, 11 cycles) and rejected (+0.0013, 5 of 11 cycles).
Evidence:
docs/STATISTICAL_MODEL.md section 5,docs/FAILURE_MODES.md FM-13,experiments/house_effects.md,experiments/batch03_pooled_house.md (house effects pooled across races: rejected)
September 23, 2026
- Four polls to fit a trend (adjustment register, test record). k=3 (c6) tested: passed (log loss -0.0395) and superseded by c7; the value 4 itself was never tested.
Evidence:
experiments/preregistration_min_polls.md,experiments/prior_only_fallback_ignores_sparse_polls.md - Sparse-race blend (adjustment register, test record). Pre-registered, confirmation cycles 2018-2024, log loss, BH q <= 0.10: -0.0474, q=0.0002, 4 of 4 cycles. Promoted.
Evidence:
experiments/preregistration_sparse_blend.md,experiments/prior_only_fallback_ignores_sparse_polls.md,experiments/sparse_prior_once_preregistration.md,experiments/sparse_prior_once_result.md (the prior counted once: passed its rule, held: the sparse races' 90% coverage fell; FM-253)
September 22, 2026
- Compute budget (docs/BACKTESTING.md §3.1). Compute budget (added after adversarial review, AR-05). [...] With several specifications and an automated challenger generator, the design as first written cannot run.
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-05 - Leakage tests (docs/BACKTESTING.md §2.2). Revised after adversarial review (AR-09). An earlier draft asserted that the visible row set at T1 is a subset of that at T2. That is false by design: a row superseded between T1 and T2 is visible at T1 and invisible at T2. The original test would have failed against a correct implementation and would then most likely have been "fixed" by weakening the bitemporal logic — which is how a real guarantee gets quietly deleted. The two assertions above are both true and both catch the leak the original was aiming at.
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-09 - Promotion gate: multiplicity (docs/BACKTESTING.md §7). Multiplicity is controlled across the challenger batch (added after adversarial review, AR-11). Challengers are pre-registered in batches before evaluation; Benjamini–Hochberg FDR control is applied across the batch at a target FDR set in configuration. Per-comparison error control, which the first draft specified, guarantees a steady trickle of false promotions once a generator is producing challengers continuously — the null-challenger test (FM-32) would have detected the symptom while nothing in the design prevented the cause.
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-11,docs/FAILURE_MODES.md FM-32 - Race elasticity identification (docs/STATISTICAL_MODEL.md §3). Identification (revised after adversarial review, AR-03).
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-03 - Reference category for log-ratios (docs/STATISTICAL_MODEL.md §2). Choice of reference category (revised after adversarial review, AR-10).
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-10 - What the nowcast estimates (docs/STATISTICAL_MODEL.md §11.0). What the nowcast actually estimates (revised after adversarial review, AR-01).
Evidence:
docs/ADVERSARIAL_REVIEW.md AR-01
Undated
- Effective sample size for the drift correction (docs/STATISTICAL_MODEL.md §11.1). Which effective sample size is a real choice, because the estimate has two (§11.1.1) and the original code used a third quantity that was neither (FM-73).
Evidence:
docs/FAILURE_MODES.md FM-73 - Electoral-college simulation (docs/STATISTICAL_MODEL.md §11.2). It does not draw the election-to-election national swing on top of states whose margins have already been measured from polls — which both earlier implementations did, with the effect that a 2024 forecast came out P(D wins) = 0.585 with a 90% interval 140 electors wide in a year decided by a point and a half (FM-66).
Evidence:
docs/FAILURE_MODES.md FM-66 - Heavy-tailed election-day errors (adjustment register, test record). Never tested; a constant.
Evidence:
docs/STATISTICAL_MODEL.md section 10,docs/FAILURE_MODES.md FM-12 - House: seats without a two-party contest (adjustment register, test record). Never tested; the gate used actual winners for these seats instead.
Evidence:
scripts/forecast_house_ground_up.py,docs/FAILURE_MODES.md FM-155, FM-173 - House: seats without a two-party contest (adjustment register, why it could not be tested). These seats get no district forecast; the walk-forward gate assigned them from actual winners (an oracle), so there is no historical 'on' arm to score. Correctness under policy (a): the old rule made a seat with no nominee on record a Republican seat (LA-2 before FM-155 repaired Louisiana's rosters); fixed by FM-173, no effect on today's House (no such seat), +0.99 seats on the pre-FM-155 data.
Evidence:
scripts/forecast_house_ground_up.py,docs/FAILURE_MODES.md FM-155, FM-173 - Multivariate-t errors (docs/STATISTICAL_MODEL.md §0 table). Multivariate-t errors, ν estimated in [4, 20] (§10): implemented (normalised mixture since FM-189).
Evidence:
docs/FAILURE_MODES.md FM-189 - Precision of today's national environment (docs/STATISTICAL_MODEL.md §11.1). 1e4/independent_n — the arithmetic this term started as — is now a printed diagnostic only.