The adjustment register and the standard test
The standard test every adjustment faces, its amendments and their dates, and the register of every adjustment with its test record.
Model adjustments: the registry, the standard test, and the rules for changing them
Every published number on futureballot.com is raw polls and results passed through a chain of
transformations: house effects, a prior, a variance budget, a scale correction, a blend with the
ground-up model. Until 2026-10-05 each of those was promoted on its own test, once, in isolation,
and two of them could disagree without anyone noticing. experiments/mean_bias_correction.md
concluded that no mean polling-bias correction should be applied; the promoted scale law
(experiments/scale_pre2026.json, final = 1.070 x poll - 1.744) applies a uniform shift of about
-1.74 points toward the Republican. Both were in force.
This document is the single place where that is controlled. It has four parts:
- Policy — when a specification may change, and what a challenge to one race's number triggers.
- The standard test — the one test every adjustment faces, fixed before any result was computed.
- The registry — every adjustment between raw inputs and the published number (machine-readable
copy:
data/model/adjustments.json). - The latest audit — what the standard test said about each one.
The test is implemented in src/elab/evaluate/adjustment_audit.py and run by
scripts/audit_adjustments.py; the scheduler re-runs it weekly and after each certified election.
1. Policy
(a) A challenge to one race's number is a check, never a specification change. When anyone — a reader, a reviewer, an agent, the owner — questions a single race's forecast, the response is to verify that every registered adjustment was applied to that race correctly: the right polls read as of the right time, the right walk-forward parameters, the right orientation, each step as the registry describes it. A defect found that way is a bug and is fixed (see (c)). The finding "this race's number looks wrong" is never, by itself, a reason to add, remove or retune an adjustment: an adjustment applies to every race or to none, and its merit is decided only by (b).
(b) Specification changes only through a pre-registered test that meets the standard. Adding an
adjustment, removing one, or changing how one is computed requires a pre-registration (in
experiments/, committed before results) and a result under the standard test in section 2, run on
the whole corpus. A flag from the audit is a recommendation to the coordinator, who decides through
the existing pre-registration and gate process; the audit itself changes nothing. Exceptions: a
fix (stateable without reference to any score, ADR-0017) does not face the standard test. A
verified bug is fixed at once ((c), section 5); a fix that changes the published model may instead
be registered as a consistency correction, adopted unless its log loss is worse than the control's
by more than 0.002, its Brier by more than 0.0005, or its 80%/90% coverage further from nominal by
more than 0.02 (experiments/detide_prior_preregistration.md). And the owner may amend a
registration to adopt a change its rule did not, written into the registration's record, as for
the House rater blend (experiments/house_rater_blend_result.json, adopted_by_amendment) and the
sparse blend's shared error (experiments/sparse_shared_once_result.md).
(c) No specification freeze. A freeze from 2026-10-13 to certification was written here on
2026-10-05; the owner withdrew it on 2026-10-12: the date was arbitrary, and a model change that
improves the forecast is made whenever it is found, through (b) like any other, up to election
day. Every change gets a new version, its date and its evidence; /changes shows it as a change of
method, and the November scoring scores whatever was published last before election day. A fix
for a verified bug (code that does not do what the registry says it does, or reads data it
should not) is made at once under ADR-0017 and recorded in section 5 (date, what was wrong, how it
was verified, the commit, the effect on published numbers).
2. The standard test
Fixed on 2026-10-05, before any audit result was computed, and committed separately from the results. Changing anything in this section is itself a specification change under (b).
2.1 What is compared
For each adjustment, two forecasts of the same race from the same standpoint with the same random seed:
- on — the full published specification, exactly as the production path computes it
(
scripts/publish_forecasts.py->elab.backtest.pipeline.forecast_race_at->scripts/apply_statewide_prior.pyfor Senate and governor;scripts/forecast_house_ground_up.pyfor the House); - off — the same specification with that one step replaced by identity (or, for a step that substitutes one estimate for another, by what it replaced). Every other step is identical.
Where an adjustment cannot act on a race (the sparse blend on a race with enough polls to fit), the off forecast equals the on forecast. Where the off forecast cannot be produced (its fit fails the convergence gate), the race is dropped from that adjustment's comparison and counted.
2.2 Walk-forward by cycle
A forecast of cycle T uses only what was knowable before T:
- parameters fitted across cycles are taken from files fitted on cycles strictly before T
(
experiments/scale_pre<T>.json,experiments/uncertainty_terms_pre<T>.json, the sparse-poll error pooled over cycles < T, the statewide ground-up prior as its gate fitted it for T, the House model's lean, error model and national term trained on cycles < T); the harness refuses any file whose recordedused_cyclesinclude T or later; - polls are read through
ObservationRepository.for_race(as_of=standpoint), the bitemporal reader every walk-forward script uses, so nothing recorded, published or released after the standpoint is seen; results enter the fundamentals prior only if knowable at the standpoint; - the realised result is attached for scoring only and is read by no forecast step (a leakage test changes it and checks every arm is unchanged).
2.3 Corpus and standpoints
- Senate and governor: every race in the stored walk-forward record of the published
specification (
single_race_latent/0.7.0election-day backtests, cycles 2006-2024, general and special), at two standpoints: eve (election day, the stored record's own standpoint) and h28 (28 days before, close to the published forecasts' horizon in October). A race whose published path produces no forecast at a standpoint (no polls, or the fit fails its gate) is not in that standpoint's corpus. Races are scored against the certified margin with the refusals ofelab.evaluate.scored: the two-way margin in the 2026-10-05 audit, D% - R% of all votes cast from 2026-10-12 (audit M07). Refused: a third candidate won, or the two took under 80%. - House: every contested two-party district in 2016-2024 (the cycles with walk-forward error residuals behind them), one standpoint (the national term as known before the election).
2.4 Scores
Per race: absolute margin error; log loss of the win probability (clipped to [0.001, 0.999], so one confident miss contributes at most 6.9); Brier score; whether the result fell inside the 80% and 90% intervals. Reported pooled and for every cycle separately, with the number of races per cycle, so a verdict resting on few cycles is visible as such.
2.5 The pass rule
Gain = log loss without the adjustment minus log loss with it (positive means the adjustment helps). At each standpoint where the adjustment changes at least one forecast, it passes if all three hold:
- pooled gain over all races is > 0;
- the mean gain is > 0 in a strict majority of the scored cycles (cycles in which it changes at least one forecast);
- its effect does not reverse sign in any major subgroup that spans at least 3 scored cycles. The major subgroups are: midterm vs presidential cycles; Senate vs governor; competitive (standpoint polling average within 8 points: the mean of the latest five usable polls, or the prior mean for a race without polls; for the House, the on-arm central margin) vs not. A reversal is a subgroup whose pooled gain has the opposite sign from the overall gain.
An adjustment is kept if it passes at every standpoint where it changes anything, flagged for review otherwise, and untestable if the corpus cannot test it (each such case is listed with the reason). MAE, Brier and coverage are reported beside the verdict and do not decide it.
For an entry that records a choice rather than a correction (sponsored polls kept unadjusted), the comparison is the published choice (on) against the alternative named in its registry entry (off), under the same rule.
2.5a Amendment 1: spread-only terms (2026-10-05, after results)
Disclosure. This amendment was adopted on 2026-10-05 after the first audit's results were seen, at the project owner's explicit direction. It is not part of the rule committed in 7f7741d.
Why. The rule in 2.5 judges every adjustment on log loss. For a term that changes only a forecast's spread (never its centre), log loss rewards narrower forecasts whenever the scored cycles hold few upsets, so it would remove terms whose intervals are correctly calibrated. ADR-0016 already requires spread changes to be judged on coverage (and CRPS); the standard as first written contradicted it.
Rule. For the terms in SPREAD_ONLY (src/elab/evaluate/adjustment_audit.py: design effect,
measured sparse-poll error, sigma_nat, sigma_race, drift to election day, and the House national,
state and race-type error terms), at each standpoint where the term changes a forecast, it passes if
(1) with it, pooled 80% and 90% interval coverage are each within 0.03 of nominal, and (2) switching
it off does not bring total coverage error (|cov80 - 0.80| + |cov90 - 0.90|) closer to nominal by
more than 0.01. Per-cycle majority and subgroup tests are not applied to these terms: with 6 to 68
races a cycle, a single cycle's coverage has a standard error of 4 to 12 points. CRPS is not used
because the stored summaries hold quantiles, not full distributions; adding it is listed as work.
The log-loss verdict is still computed and stored beside it.
Result on 2026-10-05 (re-scored from the stored audit, no new fits; the 2026-10-12 audit
flags the Senate and governor spread terms, see 4.1;
experiments/spread_amendment1_rescore_2026-10-05.json):
| Term | Verdict | 80%/90% coverage with it | without it |
|---|---|---|---|
| sigma_nat | keep | 0.793/0.905 (eve), 0.816/0.900 (4 weeks) | 0.769/0.891, 0.791/0.894 |
| sigma_race | keep | same | 0.491/0.607, 0.674/0.802 |
| drift to election day | keep | 0.816/0.900 (4 weeks) | 0.747/0.864 |
| design effect | keep | 0.790/0.903, 0.815/0.898 | 0.790/0.906, 0.812/0.901 |
| measured sparse-poll error | keep | 0.793/0.905, 0.816/0.900 | 0.775/0.891, 0.805/0.889 |
| House national, state, race-type error | flag | 0.938/0.974 (eve) | 0.92-0.93/0.96-0.97 |
The House terms fail condition (1): House intervals are too wide (90% intervals hold 97.4% of outcomes). Removing any one term does not fix that, so the remedy is a recalibration of the district error model under its own pre-registration, not deleting a term.
2.5b Amendment 2: the House rater blend (2026-10-08, after results)
Disclosure. Adopted on 2026-10-08, after the result was seen, at the project owner's explicit direction.
What changed. experiments/house_rater_blend_preregistration.md required a challenger to beat the champion on both Brier and seat error.
The rater blend failed on seat error alone:
- Seat error: 11.7 against 11.2, all from 2022, when the raters overestimated Republicans.
- Brier: better in every rated cycle but 2022, and better in the 2024 confirmation.
The owner judged the district probabilities readers act on to outweigh half a seat of average seat-total error, and the blend was adopted as house_ground_up 2.3.0. The result file records the decision (adopted_by_amendment), and the House run refuses to publish without that record.
Re-run (2026-10-09). The history the test first ran on was read from today's pages with a parser that dropped most raters (review C1, M5). On the history rebuilt from election-eve revisions the result was the same in kind: Brier better in 2018, 2020 and 2024 (worse in 2022), seat error 11.76 against 11.18, all from 2022. A race-level test of whether the raters drift toward the party paying for the polls, which could have justified a 2022 modifier, was not supported (experiments/rater_sponsor_drift_result.json). The owner re-confirmed the amendment.
Caveat on the House rater blend (review by a separate AI model session, 2026-10-08). The blend's r(c) and w were fitted on election-eve ratings, mostly from Cook, Sabato and Inside Elections. The 2026 input is the raters' current view about four weeks out. It includes model-based raters (DDHQ, Silver Bulletin, the Economist) that are rare in the history, and some ratings that have not been updated in months. The fitted weight is therefore applied to a somewhat different quantity than it was fitted on. /accuracy states this.
2.5c Amendment 3: the Alaska ranked-choice forecasts (2026-10-08, after results)
Disclosure. Adopted on 2026-10-08 at the project owner's direction that no race be left without a forecast. The method is untested.
What happened. experiments/alaska_rcv/preregistration.md §5 said a race stays withheld if the transfer-rate check fails. The check failed: other contests' transfer rates missed the 2022–2024 final rounds by 4.48 points on average (limit 4.0), and they called the 2022 special election's winner wrong (experiments/alaska_rcv/result.json).
What is published instead.
- Alaska governor (
ranked_choice_final_round/1.0.0): the average of the pollsters' published final rounds. Its ranking error, 5.05 points, is the RMS miss of the failed transfer-rate method. That is borrowed, not a measured error of pollsters' final rounds, which have no record against the count. - AK-AL (own polls): the transfer rates from the same failed check, widened by the same 5.05 points.
Both race pages say the method has not been tested on past races and that the ranking error is borrowed.
2.5d House 2.4.0 changes and safeguards (2026-10-09)
- CA-6 moves with the national draw. A Democrat-against-other seat forecast from its own polls (CA-6) now shares the simulation's national swing, with the rest of its spread drawn on its own. Its own chance is unchanged; only the tails of the seat total widen slightly. AK-AL (an independent against a Republican) is still drawn on its own.
- Declared own-poll seats are never one-party. CA-6 was counted as a certain Democratic seat
by one scheduled run after the district panel stopped recording a Republican for it; the run
was withdrawn, the forecast now refuses that state, and
scripts/audit_accounting.pychecks it. - One question per poll in an own-poll window.
2.5e Pollster sponsor pooling, tested and rejected (2026-10-10)
Combined pollster records that pair a sponsor with the firm that fielded the poll ("NBC
News/Marist", "CNN/SSRS") were tested as the fielding firm, so their house effects pool with the
firm's own polls (experiments/sponsor_pooling_preregistration.md, committed before running).
The standard test refused it: log loss slightly worse at both standpoints (-0.0002 at the eve,
-0.0011 four weeks out), better in 1 of 6 cycles four weeks out, and the effect reversed sign for
the Senate. The House check was identical to the published arm. The records stay separate; no
forecast in the corpus moved by more than 1.8 points either way
(experiments/sponsor_pooling_result.md).
2.5f The House scandal rule (2026-10-22, owner's decision)
House model 2.7.0 moves a district's margin 3.0 points against a nominee after a verified scandal
about them, from the event's first date, with the spread unchanged
(src/elab/models/district/scandal.py). The test (experiments/house_scandal_preregistration.md,
amendments 1-5, written before any number) found 12 scandal candidates in the House record
2018-2024; all 12 finished below the forecast, by 5.4 points on average (3.3 without Chris Collins).
The Senate and governor size, 3.0 points, improved CRPS on those races by -0.77
[-1.31, -0.18], Holm p 0.045, which meets the registered criterion. The registration said support
puts nothing live by itself; the owner decided on that result to put the rule live
(experiments/house_scandal/result.md). The 2026 events are listed by hand under the test's
verification rules, with two sources each, in config/house_scandal_events_2026.yaml; a race
monitor event is a lead, never an entry. Party splits (7 races, mean shortfall 3.6) are
descriptive only and not part of the rule. The 2.6.0 district spread and its record carry forward
unchanged, since the rule moves only the centre of the listed districts. Senate and governor races
take the same rule from single_race_latent 0.8.4 (scripts/apply_statewide_prior.py, list in
config/statewide_scandal_events_2026.yaml), including races forecast from the prior alone.
Whose scandal counts (owner, 2026-10-22). The event must be about the candidate themselves: their own charge, conviction, misconduct or false claim about their own record. Trouble around a running mate, relative, staff member or associate does not count, nor does an investigative step without a charge that reads as possible political retaliation (a search warrant). This is also the only class the test measured. Decisions on that basis: MA governor, running mate Shawn Oliver's OUI charge (2026-10-17), not counted; MN-5 Omar's search warrant, not counted unless she is charged.
2.5g The statewide scandal rule stops at three post-event polls (blend 0.8.7)
For a Senate or governor race the scandal shift is made only while the race has fewer than three
polls fielded after the event's first date (SCANDAL_POLLED_LIMIT in src/elab/models/shock.py).
A poll counts when its fieldwork started after that date, it is a poll of the forecast's two
contenders that the poll model reads, and it stands as of the forecast's own date: the count is
count_post_event over the same polls the off-by-default fading rule reads. Each contender's
event is counted on its own, so one side's shift can stand while the other's is held back. A held
event moves nothing and is stored as scandal_held with its count; a shift that is made is stored
as shock_shift, each event with its post_event_polls, for the race page and the rebuild.
Races forecast from the prior alone are counted the same way. The House rule (2.5f) has no such
limit.
The reason is double counting. Polls fielded after the event already carry most of its effect: on
election eve, a plain average of the post-event polls missed the scandal-hit candidate by 0.5
points more than an ordinary race of the same year and margin, while the model's own number
missed by 2.7, and within a race the shortfall shrank as post-event polls arrived
(experiments/scandal_investigation/; FM-255). A fixed shift on top of three or more such polls
counts the effect twice; below three it stands in for the polls that do not exist yet.
ELAB_SCANDAL_POLLED_LIMIT=0 restores the shift on every listed event and the blend 0.8.6. The
rule is scored prospectively after certification as arm A1 of
experiments/scandal_rule_v2_preregistration.md.
2.5h House 2.12.0: the own-poll seats' spread and one religion census (2026-10-28)
Published 2026-10-28 (promotion_audit afb47e5c, a fix promotion from 2.10.0; 2.11.0 was never stored on its own). Both changes passed their registered tests, and passed again unchanged after the House training rows were rebuilt on the corrected results (FM-254):
- Own-poll seats (
house_own_polls_spread, FM-247): AK-AL and CA-6 state the own-polls centre's measured past error, 8.41 points for 2026, with the House model's t(4) shape, instead of the House model's full error, normal, which covered 98.3% at the stated 90% (experiments/own_polls_shape_rebuilt_panel.md: 90% 0.901, 80% 0.768, log loss 0.4491 -> 0.4175). The test is a disclosed second look at the rows of the first measured-spread test. - Religion census (
religion_vintage, FM-257): the 2010 study for every cycle, so the House lean and the statewide prior train and predict on one instrument; judged as a correction by non-inferiority on 2024 (experiments/religion_vintage_rebuilt_panel.md).
Live on 2026-10-28 (published 01:48 UTC 2026-10-29): House Democratic majority 88%, about 235 seats; CA-6 Pan 87%; AK-AL Begich 70% against Hill.
2.5i Ranked choice 1.2.0: the race model's tail on the final round (2026-10-28)
ranked_choice_student_t: Alaska's Senate and governor final rounds draw the polling component as
the unit-variance t(5) every other statewide race uses, convolved with the normal ranking error;
the centre does not move. The promotion case was registered before its measurement
(experiments/rcv_student_t_promotion_result.md, largest move 0.43 points). The family keeps no
champion pointer: both 1.1.0 and 1.2.0 are research rows in model_version, and the published
Alaska races are the newest 1.2.0 run (stored 2026-10-29 00:51 UTC).
2.5j Training inputs corrected (2026-10-28)
Input corrections, each through a registered comparison before it went live (live1028b on
Skynet-Three, 2026-10-28 22:49 to 2026-10-29 01:48 UTC), the House and statewide gates rerun on
the live rows (both PASSED) and the statewide record rerun and rescored:
- Money (
money_filing_dates): pre-mid-2015 FEC filings dated by their filing number's receipt date, so 2012 and 2014 outside money is point-in-time (experiments/h128_money_and_history_result.md). - History (same registration, part 10): ND Senate 2004, Mississippi governor 2011/2015/2019 from the Secretary of State, the 1999 general date, California's 2003 recall filed as a special. Statewide record log loss 0.1628 -> 0.1629, within its limit.
- The House training rows 2012-2024 rebuilt on the corrected results (FM-254), and integrity rule D2 now alerts when a past House result is corrected after the rows were built.
2.5k Components scored prospectively on 2026 (registered 2026-10-28)
Every component kept on evidence short of its rule (TODO H132) has a registration committed before
any 2026 result, and so does every component or assumption in
experiments/validation_2026/INVENTORY.md that 2026 can inform (TODO H133; the rows 2026 cannot
inform say why). None changes a published number before 2026-11-03. Still to do: the scorers
SCORING.md lists as "to write", and the adjustment audit itself scores no 2026 race as coded
(SG_CYCLES/HOUSE_CYCLES end at 2024; SCORING.md step 9).
| Registration | Component | Arms |
|---|---|---|
experiments/scandal_rule_v2_preregistration.md |
statewide_scandal_rule |
A1 = the published rule with the polled limit |
experiments/house_rater_blend_v2_preregistration.md |
house_rater_blend |
C0 no blend, B published, M margin form (shadow job rater-v2-shadow) |
experiments/house_poll_age_measured_preregistration.md |
house_poll_age_decay |
DECAY published, LIVE equal weights, DRIFT measured |
experiments/sparse_error_accounting_preregistration.md |
sparse_measured_sd, sparse_blend |
B published, Ck, Crhok (decides: rho 0.41, 95% 0.23-0.64) |
experiments/district_poll_shared_2026_prospective_preregistration.md |
house_district_poll_shared (off) |
PUB against SH, frozen 2026-10-28 and at the cutoff (job h86-shadow) |
experiments/validation_2026/*_2026_preregistration.md (V1-V8) |
national environment, statewide error budget, chamber error structure, polls/prior/blend, House components, poll rules, ranked choice, state legislatures | sealed at the cutoff by job validation-seal (FREEZE.md); scored by SCORING.md |
The sparse prior counted once (FM-253, ELAB_SPARSE_PRIOR_ONCE) passed its rule on the record but
narrowed the sparse races' already narrow 90% ranges (0.843 -> 0.784); it stays off and is to be
made together with the range-width fix this table's sparse registration scores.
2.6 The contradiction check
Each registry entry lists the claims its evidence makes (claims in data/model/adjustments.json):
a quantity and a conclusion (correct or do_not_correct). Any two claims about the same quantity
with opposite conclusions are reported as a contradiction, whatever the status of the entries
involved (an active adjustment against a rejected experiment is exactly the case that went
unnoticed). A contradiction is resolved only by a pre-registered test under (b), or by recording in
the registry why the two claims are about different quantities.
2.7 Effect on today's forecasts
For each adjustment: the median and maximum absolute change in the election-day margin and in the win probability across today's published races when that one step is switched off, computed in memory from the same inputs the published run used. Nothing is persisted.
2.8 Automation and alerts
scripts/audit_adjustments.py --if-due --alert runs daily from the scheduler and does the work when
seven days have passed since the last audit or when the set of certified results has changed (that
is, after each certified election). It sends one phone alert (ntfy, as the integrity job does) when
an adjustment that was kept is now flagged, or when a contradiction appears that the previous audit
did not report. Fits are cached by their exact inputs outside the repository, so a re-run refits
only races whose polls, results or parameters changed.
3. The registry
One entry per transformation, found by reading the code paths that produce the published forecasts: scripts/scheduler.py jobs forecast-senate/forecast-governor (scripts/publish_forecasts.py -> src/elab/backtest/pipeline.py -> src/elab/models/*, src/elab/simulate/forecast.py), apply-statewide-prior (scripts/apply_statewide_prior.py), the chamber jobs (scripts/publish_chamber.py) and chamber-house (scripts/forecast_house_ground_up.py and the national term from scripts/backtest_national.py). This section is generated by scripts/render_adjustments_doc.py from the machine-readable registry, data/model/adjustments.json (which also holds the claims the contradiction check reads), and the committed audits (2026-10-12, 2026-10-05).
Today's effect is measured on the published forecasts of the audit's date by switching the one step off in memory: the absolute change in the election-day margin (points) and in the win probability, median and maximum across races, with the race where the margin moved most. Nothing was persisted. Each entry shows the newest audit that judged it: the 2026-10-12 audit covered Senate and governor terms only, so House and chamber entries show 2026-10-05.
Senate and governor
poll_selection_asof — Which polls a forecast reads (active)
- What it does: A forecast reads only polls recorded, published and (for reconstructed polls) released by its standpoint, the headline question only, not withdrawn or superseded; a poll with several versions keeps the first one stored; a firm's overlapping tracking waves count once.
- Where:
src/elab/repo/observations.py:ObservationRepository.for_race - Parameters: ELAB_RECON_RELEASE_BASIS=released_at, ELAB_RECON_RELEASE_LAG_DAYS=2 (since 2026-10-10, audit M13; before: field_end, 0); ELAB_POLL_WAVES=per_question (since 2026-10-12, audit M08: of a firm's overlapping waves of one question in a race, the newest and those overlapping no kept wave; before: all). Each run records its rule.
- Evidence: docs/BACKTESTING.md; tests/leakage/test_asof_canary.py; tests/leakage/test_release_basis.py
- Last tested by its own protocol: 2026-10-05. Protocol: Leakage tests (canary poll after the as_of must change nothing); a data rule, not a model adjustment.
- Standard test (2026-10-12): UNTESTABLE — A knowability rule, not an adjustment: switching it off is leakage, not an alternative forecast. Guarded by the leakage tests.
forecast_target — What a statewide margin measures (active)
- What it does: Senate and governor forecasts, and their scoring, are the gap between the two candidates as shares of all votes cast, third parties included -- the way results are reported. Polls enter with their undecideds; the scale law carries them onto that scale. House forecasts use the two-party share, D/(D+R), throughout their own model.
- Where:
src/elab/evaluate/scored.py;src/elab/evaluate/truth.py;scripts/estimate_scale_compression.py;scripts/build_district_panel.py - Parameters: Since 2026-10-11 (audit M07): scoring moved from the two-candidate renormalised result to the total-vote margin the models were already fitted on.
- Evidence: experiments/audit_m07_result.md; experiments/audit_m07_allocation.json; experiments/compression_mechanism.md
- Last tested by its own protocol: 2026-10-12. Protocol: Raw poll margins with a walk-forward scale law against undecideds allocated in proportion before averaging, 748 races, total-vote target: the allocation is worse overall (MAE +0.34, 90% CI +0.20 to +0.50) and in every undecided and third-party band.
- Standard test (2026-10-12): UNTESTABLE — The definition of the quantity forecast and scored, not a step that can be switched off; its one alternative treatment (allocating undecideds) was tested separately, as above.
lv_rv_handling — Likely-voter adjustment of registered-voter and adult polls (refused; all populations enter alike) (rejected)
- What it does: Nothing is applied: likely-voter, registered-voter and adult samples enter alike; the population column is stored and read by nothing in the model. When a release is listed in several populations, one version is kept: likely voters, else registered voters, else adults, whatever order the source lists them in (since 2026-10-10, audit M02; before, whichever was stored first). A firm's typical lean, including any from its population, is absorbed by its house effect. A likely-voter adjustment (registered-voter and adult polls moved toward the Republican by the gap measured where 538 lists a poll both ways: -1.18 points for 2026) was pre-registered and tested on 2026-10-12 and refused: slightly worse log loss, better in 1 of 3 cycles; today it would move races by 0.26 points in the median.
- Where:
src/elab/repo/observations.py (population read, unused);src/elab/ingest/wikipedia_discovery.py:_already_held - Parameters: none
- Evidence: docs/STATISTICAL_MODEL.md section 4 (describes an LV/RV term g_pop that is not implemented); docs/FAILURE_MODES.md FM-79 (the sigma_lv placeholder retired into sigma_race); experiments/preregistration_lv_rv_adjustment.md; experiments/lv_rv_adjustment_result.md (the adjustment tested and failed, so none is applied)
- Last tested by its own protocol: 2026-10-12. Protocol: Pre-registered addition (experiments/preregistration_lv_rv_adjustment.md), standard test 2020-2024, eve and h28: refused (log loss gain -0.0009 eve, -0.0004 h28; 1 of 3 cycles improved; margin MAE 4.736 with vs 4.759 without).
- Standard test (2026-10-12): UNTESTABLE — Nothing is applied, so there is nothing to switch off; 7,242 of 11,258 historical statewide polls record no population at all, so an LV/RV term cannot be fitted walk-forward from this corpus. Adding one would be a new specification under policy (b).
sponsor_handling — Campaign- and party-sponsored polls (active)
- What it does: Sponsored polls (a campaign, party committee, PAC or interest group) are kept; the sponsor's lean is left to the pollster's house effect. Roughly half of live statewide polls are flagged sponsored (327 of 676 when counted on 2026-10-05). Tested against dropping them, statewide and for the House (2026-10-23); kept by the owner's decision on that result. The in-house polling operations of campaigns and party committees (nine firms) are excluded by name (config/excluded_pollsters.yaml).
- Where:
src/elab/backtest/pipeline.py:forecast_race_at (exclude_partisan=False for statewide);src/elab/ingest/poll_sponsors.py - Parameters: exclude_partisan=False
- Evidence: experiments/preregistration_polled_nonpartisan.md (the registration of the arm that drops sponsored polls, refused in house_polled_nonpartisan.md); experiments/house_polled_nonpartisan.md (dropping sponsored polls: refused); experiments/partisan_polls_backtest_2026_10_23_preregistration.md; experiments/partisan_polls_backtest_2026_10_23_result.md
- Last tested by its own protocol: 2026-10-23. Protocol: Statewide: the champion's 2016-2024 election-eve record rerun with and without 1,194 partisan-sponsored polls (log loss 0.1640 vs 0.1620 pooled, 2016 worse by 0.011, four of five cycles worse; removal not supported). House: the deployed poll layer with and without sponsored polls (31-day lean MAE 4.15 vs 4.96; removal not supported). Earlier: House seat MAE 2018-2024, dropping refused (8.1 vs 6.0).
- Today's effect (2026-10-12): margin median 0.18 / max 27.72 (US-SD); win prob median 0.005 / max 0.395; 42 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: pooled log loss worse or equal with it (gain -0.0022); effect reverses sign in presidential years, governor races, lopsided races. four weeks out: effect reverses sign in midterm years.
pollster_house_effects — Pollster house effects (sum-to-zero) (active)
- What it does: Each pollster in a race gets its own lean, estimated from that race's polls; the leans are constrained to average zero across the race's polls (weighted by how many each firm released), so the latent path tracks 'the average pollster'.
- Where:
src/elab/models/latent/single_race.py:build_model (house, house_sd, sum-to-zero by volume) - Parameters: house_sd ~ HalfNormal(2.5); weights = poll counts
- Evidence: docs/STATISTICAL_MODEL.md section 5; docs/FAILURE_MODES.md FM-13; experiments/house_effects.md; experiments/batch03_pooled_house.md (house effects pooled across races: rejected)
- Last tested by its own protocol: 2026-09-24. Protocol: Never gated itself; the pooled cross-race alternative was tested (log loss, 11 cycles) and rejected (+0.0013, 5 of 11 cycles).
- Today's effect (2026-10-12): margin median 0.04 / max 0.45 (US-AR); win prob median 0.001 / max 0.024; 43 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: effect reverses sign in presidential years. four weeks out: pooled log loss worse or equal with it (gain -0.0004); improves log loss in 5 of 10 cycles, not a majority; effect reverses sign in midterm years, governor races, lopsided races.
design_effect — Design effect on poll sampling error: measured, walk-forward (was 1.6) (active)
- What it does: Every poll's sampling variance is multiplied by a design effect. Until 25 October 2026 it was a fixed 1.6; since then it is the value the project's own dispersion law measures from earlier cycles (0.76 for 2026, from 2018-2024), adopted on its registered test. Earlier cycles with nothing measured before them keep 1.6. A design effect below one means that national generic-ballot polls taken within three days of each other by different firms, once each firm's house effect is taken out, scatter around each other a little less than their sample sizes alone imply. A design effect below one does not mean polls are more accurate than their sample sizes imply: the design effect scales only the part of a poll's error that shrinks as the sample grows. On top of that part the forecast adds each firm's own lean (its house effect), each poll's extra noise that does not shrink with its sample, a race-specific error shared by every poll of that race, and a national polling error shared by every race. In a safe, lightly polled race those added parts are why the polls miss the result by far more than their sample sizes imply (the sparse-race error below).
- Where:
src/elab/models/latent/single_race.py:sampling_sd_points;src/elab/models/sparse.py:DESIGN_EFFECT - Parameters: experiments/poll_dispersion_pre<cycle>.json design_effect (2026: 0.757); 1.6 where none; ELAB_DESIGN_EFFECT_MEASURED on by default, 0 restores the constant 1.6
- Evidence: docs/STATISTICAL_MODEL.md section 4.1; experiments/poll_dispersion*.json (the measured values); experiments/design_effect_measured_preregistration.md; experiments/design_effect_measured_result.md
- Last tested by its own protocol: 2026-10-25. Protocol: Registered test 2026-10-25 (experiments/design_effect_measured_preregistration.md), every champion election-eve backtest of 2016-2024 against the 1.6 control: log loss 0.1641 against 0.1644, Brier 0.0507 both, 80%/90% coverage 0.863/0.945 against 0.867/0.949, no cycle worse by more than 0.0002. Adopted; the gain is small (experiments/design_effect_measured_result.md).
- Today's effect (2026-10-12): margin median 0.07 / max 0.63 (US-SD); win prob median 0.001 / max 0.024; 45 of 60 races changed
- Standard test (2026-10-12): FLAG — four weeks out: ranges miss nominal coverage by 6.2 points with it, 5.6 without.
fundamentals_prior — Fundamentals prior on the race intercept (active)
- What it does: The latent path's starting level gets an informative prior from the state's previous results (same seat where possible) with a width from how much such margins move, instead of a vague Normal(0, 15) centred on a tie. Since 27 October 2026 a gap length's width is measured only from at least eight different states, not merely eight pairs of results: the governor gaps of two, six, ten and fourteen years had come from two to five states (Vermont, New Hampshire and the odd-year states), and now take the nearest gap measured from eight or more. It passed its registered test with no measurable change in accuracy; single_race_latent 0.7.2 (the poll model) and 0.8.6 (the published blend); ELAB_VOL_MIN_STATES=0 restores 0.7.1.
- Where:
scripts/publish_forecasts.py (prior_for, measure_volatility);src/elab/models/fundamentals/prior.py:prior_for;src/elab/models/latent/single_race.py:build_model start_margin;src/elab/models/fundamentals/prior.py:measure_volatility, vol_min_states_enabled - Parameters: volatility from cycles before the target; a gap kept only with at least 8 pairs from at least 8 distinct states (VOL_MIN_STATES), else the nearest measured gap of the office scaled by sqrt(gap ratio)
- Evidence: docs/DECISIONS.md ADR-0007; docs/FAILURE_MODES.md FM-84; experiments/volatility_min_states_preregistration.md; experiments/volatility_min_states_result.md; docs/FAILURE_MODES.md FM-237
- Last tested by its own protocol: 2026-10-27. Protocol: Shipped as a fix (ADR-0017), not gated; 284 races: CRPS 3.928 -> 3.871, MAE 5.46 -> 5.40, better in 5 of 5 cycles. The volatility minimum of states (M-19) registered as a consistency correction, A/B over every champion election-eve backtest of 2016-2024 (256 races): published blend log loss 0.1641 and 0.1642, Brier 0.0516 both, coverage 0.469/0.820/0.934 both, MAE 4.57 both; adopted 2026-10-27.
- Its own test: Adopted as a fix without a test gate (ADR-0017); a later change to it passed its own pre-registered test.
- Today's effect (2026-10-12): margin median 0.08 / max 6.45 (US-LA); win prob median 0.002 / max 0.087; 60 of 60 races changed
- Standard test (2026-10-12): KEEP
prior_width_net — Statewide prior: width measured net of the national swing (active)
- What it does: The width of the starting prior (how far a seat's margin has moved between elections) is measured after the national swing between those elections is taken out, as the chamber simulation measures it, so the swing is not counted twice; the prior is about 8% narrower than the raw width.
- Where:
src/elab/models/fundamentals/prior.py:measure_volatility;scripts/publish_forecasts.py (ELAB_PRIOR_WIDTH_NET, net_of_national) - Parameters: ELAB_PRIOR_WIDTH_NET on by default; 0 restores the raw width
- Evidence: experiments/prior_width_net_preregistration.md; experiments/prior_width_net_result.md
- Last tested by its own protocol: 2026-10-24. Protocol: Consistency correction: adopted unless log loss worse by more than 0.002, Brier by more than 0.0005 or 80%/90% coverage further from nominal by more than 0.02; log loss 0.1642 -> 0.1643, Brier and coverage equal. Adopted.
- Standard test: UNTESTABLE — Tested by its own registered A/B over every champion election-eve backtest of 2016-2024 (the result file); the adjustment-audit harness has no arm for it.
prior_detide — Statewide prior: the previous margin net of its own cycle's national tide (active)
- What it does: The starting prior of a Senate or governor race is the seat's previous result, and that result carries the national mood of the year it was won in: a 2026 Senate prior taken from 2020 describes the seat in 2020's environment. Since 26 October 2026 that year's national level is taken out and this year's estimated level put in, both read from one fit of earlier cycles' Senate (or governor) results on the national House vote, built only from elections before the one forecast. It moves every 2026 Senate and governor prior toward the Democrats (Senate by 3.45 points on average, governors by 6.07) before the polls and the ground-up blend; the House is not changed. The de-tided prior passed its registered test with no measurable change in accuracy, and moved the published 2026 forecasts by 0.14 points on average over seventy-one races, with no rating or chamber change. In the Oklahoma governor's race, where the Republican is favored, the Republican's chance went from 84.2% to 83.0% and the Democrat's from 15.8% to 17.0%. Adopted together with the uncertainty-terms refit as single_race_latent 0.7.1 (the poll model) and 0.8.5 (the published blend); the 0.14-point figure is the de-tiding alone. ELAB_PRIOR_DETIDE=0 with the 2026-09-29 terms files restores 0.7.0.
- Where:
src/elab/models/fundamentals/prior.py:detide_levels, detide_from_env, prior_for;scripts/publish_forecasts.py (the race intercept prior and the sparse blend);scripts/publish_chamber.py (seats with no stored forecast);src/elab/registry/versions.py:statewide_semvers - Parameters: senate and governor only; per office, earlier cycles' level regressed on the realised national House margin (2004 to the cycle before the target), read at the source cycle's realised House margin and at the target cycle's N3 (data/national/backtest.json); 2026: Senate -1.47 + 0.787 x House margin, governor -4.97 + 0.635 x House margin, N3 +6.95
- Evidence: experiments/detide_prior_preregistration.md; experiments/detide_prior_result.md; experiments/detide_ab_control.score.txt; experiments/detide_ab_treat.score.txt; docs/FAILURE_MODES.md FM-231
- Last tested by its own protocol: 2026-10-26. Protocol: Registered as a consistency correction (amendment 1: the tide alone, the seat set cancelling), A/B over every champion election-eve backtest of 2016-2024 (256 races): adopted unless log loss worse by more than 0.002, Brier by more than 0.0005, or 80%/90% coverage further from nominal by more than 0.02. Published blend, control against treatment: log loss 0.1662 and 0.1662, Brier 0.0517 and 0.0518, coverage 0.836/0.945 and 0.840/0.945, MAE 4.58 and 4.58; adopted. The poll model alone was worse with it (log loss 0.1744 -> 0.1781, Brier 0.0554 -> 0.0571); the ground-up blend absorbs it.
- Standard test: UNTESTABLE — Tested by its own registered A/B backtest; the adjustment-audit harness has no arm for the prior's construction.
november_round — Results read at their November round in the three nominee-runoff contests (statewide panel and start prior) (active)
- What it does: For the nominee runoffs (Georgia's Senate 2008, 2020 regular and 2022), the statewide panel and the poll model's start prior read the November round rather than the runoff; all-party runoffs keep their runoff.
- Where:
scripts/build_state_panel.py:_november (ELAB_RESULT_ROUND);src/elab/repo/results.py:ResultRepository.margins (ELAB_RESULT_ROUND_PRIOR, _NOVEMBER_ROWS) - Parameters: ELAB_RESULT_ROUND=november and ELAB_RESULT_ROUND_PRIOR on by default
- Evidence: experiments/november_round_preregistration.md; experiments/november_round_result.md; experiments/november_round_prior_preregistration.md; experiments/november_round_prior_result.md
- Last tested by its own protocol: 2026-10-24. Protocol: Two corrections of stored data, each by its registered rule: the panel by the statewide gate (better on every pooled measure), the start prior adopted unless Brier worse by more than 0.0005 or MAE by more than 0.05 (Brier +0.0001, MAE equal).
- Standard test: UNTESTABLE — Tested by its own registered A/B over every champion election-eve backtest of 2016-2024 (the result file); the adjustment-audit harness has no arm for it.
min_polls_rule — Four polls to fit a trajectory (active)
- What it does: A race with four or more usable polls is fitted by the latent model; one with one to three goes to the sparse blend; one with none is not a poll forecast.
- Where:
scripts/publish_forecasts.py --min-polls 4;src/elab/backtest/pipeline.py:forecast_race_at - Parameters: min_polls = 4 (the remote worker fits from 3)
- Evidence: experiments/preregistration_min_polls.md; experiments/prior_only_fallback_ignores_sparse_polls.md
- Last tested by its own protocol: 2026-09-23. Protocol: k=3 (c6) tested: passed (log loss -0.0395) and superseded by c7; the value 4 itself was never tested.
- Standard test (2026-10-12): UNTESTABLE — A threshold with no 'off' state: below three polls the latent model cannot be fitted at all. Comparing 3 against 4 is a specification change under policy (b).
sparse_blend — Sparse-race blend (c7) (active)
- What it does: A race with one to three polls is forecast by combining the fundamentals prior with the average of its polls, each weighted by its precision, instead of the prior alone.
- Where:
src/elab/models/sparse.py:sparse_estimate;src/elab/backtest/pipeline.py:forecast_race_at (len(usable) < min_polls) - Parameters: prior (mean, sd); poll sd as sparse_measured_sd
- Evidence: experiments/preregistration_sparse_blend.md; experiments/prior_only_fallback_ignores_sparse_polls.md; experiments/sparse_prior_once_preregistration.md; experiments/sparse_prior_once_result.md (the prior counted once: passed its rule, held: the sparse races' 90% coverage fell; FM-253)
- Last tested by its own protocol: 2026-09-23. Protocol: Pre-registered, confirmation cycles 2018-2024, log loss, BH q <= 0.10: -0.0474, q=0.0002, 4 of 4 cycles. Promoted.
- Today's effect (2026-10-12): margin median 0.00 / max 11.71 (US-LA); win prob median 0.000 / max 0.175; 15 of 60 races changed
- Standard test (2026-10-12): KEEP
- 2026-10-28 (experiments/sparse_prior_once_result.md, FM-253): a race below four usable polls takes the seat prior inside the sparse blend and the ground-up prior again in apply_statewide_prior.py. Counting it once (ELAB_SPARSE_PRIOR_ONCE) passed its registered rule on the 256-race record (log loss 0.16415 -> 0.16400, Brier 0.05162 -> 0.05163), but the sparse races' 90% coverage fell 0.843 -> 0.784; the flag stays off, to be fixed together with the sparse races' range width (experiments/sparse_error_accounting_preregistration.md).
sparse_measured_sd — Realised error of a sparse race's poll average (FM-104) (active)
- What it does: In the sparse blend a handful of polls is weighted by how far such averages have actually landed from results in earlier cycles (about 9-10 points), not by their sampling error (about 4). Since 25 October 2026 the national and race-specific parts of that error are taken out of it before the blend, because the forecast adds both again afterwards: they had been counted twice (adopted by the owner's decision after its registered test tied on one condition; experiments/sparse_shared_once_result.md).
- Where:
src/elab/models/sparse.py:pooled_realised_sd;scripts/publish_forecasts.py (sparse_poll_variance.json);src/elab/models/sparse.py:sparse_estimate (shared_sd: ELAB_SPARSE_SHARED_ONCE, on by default) - Parameters: Realised sd of a 1-3-poll average around the result, pooled over earlier cycles: 9.37 pts for 2026 (measured on the total-vote target since 2026-10-11, audit M07; 9.98 against the two-candidate renormalised result before); ELAB_SPARSE_SHARED_ONCE on by default (the national and race-specific parts taken out once), 0 restores the double count
- Evidence: experiments/sparse_blend_poll_weight.md; docs/FAILURE_MODES.md FM-104; experiments/sparse_variance.md (the first registration, rejected on its metric); experiments/sparse_shared_once_result.md (the shared parts counted once: adopted by the owner's amendment); experiments/sparse_error_accounting_preregistration.md (how a sparse race counts the shared error and how wide it is; section 3a, the overlap of prior and poll error; scored on 2026)
- Last tested by its own protocol: 2026-09-28. Protocol: Shipped as a defect fix: 67 sparse races, MAE 8.01 -> 6.34, 90% coverage 0.791 -> 0.925. As challenger c9 (sparse_variance.md, 2026-09-25) the same change was refused on log loss (+0.0150, 0 of 3 folds).
- Today's effect (2026-10-12): margin median 0.00 / max 6.38 (US-LA); win prob median 0.000 / max 0.097; 15 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: ranges miss nominal coverage by 4.4 points with it, 2.0 without. four weeks out: ranges miss nominal coverage by 6.5 points with it, 4.6 without.
- 2026-10-25 (experiments/sparse_shared_once_preregistration.md): shared error counted once. Log loss 0.1630 against 0.1644, Brier 0.0504 against 0.0507, 80% coverage 0.847 against 0.867, 90% 0.949 both (the rule's condition not met on the tie). Adopted by the owner's amendment; both versions scored after November 2026.
- 2026-10-28 (experiments/sparse_error_accounting_preregistration.md, registered with section 3a): the published shared-once arm B against the exact posterior with its own residual (Ck) and, because the prior's and the polls' errors overlap (rho 0.41, 95% 0.23 to 0.64, from the thirty-three sparse races of 2020-2024; k_rho^2 25.40), the arm that allows for the overlap (Crhok), to which the decision rule applies; scored on 2026 after certification.
sigma_nat — Shared national polling error (sigma_nat) (active)
- What it does: Every race's election-day spread includes an error shared by all races in a cycle, the size of past cycles' average polling misses (2.63 points for 2026; 2.44 before the 26 October 2026 refit, which fitted it against the November round the polls measured instead of runoff or superseded result rows).
- Where:
src/elab/simulate/forecast.py:election_day;scripts/publish_forecasts.py (terms['systematic']['sigma_nat']) - Parameters: experiments/uncertainty_terms_pre<cycle>.json (since 2026-10-26 the fixed-selection refit for 2016-2026; single_race_latent 0.7.1)
- Evidence: experiments/uncertainty_terms.md; docs/FAILURE_MODES.md FM-82; experiments/uncertainty_terms_refit_preregistration.md; experiments/uncertainty_terms_refit_result.md; docs/FAILURE_MODES.md FM-232
- Last tested by its own protocol: 2026-10-26. Protocol: Estimated (variance components over earlier cycles). Refitted 2026-10-26 with the fixed result selection (the round the polls measured, live rows, every party spelling; FM-232) and adopted by its registered consistency test (experiments/uncertainty_terms_refit_result.md): over every election-eve backtest of 2016-2024 the published forecast's log loss 0.1662 -> 0.1641, Brier 0.0517 -> 0.0516, 80%/90% coverage 0.836/0.945 -> 0.824/0.938, MAE 4.58 -> 4.57. The 2026-09-29 files are kept as experiments/uncertainty_terms_pre<cycle>_2026-09-29.json.
- Today's effect (2026-10-12): margin median 0.10 / max 0.42 (US-OR); win prob median 0.005 / max 0.021; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: ranges miss nominal coverage by 4.4 points with it, 0.9 without. four weeks out: ranges miss nominal coverage by 6.5 points with it, 4.6 without.
- Re-estimated on today's data and tested 2026-10-12 (experiments/statewide_spread_result.md): the wider refreshed terms (2026: 3.45, or 3.00 on federal cycles) moved coverage further from nominal; the 2026-09-29 files stay.
sigma_race — Race-specific residual error (sigma_race) (active)
- What it does: Every race's spread includes a race-specific error beyond its polls (4.93 points for 2026; 4.97 before the 26 October 2026 refit), the residual of the same variance-components fit.
- Where:
src/elab/simulate/forecast.py:election_day;scripts/publish_forecasts.py (terms['systematic']['sigma_race']) - Parameters: experiments/uncertainty_terms_pre<cycle>.json (since 2026-10-26 the fixed-selection refit for 2016-2026; single_race_latent 0.7.1)
- Evidence: experiments/uncertainty_terms.md; docs/FAILURE_MODES.md FM-79; docs/FAILURE_MODES.md FM-89; experiments/uncertainty_terms_refit_preregistration.md; experiments/uncertainty_terms_refit_result.md; docs/FAILURE_MODES.md FM-232
- Last tested by its own protocol: 2026-10-26. Protocol: Restored as a fix (FM-79): 247 races, 90% coverage 0.680 -> 0.895 walk-forward. Refitted 2026-10-26 with the fixed result selection (the round the polls measured, live rows, every party spelling; FM-232) and adopted by its registered consistency test (experiments/uncertainty_terms_refit_result.md): over every election-eve backtest of 2016-2024 the published forecast's log loss 0.1662 -> 0.1641, Brier 0.0517 -> 0.0516, 80%/90% coverage 0.836/0.945 -> 0.824/0.938, MAE 4.58 -> 4.57. The 2026-09-29 files are kept as experiments/uncertainty_terms_pre<cycle>_2026-09-29.json.
- Today's effect (2026-10-12): margin median 0.42 / max 1.88 (US-OR); win prob median 0.027 / max 0.108; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — four weeks out: ranges miss nominal coverage by 6.5 points with it, 15.3 without.
drift_law — Drift to election day (active)
- What it does: A forecast made before election day is widened by how far races have historically moved between that horizon and the vote: variance ah^b, h in days (1.650h^0.669 for 2026, about 3.9 points of sd at four weeks, in margin points; 1.477*h^0.691 before the 26 October 2026 refit).
- Where:
scripts/publish_forecasts.py (drift VarianceTerm);src/elab/simulate/forecast.py:election_day - Parameters: experiments/uncertainty_terms_pre<cycle>.json drift a, b (the fixed-selection refit since 2026-10-26)
- Evidence: experiments/uncertainty_terms.md; experiments/uncertainty_terms_refit_preregistration.md; experiments/uncertainty_terms_refit_result.md; docs/FAILURE_MODES.md FM-232
- Last tested by its own protocol: 2026-10-26. Protocol: Estimated from historical trajectories. Refitted with the fixed result selection on 2026-10-26 (FM-232) as part of the adopted terms refit; its test scored election eve only, where the drift term is near zero, so the refitted law is untested at the horizons it acts on.
- Its own test: Refitted and tested on election eve only, not at the horizons it acts on.
- Today's effect (2026-10-12): margin median 0.22 / max 0.96 (US-OR); win prob median 0.012 / max 0.050; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — four weeks out: ranges miss nominal coverage by 6.5 points with it, 2.4 without.
- A walk-forward drift multiplier was tested 2026-10-12 (experiments/statewide_spread_result.md) and not adopted: coverage error 0.083 -> 0.080 four weeks out, short of the 0.01 required.
student_t_shocks — Fat-tailed (Student-t, nu=5) election-day errors (active)
- What it does: The non-poll error terms are drawn from a Student-t with 5 degrees of freedom rather than a normal, so large misses are more likely than a bell curve allows. A race's total error adds these to the poll model's own uncertainty (normal for a race with too few polls to fit), so the total is fatter-tailed than a normal but is not itself a Student-t with 5 degrees of freedom.
- Where:
src/elab/simulate/forecast.py:_simulate (nu=5.0) - Parameters: nu = 5 (constant)
- Evidence: docs/STATISTICAL_MODEL.md section 10; docs/FAILURE_MODES.md FM-12
- Last tested by its own protocol: never. Protocol: Never tested; a constant.
- Today's effect (2026-10-12): margin median 0.05 / max 0.28 (US-OR); win prob median 0.007 / max 0.040; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: improves log loss in 5 of 10 cycles, not a majority; effect reverses sign in lopsided races. four weeks out: effect reverses sign in presidential years.
scale_law — Scale law c15 as a whole (reference entry: the slope is applied, as scale_slope; the intercept is removed since 2026-10-12) (active)
- What it does: The whole forecast distribution was translated so its centre became a + b x centre, with a and b fitted on earlier cycles' final polling averages against results (2026: -1.744 + 1.070 x centre). Since 12 October 2026 (0.8.3) the intercept a is taken back out after the law, so what is applied is the stretch b x centre alone. Reported for reference; the two parts are judged separately below.
- Where:
src/elab/models/calibration/scale.py:ScaleLaw.apply;src/elab/backtest/pipeline.py:forecast_race_at - Parameters: experiments/scale_pre<cycle>.json
- Evidence: experiments/preregistration_scale.md; experiments/scale_compression.md; experiments/scale_calibration_2016.md; experiments/scale_intercept_check_2026-10-05.md
- Last tested by its own protocol: 2026-09-26. Protocol: Pre-registered, walk-forward 2016-2024, CRPS: -0.530 against threshold 0.073, 4 of 5 folds, log loss -0.0088. Promoted as 0.6.0.
- Today's effect (2026-10-12): margin median 1.30 / max 2.94 (US-MT); win prob median 0.024 / max 0.119; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: pooled log loss worse or equal with it (gain -0.0028); effect reverses sign in presidential years, lopsided races. four weeks out: improves log loss in 5 of 10 cycles, not a majority; effect reverses sign in midterm years, close races.
scale_slope — Scale law: slope (anti-compression) (active)
- What it does: Moves the centre of every forecast away from a tie by the fitted slope (1.070 for 2026: a 10-point lead becomes 10.7), because final polling averages have been more compressed than results; the spread is not changed. Since 0.8.3 (12 October 2026) it is the only part of the scale law applied: the slope is fitted together with an intercept that is no longer used (a slope fitted without one would be 1.048).
- Where:
src/elab/models/calibration/scale.py:ScaleLaw.apply - Parameters: experiments/scale_pre<cycle>.json slope
- Evidence: experiments/preregistration_scale.md; experiments/scale_compression.md; experiments/compression_mechanism.md
- Last tested by its own protocol: 2026-10-12. Protocol: Promoted jointly with the intercept (c15, 2026-09-26); applied without it, tested as arm a0 (experiments/scale_intercept_check_2026-10-05.md) and in the adjustment audit of 2026-10-12 (KEEP).
- Today's effect (2026-10-12): margin median 0.42 / max 1.25 (US-MA); win prob median 0.007 / max 0.020; 60 of 60 races changed
- Standard test (2026-10-12): KEEP
scale_intercept — Scale law: intercept (a uniform shift; removed 2026-10-12) (rejected)
- What it does: Moved every forecast by the fitted intercept, -1.744 points toward the Republican for 2026: a mean polling-bias correction learned from all past cycles together. Removed on 12 October 2026 by the owner's decision on audit M11 (single_race_latent/0.8.3): on the record it helped in presidential years and hurt in midterms, its pooled log loss was worse, and 2026 is a midterm. Until then it moved the published Senate and governor forecasts 1.4 points toward the Republican in the median race (at most 1.7) and win probabilities by 2.6 points (at most 11.7). The slope stays.
- Where:
scripts/apply_statewide_prior.py:apply (every blend from 0.8.3 on subtracts each source run's stored intercept) - Parameters: experiments/scale_pre<cycle>.json intercept (walk-forward values from +1.52 to -1.96)
- Evidence: experiments/preregistration_scale.md; experiments/scale_intercept_check_2026-10-05.md; experiments/mean_bias_challengers_2026-10-05.md
- Last tested by its own protocol: 2026-10-12. Protocol: Standard test 2026-10-12 on the total-vote target (adjustment_audit_2026-10-12): pooled log loss 0.1604 with it, 0.1529 without (375 races, election eve); midterms worse with it (-0.020 log loss gain, 217 races), presidential years better (+0.010, 158); same direction 28 days out. Flagged for review. The 2026-10-05 check found the same split.
- Today's effect (2026-10-12): margin median 1.38 / max 1.74 (US-NE); win prob median 0.026 / max 0.117; 60 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: pooled log loss worse or equal with it (gain -0.0075); effect reverses sign in presidential years. four weeks out: pooled log loss worse or equal with it (gain -0.0034); effect reverses sign in presidential years.
- Challenged 2026-10-05 (pre-registered at b5a423d;
experiments/mean_bias_challengers_2026-10-05.md): slope only (S1) and a same-kind intercept (S2) both not promoted on the citable folds 2006-2016. The champion's intercept stays. - Removed 2026-10-12 (owner's decision on audit M11). On the stored 2016-2024 record, paired on the same source runs (234 races, election eve), 0.8.3 against 0.8.2: log loss 0.1733 -> 0.1691; midterms 0.1857 -> 0.1691 (121 races), presidential years 0.1600 -> 0.1690 (113); nominal 80%/90% ranges cover 0.876/0.957 overall and 0.926/0.984 in midterms (experiments/scale_intercept_removal_2026-10-12.md).
mean_bias_correction — Mean polling-bias correction (rejected)
- What it does: Subtracting the average past polling miss from every forecast. Rejected; listed for its evidence.
- Where: -
- Parameters: n/a
- Evidence: experiments/mean_bias_correction.md; experiments/bias_centre.md; experiments/preregistration_bias_centre.md; experiments/preregistration_recency_centre.md (c19, a later re-registration of a recency mean: suggestive only on earlier cycles, not adopted; 2026 is its confirmatory fold)
- Last tested by its own protocol: 2026-09-29. Protocol: mean_bias_correction.md (2026-09-23): out-of-sample |error| 3.22 -> 3.09, judged noise. c8_bias_centre (2026-09-25): rejected three times on log loss. c19 recency centre (2026-09-29): not promoted.
- Standard test (2026-10-12): UNTESTABLE — Not in the published path (rejected); its question is answered by scale_intercept.
statewide_groundup_blend — Statewide ground-up prior blend (the published Senate and governor forecast) (active)
- What it does: The election-day centre of every Democrat-vs-other race is averaged with a ground-up prediction (a gradient-boosted model of the state's lean from its results, demographics, economy, candidates and money, plus the national House environment N3), weighted by each one's variance; the spread is kept and the win probability is read from that same moved distribution (0.8.2 onward; 0.8.1 recomputed it from a normal curve the band did not follow, audit M09). Since 0.8.3 the blend first takes the scale law's intercept back out of the poll model's forecast.
- Where:
scripts/apply_statewide_prior.py:apply;scripts/gate_statewide.py:prior_for_cycle;src/elab/registry/versions.py:STATEWIDE_SEMVERS - Parameters: data/national/state_panel.json, data/national/backtest.json (N3), prior variance from walk-forward residuals
- Evidence: experiments/statewide_gate_preregistration.md; experiments/statewide_gate_result.md
- Last tested by its own protocol: 2026-10-01. Protocol: Pre-registered gate, selection 2016-2022, confirmation 2024; Brier and MAE (not log loss): Brier 0.0677 -> 0.0585, MAE 5.60 -> 5.38 on 175 races; most of the gain is 2020.
- Today's effect (2026-10-12): margin median 1.39 / max 5.65 (US-AK); win prob median 0.022 / max 0.274; 56 of 60 races changed
- Standard test (2026-10-12): FLAG — eve: effect reverses sign in governor races, lopsided races. four weeks out: effect reverses sign in governor races.
prior_sd_class — Statewide ground-up prior: spread by class of predicted margin (active)
- What it does: The ground-up prior's variance in the blend is measured separately for races predicted under 10 points, 10 to 20, and 20 or more, instead of one pooled variance.
- Where:
scripts/gate_statewide.py:prior_var_by_class;scripts/apply_statewide_prior.py:apply (ELAB_PRIOR_SD_CLASS) - Parameters: ELAB_PRIOR_SD_CLASS on by default; 0 restores the pooled variance
- Evidence: experiments/prior_sd_by_margin_preregistration.md; experiments/prior_sd_by_margin_result.md
- Last tested by its own protocol: 2026-10-24. Protocol: Registered rule: lower pooled CRPS of the prior alone (6.075 against 6.087) and 90% coverage in every class within 0.05 of nominal or closer than the control's: met. Adopted, on the prior alone and in the blend's whole-record test.
- Standard test: UNTESTABLE — Tested by its own registered A/B over every champion election-eve backtest of 2016-2024 (the result file); the adjustment-audit harness has no arm for it.
prior_all_vote — Statewide ground-up prior carried onto the all-vote scale (active)
- What it does: The ground-up prior is a two-party margin; before it is blended with the poll forecast, which is on the all-vote scale, it is multiplied by the share of the state's previous general election for the office won by every Democrat and Republican plus any candidate who finished first or second, each at their best round.
- Where:
scripts/apply_statewide_prior.py:major_share_factors;scripts/apply_statewide_prior.py:apply (ELAB_PRIOR_ALL_VOTE) - Parameters: ELAB_PRIOR_ALL_VOTE on by default; 0 restores the two-party prior
- Evidence: experiments/prior_all_vote_preregistration.md; experiments/prior_all_vote_result.md
- Last tested by its own protocol: 2026-10-25. Protocol: Consistency correction (log loss worse by no more than 0.002, Brier by no more than 0.0005, coverage no further from nominal by more than 0.02): adopted; rerun with the corrected factor, MAE 0.05 points worse.
- Standard test: UNTESTABLE — Tested by its own registered A/B over every champion election-eve backtest of 2016-2024 (the result file); the adjustment-audit harness has no arm for it.
prior_only_publication — Unpolled races published from the ground-up prior (FM-140) (active)
- What it does: A Senate or governor race with no poll at all is published from the ground-up prior alone, with the prior's full measured error as its spread, labelled 'fundamentals only, no polls'.
- Where:
scripts/apply_statewide_prior.py:publish_prior_only - Parameters: prior sd from walk-forward residuals
- Evidence: docs/FAILURE_MODES.md FM-140; experiments/unpolled_seat_tails.md
- Last tested by its own protocol: 2026-10-02. Protocol: No gate; justified by the prior's own error (MAE 8.3 on 2016-2022, 6.7 in 2024) and a tail check (0 of 87 seats favoured by 20+ flipped).
- Standard test (2026-10-12): UNTESTABLE — Switching it off leaves the race unpublished, not forecast differently: there is no 'off' forecast to score. The prior's own accuracy is the evidence.
shock_rules — Shock uncertainty widening and fading centre-shift rules (off) (inactive)
- What it does: Widening a race's range after a confirmed severe event, and a centre shift that fades as new polls arrive. Built, registered, failed their tests, switched off. The scandal shift that is live for Senate and governor races is a separate entry (statewide_scandal_rule).
- Where:
scripts/apply_statewide_prior.py (ELAB_SHOCK_UNCERTAINTY_SD and ELAB_SHOCK_SHIFT_POINTS unset; ELAB_SHOCK_UNCERTAINTY_HALF_LIFE_DAYS, the widening rule's half-life, is read only with ELAB_SHOCK_UNCERTAINTY_SD);src/elab/models/shock.py - Parameters: unset
- Evidence: experiments/shock_uncertainty_result.md; experiments/shock_shift_result.md
- Last tested by its own protocol: 2026-10-04. Protocol: Pre-registered; both failed.
- Standard test (2026-10-12): UNTESTABLE — Not in the published path (switched off after failing their own pre-registered tests).
statewide_scandal_rule — Senate and governor: scandal rule (3.0 points against a listed contender, until three polls after the event) (active)
- What it does: After a verified scandal about the candidate themselves (a criminal charge or conviction, a specific misconduct allegation or official finding, or a documented false claim about their own record, reported by two outlets doing their own reporting), the race's margin moves 3.0 points against that candidate from the event's first date, only while the race has fewer than three polls taken after the event. Polls taken after the event already show most of its effect, so with three or more of them the margin is the polls' and the blend's alone. The spread is unchanged. A running mate's, relative's or staffer's trouble does not count. Events are listed by hand in config/statewide_scandal_events_2026.yaml with their sources; races forecast without polls take the shift too. One event is listed: Colorado governor (Victor Marx, from 2026-07-24). Borderline cases (ME, AK, TX, MN and KS Senate) are listed with the evidence that would make them count and are re-checked at each daily review.
- Where:
scripts/apply_statewide_prior.py (scandal_shift, stored as shock_shift; an event held back by its post-event polls is stored as scandal_held);src/elab/models/shock.py (SCANDAL_POLLED_LIMIT, scandal_shift_applies; ELAB_SCANDAL_POLLED_LIMIT, default on, 0 restores the shift on every listed event);config/statewide_scandal_events_2026.yaml - Parameters: 3.0 points, persistent while the race has fewer than three polls whose fieldwork started after the event's first date (counted per contender, as of each forecast); none after that. PROVISIONAL: the size rests on 17 statewide and 12 House cases, the statewide evidence registered as information rather than a confirmatory test; on election eve the polls taken after the event already carried most of the effect (experiments/scandal_investigation/), which is why the shift stops at three of them; widening the range was tested and rejected (shock_uncertainty_result.md: 90% ranges already held 0.92 of shocked races); scored prospectively after certification (scandal_rule_v2_preregistration.md, arm A1)
- Evidence: experiments/shock_shift_result.md (17 Senate and governor races, registered as information); experiments/house_scandal/result.md (the House test, which met its registered criterion); config/statewide_scandal_events_2026.yaml (the verified events); experiments/shock_shift_prospective_preregistration.md (scored after certification); experiments/scandal_investigation/ (why the shift stops at three post-event polls); experiments/scandal_rule_v2_preregistration.md (the second test; arm A1 is this rule); experiments/scandal_polled_limit_result.md
- Last tested by its own protocol: 2026-10-22. Protocol: Senate and governor 17 shocked races (shock_shift_result.md, registered as information): the persistent shift at 3.0 scored better than the champion on every metric (leave-one-out CRPS 3.37 to 3.19, Brier 0.134 to 0.096, winner calls 76% to 87%), but its CRPS interval [-0.71, +0.38] includes zero and the gain rests largely on Akin. The House test of the same size (12 candidates, CRPS -0.77 [-1.31, -0.18], Holm 0.045) met its registered criterion. Adopted by the owner on both; scored prospectively on 2026 after certification.
- Its own test: The registered version failed; the version in use was tested for information only and adopted by the project owner's decision.
- Standard test: UNTESTABLE — the audit's arms have no event list.
ranked_choice_student_t — Alaska's ranked-choice races: the final round drawn with the race model's tail (active)
- What it does: Alaska's Senate and governor races are forecast on their final ranked-choice round. The centre is the scale law applied to the pollsters' average final-round margin. The polling component is drawn with the race model's election-day shape, a unit-variance Student-t with five degrees of freedom, its sd read from the same state's two-way Senate forecast's 90% interval under that shape; the ranking error (ballots that exhaust or transfer differently from what respondents said) is normal, and the two are convolved numerically. ELAB_RCV_STUDENT_T=0 restores a normal polling component (ranked_choice_final_round 1.1.0). AK-AL is forecast from its own polls and does not read this model.
- Where:
src/elab/models/ranked_choice.py (student_t_enabled, TAIL_NU, RANKING_SD, RankedChoiceForecast.cdf; ELAB_RCV_STUDENT_T, default on);scripts/publish_ranked_choice.py (SEMVER_STUDENT_T 1.2.0) - Parameters: polling component: unit-variance Student-t, 5 degrees of freedom; ranking error: normal, sd 5.05 points; centre unchanged
- Evidence: experiments/audit7_preregistration.md (item 4); experiments/audit7_result.md; experiments/rcv_student_t_promotion_preregistration.md; experiments/rcv_student_t_promotion_result.md; experiments/validation_2026/ranked_choice_2026_preregistration.md (scored on 2026, descriptively)
- Last tested by its own protocol: 2026-10-28. Protocol: Promotion case registered before the measurement (experiments/rcv_student_t_promotion_preregistration.md) on the rule of experiments/audit7_preregistration.md item 4 (passed 2026-10-27): the ranked-choice races take the shape every other statewide race is drawn with. Same-data dry run: the four Alaska rows move 0.24 to 0.43 points (largest the governor's election-day chance, 88.94% -> 89.38%), centres unchanged. ranked_choice_final_round 1.2.0, published 2026-10-28; three contests cannot test the shape on their own.
- Its own test: Tested by its own pre-registered test.
- Standard test: UNTESTABLE — the audit corpus has too few ranked-choice final rounds; the shape follows the statewide race model's tested one.
pollster_sponsor_pooling — Combined sponsor/firm pollster records read as their fielding firm (rejected)
- What it does: Reading records such as "NBC News/Marist" as the firm that fielded the poll (Marist), so its house effect is pooled with the firm's own polls. Tested and rejected: it made the Senate and governor forecasts slightly worse at both standpoints and changed nothing in the House.
- Where: -
- Parameters: data/model/pollster_fielding_firms.json (72 records)
- Evidence: experiments/sponsor_pooling_preregistration.md; experiments/sponsor_pooling_result.md; experiments/sponsor_pooling_result.json; experiments/sponsor_pooling_house_result.json
- Last tested by its own protocol: 2026-10-10. Protocol: Pre-registered standard test (eve and four weeks out, 2006-2024): log loss -0.0002 and -0.0011 with pooling (worse), better in 1 of 6 cycles four weeks out, sign reversal for the Senate; House check identical to the published arm. Not adopted.
- Standard test (2026-10-12): UNTESTABLE — Not in the published path (rejected).
Chambers
chamber_correlation — Chamber simulation: correlated errors and bias sensitivity (active)
- What it does: Senate and governor control probabilities draw one shared national error (and regional and office terms) per simulated election across all races; the model's measured lean is shown only as a sensitivity beside the headline, never applied to it. The shared national error is drawn wider there than on a race page, on purpose (FM-14): its size is itself estimated from past cycles, so it is a Student-t with scale sigma_nat times sqrt(1 + 1/n) and n - 1 degrees of freedom; for 2026 (17 cycles, 2026-10-24 run) that is about 10% wider in sd. Its tail weight is estimated from the past cycles (nu between 4 and 20; 20 for 2026, close to a normal), while each race's own error is a Student-t with 5 degrees of freedom on both the race page and in the chamber.
- Where:
scripts/publish_chamber.py;src/elab/models/correlation/structure.py:estimate_structure;src/elab/simulate/chamber.py - Parameters: estimated from history at run time
- Evidence: experiments/chamber_gate_preregistration.md; experiments/chamber_gate_result.md
- Last tested by its own protocol: 2026-10-01. Protocol: Chamber gate (pre-registered).
- Standard test (2026-10-12): UNTESTABLE — Acts on chamber totals, not race forecasts: one chamber outcome per cycle gives five to ten scored points, too few for the per-cycle rule. Race-level numbers are unaffected.
chamber_marginals — Chamber simulations keep each race's published distribution (active)
- What it does: After the Senate and governor control simulations draw their joint margins, each race's column is mapped by rank onto that race's own published election-day distribution: the dependence between races is kept, and a race's chance inside the chamber is its chance on its own page. Seats with no stored distribution (filled from fundamentals) keep their draws.
- Where:
src/elab/simulate/chamber.py:marginals_mode (ELAB_CHAMBER_MARGINALS) - Parameters: ELAB_CHAMBER_MARGINALS=race by default; structure restores the old construction
- Evidence: experiments/chamber_marginals_preregistration.md; experiments/chamber_marginals_result.md
- Last tested by its own protocol: 2026-10-25. Protocol: Registered rules: every race within 1 point of its published chance (largest gap 0.0), the chamber gate passed, log score not worse by more than 0.10 (slightly better), in both chambers. Adopted.
- Standard test: UNTESTABLE — Tested by its own registered runs at each election eve 2016-2024 (the result file); the adjustment-audit harness has no arm for it.
House (ground-up model)
house_national_adjustments — House: national term N3 instead of the generic ballot at face value (active)
- What it does: The House model's national environment is N3 (the generic-ballot average with its mean past miss removed, blended with a fundamentals regression) rather than the generic ballot as it stands. Both corrections switched off together.
- Where:
scripts/backtest_national.py (n1, n2, blend);scripts/forecast_house_ground_up.py (NAT[2026]['N3']) - Parameters: data/national/backtest.json; 2026 (2026-10-11): N0 9.12 -> N3 7.28, from today's average
- Evidence: experiments/national_environment.md; experiments/district_gate_result.md
- Last tested by its own protocol: 2026-10-01. Protocol: National: walk-forward MAE 2004-2024, N0 3.01, N3 2.16. District gate tested N3 only as part of a package (lean model, national term and errors changed together).
- Today's effect (2026-10-05): margin median 2.02 / max 2.14 (US-NY-23); win prob median 0.004 / max 0.098; 418 of 418 races changed; seats 243.9 with it, 253.5 without
- Standard test (2026-10-05): FLAG — eve: improves log loss in 2 of 5 cycles, not a majority.
house_gb_bias_correction — Generic-ballot bias correction inside N3 (House, and indirectly the Senate and governor forecasts) (active)
- What it does: Subtracts the generic ballot's average past miss (2.62 points toward the Democrat, from 1998-2024) before it enters the national term. The same national term is in the ground-up prior of the Senate and governor forecasts (scripts/gate_statewide.py), so it reaches them too. It is kept while the statewide polls' own intercept was removed on 12 October 2026 because the two rest on different evidence: that intercept was fitted on race polls against race results and hurt in midterms; this correction is fitted on the generic ballot against the national House vote, and its midterm-only miss (2.70 points over seven midterms) is the same size as its pooled one, so the midterm evidence that removed the one does not apply to the other. It has not had the standard test on its own.
- Where:
scripts/backtest_national.py:n2 - Parameters: mean(generic ballot - result) over 1998..t-2
- Evidence: experiments/national_environment.md; experiments/house_generic_ballot_bias.md (refused: the same kind of correction, tested in the retired house_prior/1.7.0 and its corpus, not in N3); experiments/preregistration_generic_ballot_bias.md; experiments/mean_bias_challengers_2026-10-05.md (the challengers to it, no correction and a same-kind one: not promoted)
- Last tested by its own protocol: 2026-10-01. Protocol: Never isolated: national_environment.md (N2 MAE 2.44 vs 3.01) ran the same day house_generic_ballot_bias.md refused the same correction for the old House model (environment MAE 2.72 vs 2.73).
- Today's effect (2026-10-05): margin median 1.69 / max 1.82 (US-NY-23); win prob median 0.003 / max 0.083; 418 of 418 races changed; seats 243.9 with it, 251.8 without
- Standard test (2026-10-05): FLAG — eve: improves log loss in 2 of 5 cycles, not a majority; effect reverses sign in close races.
- Challenged 2026-10-05 (pre-registered at b5a423d;
experiments/mean_bias_challengers_2026-10-05.md): no correction (H1) and a same-kind correction (H2) not promoted. The ledger leaves only 2016 citable (2018-2024 are burned), so neither could be promoted, and on 2016 both have worse seat error. The midterm-only generic-ballot miss (2.70, seven midterms) is the same size as the pooled one (2.62), so a same-kind correction leaves 2026 where it is.
house_fundamentals_in_national — House: fundamentals regression blended into the national term (active)
- What it does: Blends the bias-corrected generic ballot with a regression on midterm penalty, approval, income growth and the previous House vote, weighted by each one's past error.
- Where:
scripts/backtest_national.py:n1 and the N3 blend - Parameters: N1 coefficients refitted on 1960..t-2
- Evidence: experiments/national_environment.md
- Last tested by its own protocol: 2026-10-01. Protocol: Walk-forward national MAE 2004-2024 (N2 2.44, N3 2.16); not isolated at the district level.
- Today's effect (2026-10-05): margin median 0.56 / max 0.63 (US-NY-2); win prob median 0.001 / max 0.028; 418 of 418 races changed; seats 243.9 with it, 241.4 without
- Standard test (2026-10-05): FLAG — eve: improves log loss in 2 of 5 cycles, not a majority; effect reverses sign in presidential years.
house_nat_sd — House: national uncertainty (active)
- What it does: Every simulated House election draws one national environment around N3. Its spread is the national estimator's walk-forward error at the end of the campaign (2.75 points), plus the movement still to come before then and the extra measurement error of today's generic-ballot average beyond what past end-of-campaign averages had (3.48 points in all on 2026-10-11; house_ground_up 2.5.0, audit M12).
- Where:
scripts/gate_district_model.py:nat_sd;src/elab/models/district/model.py:simulate;scripts/forecast_house_ground_up.py:live_national_sd (the live spread: the walk-forward error plus the movement still to come and today's measurement excess) - Parameters: RMSE of N3's errors before the cycle; experiments/environment_drift_pre2026.json; today's generic-ballot estimation sd (data/national/inputs.json)
- Evidence: experiments/district_gate_preregistration.md; experiments/audit_m12_result.md (the live spread adds the movement still to come and today's measurement excess, house_ground_up 2.5.0; FM-195)
- Last tested by its own protocol: 2026-10-01. Protocol: Part of the district gate package; not isolated.
- Today's effect (2026-10-05): margin median 0.08 / max 0.57 (US-CA-30); win prob median 0.002 / max 0.020; 418 of 418 races changed; seats 243.9 with it, 243.1 without
- Standard test (2026-10-05): FLAG — eve: 80%/90% ranges cover 0.938/0.974 with it, 0.929/0.967 without.
house_scandal_rule — House: scandal rule (3.0 points against a listed nominee) (active)
- What it does: After a verified scandal about a House nominee (a criminal charge or conviction, a specific misconduct allegation, or a documented false claim about their own record, reported by two outlets doing their own reporting), the district's margin moves 3.0 points against that nominee from the event's first date; the spread is unchanged. The events are listed by hand in config/house_scandal_events_2026.yaml with their sources; party splits are not part of the rule. House model 2.7.0, adopted by the owner on 2026-10-22.
- Where:
src/elab/models/district/scandal.py;scripts/forecast_house_ground_up.py (scandal.shifts, from house_ground_up 2.7.0);config/house_scandal_events_2026.yaml - Parameters: 3.0 points (the Senate and governor size, shock_shift_result.md), persistent
- Evidence: experiments/house_scandal_preregistration.md; experiments/house_scandal/result.md
- Last tested by its own protocol: 2026-10-22. Protocol: Pre-registered House test 2018-2024: 12 scandal candidates, all finished below the forecast (mean 5.4 points; 3.3 without Collins); the 3.0-point shift improved CRPS -0.77 [-1.31, -0.18], Holm p 0.045. Also scored prospectively on 2026 after certification.
- Standard test: UNTESTABLE — Tested by its own pre-registered event test, not the walk-forward audit: the audit's arms have no event list.
house_state_sd — House: shared state error (active)
- What it does: Districts in the same state share an error, measured from walk-forward residuals about each cycle's mean; the cycle's own lean bias is drawn with the national shock every district shares (house_ground_up 2.10.0 on, the 'shared' arm of ELAB_DISTRICT_STATE_VAR).
- Where:
src/elab/models/district/model.py:ErrorModel.from_residuals;scripts/forecast_house_ground_up.py (STATE_VAR_ARM) - Parameters: walk-forward residuals 2014..t-2, about each cycle's mean (arm 'shared')
- Evidence: experiments/district_gate_preregistration.md; experiments/district_state_var_preregistration.md; experiments/district_state_var_result.md; docs/FAILURE_MODES.md FM-236
- Last tested by its own protocol: 2026-10-27. Protocol: Registered three-arm test on the district gate (experiments/district_state_var_preregistration.md): arm 'shared' adopted by its consistency rule (competitive Brier 0.1415 = 0.1415, coverage 4 = 4; experiments/district_state_var_result.md).
- Today's effect (2026-10-05): margin median 0.01 / max 0.06 (US-MO-3); win prob median 0.003 / max 0.024; 418 of 418 races changed; seats 243.9 with it, 243.0 without
- Standard test (2026-10-05): FLAG — eve: 80%/90% ranges cover 0.938/0.974 with it, 0.919/0.960 without.
- 2.6.0 (2026-10-12): state and district shocks drawn from Student-t (4 df) at 0.5 x the walk-forward sd, chosen by a pre-registered coverage test (experiments/house_spread_result.md): 80%/90% coverage 0.925/0.976 -> 0.836/0.935, Brier and seat intervals no worse.
house_idio_buckets — House: district error by race type (active)
- What it does: A district's own error is sized by its type: incumbent on the same lines (7.25), open seat (7.86) or new lines (8.11), rather than one size for all.
- Where:
src/elab/models/district/model.py:bucket, ErrorModel - Parameters: walk-forward residuals 2014..t-2
- Evidence: experiments/district_gate_preregistration.md
- Last tested by its own protocol: 2026-10-01. Protocol: Part of the district gate package; not isolated.
- Today's effect (2026-10-05): margin median 0.00 / max 0.01 (US-CA-36); win prob median 0.001 / max 0.014; 418 of 418 races changed; seats 243.9 with it, 243.5 without
- Standard test (2026-10-05): FLAG — eve: 80%/90% ranges cover 0.938/0.974 with it, 0.922/0.973 without.
- 2.6.0 (2026-10-12): state and district shocks drawn from Student-t (4 df) at 0.5 x the walk-forward sd, chosen by a pre-registered coverage test (experiments/house_spread_result.md): 80%/90% coverage 0.925/0.976 -> 0.836/0.935, Brier and seat intervals no worse.
house_nat_swing — House: district-specific response to the national swing (active)
- What it does: The lean model takes the national swing as a feature, so each district's response to a wave is learned rather than uniform. Off = uniform swing.
- Where:
scripts/backtest_district_model.py:feature_columns (nat_swing);src/elab/models/district/model.py:lean_grid - Parameters: learned
- Evidence: scripts/backtest_district_model.py (pre-registered docstring); experiments/district_gate_result.md
- Last tested by its own protocol: 2026-10-01. Protocol: Part of the district gate package; not isolated.
- Today's effect (2026-10-05): margin median 0.54 / max 3.67 (US-TX-16); win prob median 0.001 / max 0.082; 418 of 418 races changed; seats 243.9 with it, 243.8 without
- Standard test (2026-10-05): FLAG — eve: effect reverses sign in close races.
house_uncontested_rule — House: seats without a two-party contest (active)
- What it does: A district without both a Democrat and a Republican on the ballot is given, unsimulated, to the one major party with a nominee on record (qual_D or qual_R present); a district where neither side, or both, has a nominee on record has an unknown field and is simulated from its lean (FM-173; before, it went to the Republican). A seat declared as forecast from its own polls (AK-AL, CA-6) is never given away this way (2026-10-09: CA-6 was briefly counted as a certain Democratic seat).
- Where:
src/elab/models/district/model.py:split_one_party;scripts/forecast_house_ground_up.py (fixed_d, one_party) - Parameters: none
- Evidence: scripts/forecast_house_ground_up.py; docs/FAILURE_MODES.md FM-155, FM-173
- Last tested by its own protocol: never. Protocol: Never tested; the gate used actual winners for these seats instead.
- Standard test (2026-10-12): UNTESTABLE — These seats get no district forecast; the walk-forward gate assigned them from actual winners (an oracle), so there is no historical 'on' arm to score. Correctness under policy (a): the old rule made a seat with no nominee on record a Republican seat (LA-2 before FM-155 repaired Louisiana's rosters); fixed by FM-173, no effect on today's House (no such seat), +0.99 seats on the pre-FM-155 data.
house_district_polls — House: district poll update (v2, on) (active)
- What it does: Moves a polled district's lean toward its own polls: each pollster's latest poll, corrected for its sponsor class's measured lean (2018-2024), combined with the model's estimate by inverse variance with a shared cycle error.
- Where:
scripts/forecast_house_ground_up.py (on by default; --no-polls for 2.0.0);src/elab/models/district/polls.py (load_v2, shift_v2) - Parameters: sponsor bias N +1.1, D +3.3, R -0.9 (measured 2018-2024); shared cycle error
- Evidence: scripts/test_district_polls_v2.py; experiments/district_polls_v2_result.json; experiments/competitive_brier_2026-10-06.json; experiments/review_house_changes_2026-10-06.md; experiments/district_polls_review.md
- Last tested by its own protocol: 2026-10-06. Protocol: Pre-registered (scripts/test_district_polls_v2.py, committed before running): better lean MAE, Brier and seat error than the model alone in 2020, 2022 and 2024, polls as of four weeks out. A review by a separate AI model session on 2026-10-06 found the district poll update sound but flagged the age of the polls, calibration over the final thirty days and uncertainty in the sponsor correction; the review's 2026 variants give 235.7 to 241.5 Democratic seats, against 237.7 published.
- Its own test: Tested by its own pre-registered test.
- Standard test (2026-10-12): UNTESTABLE — Tested by its own pre-registered test (scripts/test_district_polls_v2.py); the adjustment-audit harness has no House district-poll arm.
house_rater_blend — House: rater blend (kept under an amended rule) (active)
- What it does: Moves each rated district's lean part of the way toward the average category of the race raters, by a weight fitted on past cycles (about 15% today). It was pre-registered to be kept only if it lowered both the Brier score and the seat error; it lowered the Brier score (0.0331 against 0.0353) but raised the seat error (11.76 against 11.18, all of it from 2022). The project owner approved keeping it on 2026-10-08 and re-confirmed it on 2026-10-09 after the test was re-run on the election-eve rating history; the cost is stated on /accuracy and in ADJUSTMENTS §2.5b.
- Where:
scripts/forecast_house_ground_up.py;src/elab/models/district/rater_blend.py - Parameters: weight w 0.149 in the live fit of 2026-10-28 (data/forecasts/ground_up_2026.json, rater_blend.w, written by scripts/forecast_house_ground_up.py on each House run; shown as 15%); category offsets from 2018-2024
- Evidence: experiments/house_rater_blend_preregistration.md; experiments/house_rater_blend_result.json; experiments/house_rater_blend_result.md (not adopted by its pre-registered rule; kept by the owner's amendment, adopted_by_amendment in the .json); experiments/house_rater_blend_v2_preregistration.md (the margin form, arm M, against the live blend B and no blend C0; scored on 2026 after certification)
- Last tested by its own protocol: 2026-10-09. Protocol: Pre-registered (experiments/house_rater_blend_preregistration.md, committed before running): keep only if lower Brier AND lower seat error than the model alone, 2018-2024 walk-forward. Result: lower Brier, higher seat error, so the rule as registered refused it. Kept by the project owner's amendment (2026-10-08, re-confirmed 2026-10-09), recorded in the result file as adopted_by_amendment; review point: the November 2026 House result, scored by the same rule.
- Its own test: Failed its pre-registered test; kept by the project owner's decision.
- Standard test (2026-10-12): UNTESTABLE — Tested by its own pre-registered test; the adjustment-audit harness has no House rater arm.
- 2026-10-28 (experiments/house_rater_blend_v2_preregistration.md, registered before any run): the live blend B against no blend C0 and a margin-form blend M (each rating read as a margin in its own year, minus that year's national estimate); 2012-2014 ratings added for a walk-forward 2016; decided on the district scores with seat-distribution scores and the mechanism check as guards; the 2026 arms written once at the final cutoff by job rater-v2-shadow and scored after certification. No setting changes before 2026-11-03.
house_rater_core — House: raters averaged over the core three (active)
- What it does: The rater average uses Cook, Sabato and Inside Elections only. Passed its pre-registered test against averaging every rater (Brier 0.0329 against 0.0331, seat error 11.66 against 11.76).
- Where:
src/elab/models/district/rater_blend.py (CORE);scripts/forecast_house_ground_up.py (RATERS_CORE) - Parameters: Cook, Sabato's Crystal Ball, Inside Elections
- Evidence: experiments/house_rater_core_preregistration.md; experiments/house_rater_core_result.json
- Last tested by its own protocol: 2026-10-09. Protocol: Pre-registered (experiments/house_rater_core_preregistration.md): passed as registered.
- Its own test: Tested by its own pre-registered test.
- Standard test (2026-10-12): UNTESTABLE — Tested by its own pre-registered test; the adjustment-audit harness has no House rater arm.
house_poll_prior_from_blend — House: district poll update weighs polls against the blended lean (active)
- What it does: When a district has polls, they are combined with the rater-blended lean using that blended lean's own measured error, not the unblended model's. Passed its pre-registered test (arm B): mean seat error 12.16 against 13.47 for the blend alone, Brier 0.112 against 0.121.
- Where:
scripts/forecast_house_ground_up.py (POLL_PRIOR_FROM_BLEND) - Parameters: prior variance from the blended model's walk-forward residuals
- Evidence: experiments/blend_plus_polls_preregistration.md; experiments/blend_plus_polls_result.json
- Last tested by its own protocol: 2026-10-09. Protocol: Pre-registered (experiments/blend_plus_polls_preregistration.md): arm B passed as registered and replaced arm A.
- Its own test: Tested by its own pre-registered test.
- Standard test (2026-10-12): UNTESTABLE — Tested by its own pre-registered test; the adjustment-audit harness has no House arm for it.
house_poll_age_decay — House: district polls discounted by their age (kept under an amended rule) (active)
- What it does: Each district poll counts for less the further it was taken from election day: full weight within 60 days, falling steadily to half at 210 days and a quarter beyond, and never more for an older poll. This is the owner's judgement that equal weighting of old and new polls is contrary to what is known about polling, adopted by an amendment written before its backtest; the walk-forward backtest on 2016-2024 found it not harmful (squared error -0.58 against the previous rule, better at 120, 90 and 60 days, neutral in the final month).
- Where:
src/elab/models/district/polls.py (age_factor, shift_v2 election_day);scripts/forecast_house_ground_up.py (every house_ground_up version from 2.9.0 on) - Parameters: factor 1 to 60 days, linear to 0.5 at 210 days, 0.25 beyond (fixed, not fitted)
- Evidence: experiments/house_poll_rules_v2_preregistration.md; experiments/house_poll_rules_v2_result.json; experiments/house_poll_age_measured_preregistration.md (the measured loss of accuracy with age, arm DRIFT, against the published schedule; scored on 2026)
- Last tested by its own protocol: 2026-10-26. Protocol: Owner's amendment to experiments/house_poll_rules_v2_preregistration.md (written before the rerun): adopted unless clearly harmful (difference above 0 with p < 0.05); result -0.58, 95% -2.70 to +1.10, adopted.
- Standard test: UNTESTABLE — Tested by its own pre-registered backtest; the adjustment-audit harness has no House arm for it.
- 2026-10-28 (experiments/house_poll_age_measured_preregistration.md, registered before any 2026 result): from 2,296 whole-sample district polls of 2004-2024 the miss grows with age, E[miss^2] = 52.17 + 0.1149 x days (95% for the slope 0.023 to 0.239), a weight relative to an eve poll of about 0.79 at 120 days and 0.68 at 210 against the schedule's 0.80 and 0.50; arms DECAY (published), LIVE (equal weights) and DRIFT (measured), rebuilt from the sealed bundles of the last House run on or before 2026-10-27 and the last before 2026-11-03; scored on 2026 after certification.
house_own_polls_spread — House: spread of the seats forecast from their own polls (AK-AL, CA-6) (active)
- What it does: AK-AL (Begich, R, against Hill, independent; ranked choice) and CA-6 (Pan, D, against Kiley, no party) are forecast from polls of their own pair, which the House model's Democrat-against-Republican frame cannot represent. The spread is the own-polls centre's measured past error, the walk-forward root mean square of its misses on every earlier polled district-cycle, drawn with the House model's Student-t shape with four degrees of freedom; for ranked choice the ranking error is added. ELAB_OWN_POLLS_SPREAD=model restores the House model's full error, normal (house_ground_up 2.10.0).
- Where:
src/elab/models/own_polls.py (spread_mode, measured_sd, T_DF, unit_t; ELAB_OWN_POLLS_SPREAD, default measured_t);scripts/forecast_house_ground_up.py (house_ground_up 2.11.0 and later) - Parameters: size 8.41 points for 2026 (W1.sd_2026 of experiments/own_polls_shape_rebuilt_panel.json, measured on 644 earlier polled district-cycles); unit-variance Student-t, 4 degrees of freedom; ranked choice adds the ranking error, 5.05 points, normal (elab.models.ranked_choice.RANKING_SD)
- Evidence: experiments/own_polls_band_result.md (the measured spread with a normal shape: missed its 80% limit by 0.004); experiments/own_polls_shape_preregistration.md; experiments/own_polls_shape_result.md; experiments/own_polls_shape_rebuilt_panel.md (rerun on the rebuilt House panel); docs/FAILURE_MODES.md FM-247; experiments/validation_2026/house_components_2026_preregistration.md (scored on 2026)
- Last tested by its own protocol: 2026-10-28. Protocol: Registered shape test (experiments/own_polls_shape_preregistration.md, disclosed as a second look at the rows of the first measured-spread test): on the polled district-cycles of 2018-2024, 90% coverage in [0.86, 0.94], 80% coverage in [0.75, 0.85], log loss not worse than the published spread by more than 0.002, Brier not worse by more than 0.0005, every cycle's 90% coverage at least 0.80. Rerun on the rebuilt House panel, 406 rows: 0.901, 0.768, log loss 0.4491 -> 0.4175, Brier 0.1468 -> 0.1362, cycles 0.867-0.947; all five pass (the published full error covered 0.983 at 90%). In house_ground_up 2.11.0, published as 2.12.0 on 2026-10-28.
- Its own test: Tested by its own pre-registered test.
- Standard test: UNTESTABLE — the adjustment-audit harness has no own-polls arm; tested by its own registered backtest.
religion_vintage — House and statewide prior: one religion census for every election (active)
- What it does: The religion features of the district panel (rel_*) come from the 2010 U.S. Religion Census for every cycle from 2012 to 2026, so the House lean and the statewide ground-up prior train and predict on one study. The 2020 study counts some groups on another basis (its Muslim and mainline counts: ME-1 rel_muslim 0.0014 against 0.0175), so giving it to 2024 and 2026 only made a change of instrument read as a change in the electorate. ELAB_RELIGION_VINTAGE=as_published restores the 2020 study for 2024 and 2026 (house_ground_up 2.11.0, with the rows rebuilt).
- Where:
src/elab/panel/religion.py (study_for, vintage_mode; ELAB_RELIGION_VINTAGE, default 2010);scripts/add_panel_religion.py, scripts/check_panel_religion.py;scripts/forecast_house_ground_up.py (religion_rows_problem: a run is stored only when the rows carry the study the code names; house_ground_up 2.12.0) - Parameters: the 2010 study for every cycle; three Alaska areas and one South Dakota county are not in the study
- Evidence: experiments/religion_vintage_preregistration.md; experiments/religion_vintage_result.md; experiments/religion_vintage_rebuilt_panel.md (rerun on the rebuilt House panel); docs/FAILURE_MODES.md FM-257
- Last tested by its own protocol: 2026-10-28. Protocol: Registered as a correction judged by non-inferiority on 2024, the only cycle the defect touches (experiments/religion_vintage_preregistration.md): House Brier and margin MAE, statewide blend Brier and margin MAE, each not worse than its limit. Rerun on the rebuilt House panel: House Brier 0.01982 -> 0.01946, MAE 3.565 -> 3.589 (limit +0.05), blend Brier 0.02554 -> 0.02562 (limit +0.0005), blend MAE 3.124 -> 3.106; all four pass, and the rule lets the consistent input win the tie. house_ground_up 2.12.0, published 2026-10-28.
- Its own test: Tested by its own pre-registered test.
- Standard test: UNTESTABLE — an input correction, tested by its own registered comparison; the adjustment-audit harness has no arm for it.
money_filing_dates — Training money dated by when each filing was received (2012 and 2014) (active)
- What it does: Outside, party and PAC money in the training rows counts only spending in filings the FEC had received by 20 September of the cycle (point-in-time money, FM-182). Before mid-2015 an FEC image number has eleven digits and no date, and the receipt check compared it as text, so nearly every 2012 and 2014 filing counted (pro-Democratic outside money $34.7M in 2012 and $34.9M in 2014, against $0.6M and $2.2M when dated). An eleven-digit filing is dated by its filing number's receipt date (OpenFEC /v1/filings/, cached in data/cache/fec_filing_receipts.json). The House lean and, through the 2012 and 2014 Senate money, the statewide ground-up prior train on these rows.
- Where:
src/elab/panel/money.py (the receipt check; receipts cached under data/cache/, not in the repository);scripts/add_panel_money.py --cycle 2012 --cycle 2014 - Parameters: filings received by 20 September of the cycle; transactions before 1 October
- Evidence: experiments/h128_money_and_history_preregistration.md; experiments/h128_money_and_history_result.md
- Last tested by its own protocol: 2026-10-28. Protocol: Registered before any run (experiments/h128_money_and_history_preregistration.md), measured on copies of the live database: House gate selection Brier 0.03681 -> 0.03618, seat error 12.20 -> 11.87, wrong winners 20.0 -> 18.25, backtest M2 MAE 6.160 -> 5.964, selection rule and 2024 confirmation passed; statewide gate passed; blend 0.8.7 record (256 races) log loss 0.1634 -> 0.1630, Brier 0.0514 -> 0.0513, 80%/90% coverage 0.832/0.930 -> 0.828/0.930. Every condition passed; applied live on 2026-10-28 and the House gate rerun on the live rows (PASSED).
- Its own test: Tested by its own pre-registered test.
- Standard test: UNTESTABLE — an input correction, tested by its own registered comparison; the adjustment-audit harness has no arm for it.
house_district_poll_shared — House: a shared district-poll term (built, off; scored on 2026) (inactive)
- What it does: Moves the national estimate, and so every district, by the average gap between the sponsor-corrected district polls and the model's baseline lean in the polled districts (S_raw), shrunk by kappa = nat_sd^2 / (nat_sd^2 + s_c2 + v / n). Today each polled seat moves by its own polls; unpolled seats and the national term do not. ELAB_DISTRICT_POLL_SHARED=1 turns it on; it is off in every published run.
- Where:
src/elab/models/district/polls.py (shared_on, shared_term);scripts/forecast_house_ground_up.py (ELAB_DISTRICT_POLL_SHARED, default off);scripts/shadow_district_poll_shared.py (job h86-shadow) - Parameters: off. At the frozen standpoint of 2026-10-28 (run d641b8d4): S_raw 2.14 points toward Republicans over 97 polled districts, kappa 0.919, national D+6.91 -> D+4.94, P(D majority) 87.4% -> 81.8%, IA-2 and NY-17 favourites turn
- Evidence: experiments/district_poll_shared_preregistration.md; experiments/district_poll_shared_result.md; experiments/district_poll_shared_2026_prospective_preregistration.md; experiments/district_poll_shared_2026_frozen_20261028.json
- Last tested by its own protocol: 2026-10-27. Protocol: Registered (experiments/district_poll_shared_preregistration.md) and passed its four conditions on 2020-2024: pooled Brier 0.02699 -> 0.02566, better in each cycle; 2024 margin MAE worse. Held off before the election (on copies the House majority chance moved 87.8% -> 82.3%) and registered as a prospective test (experiments/district_poll_shared_2026_prospective_preregistration.md): PUB against SH, both rebuilt from one run's sealed bundle, frozen 2026-10-28 and at the final cutoff (job h86-shadow); used from 2028 only if the pooled 2020-2026 Brier is lower, the pooled log loss is not worse by more than 0.002, the 2026 Brier is not worse by more than 0.002 and the four-cycle seat error is not worse by more than one seat.
- Its own test: Tested by its own pre-registered test.
- Standard test: UNTESTABLE — the adjustment-audit harness has no House district-poll arm; tested by its own registered backtest and scored prospectively on 2026.
turnout_models — Turnout and enthusiasm (not modelled; two House turnout models rejected) (rejected)
- What it does: No published forecast estimates who will turn out in November or how enthusiastic each side is. That enters only indirectly: through the polls (likely-voter screens are the pollsters' own turnout models), past results, demographics, the national environment, and an error band measured on past elections whose turnouts varied. Two bottom-up turnout models were tested for the House and not adopted: one built from each district's demographic groups made the seat count and the probabilities worse; one built from a large voter survey was no better than noise and worse in 2024. Both counted Hispanic voters inside other groups too, and the first used a 2021 estimate of how groups voted for its 2018 and 2020 tests, so they rule out those two designs, not turnout modelling in general (model audit 2026-10-10, M01, M04-M06).
- Where: -
- Parameters: n/a
- Evidence: experiments/electorate_backtest_result.md; experiments/ces_bottom_up_result.md
- Last tested by its own protocol: 2026-10-10. Protocol: Ecological: 2022 Brier 0.0363 -> 0.0422, expected-seat error 7.5 -> 12.5 (lean MAE 5.93 -> 5.25). Survey-based: selection Brier 0.03621 -> 0.03560 with a bootstrap interval including zero; 2024 Brier 0.0218 -> 0.0228. Neither adopted.
- Standard test (2026-10-12): UNTESTABLE — Not in the published path (rejected); no turnout model is published.
National environment (all chambers through N3)
generic_ballot_house_effects — Generic ballot: each pollster's lean removed before averaging (active)
- What it does: Before the national generic-ballot polls are averaged, each firm's lean is measured against the other firms' polls taken at the same time, shrunk toward zero when the firm has few polls (a firm with four keeps half its lean), and the leans are centred so the typical firm counts as zero. A firm that polls every week at a lean then no longer pulls the average toward itself. The industry-wide miss is untouched by this and stays with the generic-ballot bias correction. Walk-forward 2004-2024 it lowered the squared error of the corrected average (7.84 against 8.82) and was better in 8 of 11 cycles at 31 and 1 days.
- Where:
src/elab/models/fundamentals/environment.py (house_effects, remove_house_effects);scripts/build_national_inputs.py:main (generic_now in data/national/inputs.json, which the House model's national term reads; environment_series, the /environment weekly series);scripts/forecast_house.py (the house_prior path, kept for comparison; a --persist run of it refuses while house_ground_up is the champion) - Parameters: shrinkage k/(k+4); consensus over the 60 days before each poll, that firm left out; equal-firm centring; cycle polls from 1 January of the year before
- Evidence: experiments/generic_house_effects_preregistration.md; experiments/generic_house_effects_result.json
- Last tested by its own protocol: 2026-10-26. Protocol: Registered walk-forward test 2004-2024 (arms A plain, B centred house effects, C anchored), decision rule: B replaces A if lower MSE and better in at least 7 of 11 cycles at 31+1 days; B passed (8 of 11; bootstrap over cycles -0.98, 95% -1.78 to -0.44).
- Standard test: UNTESTABLE — Tested by its own pre-registered backtest; the adjustment-audit harness has no national-average arm for it.
4. The latest audits
4.1 Senate and governor, 2026-10-12
Full report: experiments/adjustment_audit_2026-10-12.md and .json (commit 58513b8, 51 minutes on
Skynet-Three, nothing persisted). Run with --no-house: it re-tests the Senate and governor terms
on the specification as published after audits M07 and M08 -- the 0.8.2 ground-up blend, the
total-vote target (D% - R% of all votes cast) and the overlapping-waves rule. House and chamber
terms keep their 2026-10-05 verdicts (4.2). Corpus: 458 races at each standpoint, 375 scoreable on
the eve and 358 four weeks out. The engine's "on" arm matches the stored backtests to a median 0.03
points and today's 60 published poll-based forecasts to a median 0.04 (largest gap 0.80).
| adjustment | verdict | eve | four weeks out | today: margin shift median / max | today: win-prob shift median / max |
|---|---|---|---|---|---|
| fundamentals_prior | keep | +0.0012, 8/10 | +0.0018, 8/10 | 0.08 / 6.45 | 0.002 / 0.087 |
| sparse_blend | keep | +0.0396, 9/9 | +0.0620, 10/10 | 0.00 / 11.71 | 0.000 / 0.175 |
| scale_slope | keep | +0.0052, 8/10 | +0.0056, 8/10 | 0.42 / 1.25 | 0.007 / 0.020 |
| scale_intercept | flag | -0.0075, 6/10; reverses (presidential) | -0.0034, 6/10; reverses (presidential) | 1.38 / 1.74 | 0.026 / 0.117 |
| scale_law (both parts) | flag | -0.0028, 6/10; reverses (presidential, lopsided) | +0.0016, 5/10; reverses (midterm, close) | 1.30 / 2.94 | 0.024 / 0.119 |
| statewide_groundup_blend | flag | +0.0060, 5/5; reverses (governor, lopsided) | +0.0113, 4/5; reverses (governor) | 1.39 / 5.65 | 0.022 / 0.274 |
| student_t_shocks | flag | +0.0016, 5/10; reverses (lopsided) | +0.0029, 7/10; reverses (presidential) | 0.05 / 0.28 | 0.007 / 0.040 |
| sponsor_handling (keep sponsored polls) | flag | -0.0022, 6/8; reverses (presidential, governor, lopsided) | +0.0049, 6/8; reverses (midterm) | 0.18 / 27.72 | 0.005 / 0.395 |
| pollster_house_effects | flag | +0.0010, 6/10; reverses (presidential) | -0.0004, 5/10; reverses (midterm, governor, lopsided) | 0.04 / 0.45 | 0.001 / 0.024 |
| sigma_race | flag (coverage) | 80/90% cover 0.824/0.920 (0.528/0.645 without) | 0.855/0.911 (0.707/0.841 without) | 0.42 / 1.88 | 0.027 / 0.108 |
| sigma_nat | flag (coverage) | 0.824/0.920 (0.795/0.904 without) | 0.855/0.911 (0.844/0.902 without) | 0.10 / 0.42 | 0.005 / 0.021 |
| drift_law | flag (coverage) | (no effect on the eve) | 0.855/0.911 (0.791/0.885 without) | 0.22 / 0.96 | 0.012 / 0.050 |
| sparse_measured_sd | flag (coverage) | 0.824/0.920 (0.808/0.912 without) | 0.855/0.911 (0.838/0.908 without) | 0.00 / 6.38 | 0.000 / 0.097 |
| design_effect | flag (coverage) | 0.826/0.918 (the same without: passes) | 0.853/0.909 (0.853/0.904 without) | 0.07 / 0.63 | 0.001 / 0.024 |
Log-loss terms: gain (log loss without minus with; positive helps), then cycles improved. Spread-only terms are judged under amendment 1 (2.5a) on interval coverage.
What changed since 2026-10-05, and what it means. These are recommendations to the owner; nothing has been changed.
- scale_intercept: flagged again, and the owner's decision is open (audit M11,
experiments/audit_m11_result.md). With it, pooled log loss is 0.1604 against 0.1529 without on the eve. It hurts midterms (-0.020, 217 races) and helps presidential years (+0.010, 158 races), and 2026 is a midterm. Today it moves every Senate and governor race toward the Republican: 1.38 points in the median race, 1.74 at most; win probabilities move 2.6 points in the median race and 11.7 at most. The owner's options: keep it as now (disclosed with today's effect); a kind-specific intercept, fitted only on earlier cycles of the same kind, registered now and judged prospectively; or remove it (slope only), which the pooled log loss favours. The slope passes and stays whatever is decided. - The ranges are now somewhat wide on the total-vote target. Four weeks out, 80% ranges hold 85.5% of results and 90% ranges 91.1%. Under 2.5a's first condition (each coverage within 0.03 of nominal) that fails, so every spread-only term that changes a forecast four weeks out is flagged, including sigma_race, without which coverage collapses to 0.707/0.841. Removing any one term is not the remedy; the next step is a pre-registered recalibration of the statewide spread on the total-vote target, the same remedy 4.2 gives for the House.
- student_t_shocks passed on 2026-10-05 and now fails: on the eve it improves 5 of 10 cycles (a majority needs 6) and reverses in lopsided races; four weeks out it reverses in presidential years. Its pooled gain is positive at both standpoints and its effect small (median 0.05 points). Recommendation: no change.
- statewide_groundup_blend, sponsor_handling, pollster_house_effects, scale_law: the same verdicts as on 2026-10-05, for the same reasons (4.2).
4.1a Follow-up tests of the spread (2026-10-12)
Both recalibrations recommended in 4.1 (statewide) and 4.2 (House) were pre-registered and run the same day.
- Statewide (
experiments/statewide_spread_result.md): the spread terms re-estimated on today's data are wider (2026 sigma_nat 2.44 -> 3.45) and move coverage further from nominal; a walk-forward drift multiplier does not reach the required improvement. Nothing changed. The excess four-week coverage comes from calm recent cycles (2018-2024); 2014 and 2020 under-covered. - House (
experiments/house_spread_result.md): adopted ashouse_ground_up2.6.0. The state and district shocks are Student-t (4 df) at a scale fitted walk-forward to the ranges' coverage (0.5 times the walk-forward residual sd for 2026). On 2018-2024, 80%/90% coverage moves from 0.925/0.976 to 0.836/0.935, Brier from 0.0332 to 0.0329, and the seat intervals are no worse. Today: 232.4 expected Democratic seats (236.2 before), P(D majority) 0.831 (0.894).
4.2 The first full audit, 2026-10-05 (still the latest for House and chamber terms)
Full report: experiments/adjustment_audit_2026-10-05.md (per-cycle tables for every adjustment at
every standpoint, subgroups, races dropped and why) and .json. The run's header names commit
7f7741d (the pre-registration) because the harness was not yet committed; the harness is commit
8e93e76, identical in its arithmetic to what ran (that commit adds only quieter worker logs and
--check-race). Thirty minutes on Skynet-Three:
3,860 walk-forward fits in a process pool, nothing persisted. The corpus is 459 Senate and governor
races from 2006-2024 at each standpoint (377 scoreable on the eve, 359 four weeks out; 78 are out of
scoring because elab.evaluate.scored refuses them, 77 of those for an uncertified result) and
1,971 contested House districts from 2016-2024.
The engine reproduces the published numbers. Its "on" arm matches today's stored Senate and governor forecasts to a median 0.002 points (98% within 0.1; largest gap 0.51) and today's House file exactly (418 districts, 243.9 seats, P(D majority) 0.984). Against the stored backtests it matches to a median 0.03 points; of the 69 races more than 0.1 apart, 42 (and all of the largest, up to 4.1) are thin sparse-blend races whose stored backtests predate the 2026-09-29 refit of the sparse-poll error and the poll backfills since. The audit scores today's specification on today's data, as it should.
Verdicts
| adjustment | verdict | eve: gain, cycles improved | four weeks out: gain, cycles improved | today: margin shift median / max | today: win-prob shift median / max |
|---|---|---|---|---|---|
| fundamentals_prior | keep | +0.0015, 8/10 | +0.0026, 8/10 | 0.10 / 6.94 | 0.002 / 0.082 |
| sparse_blend | keep | +0.0334, 8/8 | +0.0568, 10/10 | 0.00 / 11.78 | 0.000 / 0.172 |
| student_t_shocks | keep | +0.0022, 8/10 | +0.0030, 8/10 | 0.05 / 0.28 | 0.003 / 0.023 |
| scale_slope | keep | +0.0054, 8/10 | +0.0068, 7/10 | 0.43 / 1.26 | 0.007 / 0.019 |
| scale_intercept | flag | -0.0065, 6/10; reverses (presidential) | -0.0025, 6/10; reverses (presidential, Senate) | 1.37 / 1.74 | 0.028 / 0.113 |
| scale_law (both parts) | flag | -0.0016, 6/10; reverses | +0.0037, 6/10; reverses (midterm) | 1.31 / 2.85 | 0.028 / 0.115 |
| statewide_groundup_blend | flag | +0.0078, 5/5 (passes) | +0.0108, 3/5; reverses (governor, lopsided) | 1.46 / 6.21 | 0.019 / 0.305 |
| sponsor_handling (keep sponsored polls) | flag | -0.0020, 6/8; reverses | +0.0040, 6/8; reverses (midterm) | 0.97 / 27.53 | 0.021 / 0.399 |
| pollster_house_effects | flag | +0.0016, 4/10 | +0.0001, 5/10; reverses | 0.04 / 0.84 | 0.001 / 0.036 |
| design_effect | flag | -0.0009, 3/10 | -0.0023, 3/10 | 0.06 / 0.73 | 0.001 / 0.021 |
| sparse_measured_sd | flag | -0.0011, 0/3 | -0.0014, 0/3 | 0.00 / 7.44 | 0.000 / 0.105 |
| sigma_nat | flag | -0.0025, 2/10 | -0.0027, 3/10 | 0.08 / 0.46 | 0.005 / 0.020 |
| sigma_race | flag | +0.0137, 5/10; reverses | -0.0095, 3/10; reverses | 0.36 / 2.04 | 0.023 / 0.106 |
| drift_law | flag | (no effect on the eve) | -0.0080, 3/10 | 0.21 / 1.22 | 0.014 / 0.057 |
| house_national_adjustments (N3 vs face value) | flag | +0.0049, 2/5 | - | 2.02 / 2.14; seats 243.9 -> 253.5 | 0.004 / 0.098 |
| house_gb_bias_correction | flag | +0.0014, 2/5; reverses (close races) | - | 1.69 / 1.82; seats 243.9 -> 251.8 | 0.003 / 0.083 |
| house_fundamentals_in_national | flag | +0.0002, 2/5; reverses | - | 0.56 / 0.63; seats -> 241.4 | 0.001 / 0.028 |
| house_nat_sd | flag | -0.0020, 0/5 | - | 0.08 / 0.57 | 0.002 / 0.020 |
| house_state_sd | flag | -0.0035, 0/5 | - | 0.01 / 0.06 | 0.003 / 0.024 |
| house_idio_buckets | flag | -0.0019, 2/5 | - | 0.00 / 0.01 | 0.001 / 0.014 |
| house_nat_swing | flag | +0.0002, 2/3; reverses (close races) | - | 0.54 / 3.67 | 0.001 / 0.082 |
Gain is log loss without minus with, per race (positive: the adjustment helps). Four of 21 tested adjustments pass the standard. Nine entries cannot be tested (section 3 gives each reason).
What the flags mean, and the recommendation for each
These are recommendations to the coordinator. Nothing has been changed; any change goes through a pre-registration under policy (b).
- scale_intercept: fails, and the failure is the mean-bias contradiction. Pooled log loss is
worse with it at both standpoints. By kind of year it helps in presidential years (+0.0101 on the
eve, five cycles) and hurts in midterms (-0.0186, five cycles; 2014 -0.043, 2022 -0.038). 2026 is
a midterm. Today it moves every Senate and governor race 1.37 points toward the Republican
(median; 1.74 before the ground-up blend dilutes it) and win probabilities by up to 11 points.
This agrees with entry 1 (
experiments/scale_intercept_check_2026-10-05.md, CRPS protocol). Recommendation: pre-register a test of the slope alone (and, as the alternative entry 1 proposes, a same-kind intercept) against the current law, decided under this standard. The slope passes and should stay whatever is decided about the intercept. - The variance terms (sigma_nat, sigma_race four weeks out, drift_law, design_effect, sparse_measured_sd, house_nat_sd, house_state_sd, house_idio_buckets) fail in the same way: per-race log loss prefers narrower distributions. Interval coverage with them is on target for Senate and governor (90% intervals cover 0.905 on the eve and 0.900 four weeks out; without the drift law 0.864, without sigma_nat 0.891), so removing them would buy log loss with overconfident intervals. The House is different: its 90% intervals cover 0.974 and its 80% intervals 0.938, so the House district error model is genuinely too wide. Recommendation: no removal. Note the limitation of the standard as registered: it judges every adjustment on log loss, while ADR-0016 says a change to a distribution's spread should be judged on CRPS and coverage; amending the standard to do that for spread-only terms is a specification change for the coordinator to decide under (b), not something to adopt after seeing these results. Separately, pre-register a recalibration test of the House district error model (state and idiosyncratic terms).
- statewide_groundup_blend: passes on the eve (5 of 5 cycles), fails four weeks out on two near-zero subgroup reversals (governor -0.0001, lopsided races -0.0023), with its gains concentrated in presidential years (2020 +0.078). Its prior's national term is N3, which carries the generic-ballot bias correction (contradictions below). Recommendation: keep; include it in the midterm-bias test of item 1, since part of what it adds is the same level correction.
- House national term. N3 beats the generic ballot at face value overall (+0.0049) but only in 2 of 5 cycles: it helps in presidential years and hurts in both midterms (2018, 2022). The generic-ballot bias correction alone is worth 7.9 seats today (243.9 with it, 251.8 without) and hurts close races. Recommendation: the same question as item 1 on the House side: pre-register a test of the bias correction in midterm years. Five House cycles (two midterms) is thin; say so.
- sponsor_handling. Keeping sponsored polls is worse on the eve (driven by 2022, -0.032) and better four weeks out (+0.004); neither standpoint passes. Dropping them would move today's South Dakota race 27.5 points (its only polls are sponsored). Recommendation: no change; if this is revisited, test a sponsor-lean term rather than exclusion.
- pollster_house_effects, house_fundamentals_in_national, house_nat_swing. Effects at noise level (|gain| under 0.002; today's median margin shift 0.04 to 0.56). Recommendation: no action; they fail the majority rule by a coin's width.
Contradictions found
Opposite conclusions about the same quantity (the check in 2.6; seven quantities):
- Mean polling bias. Corrected three times in the published path (the scale intercept, -1.744;
the generic-ballot bias correction in N3, used by the House model and inside the statewide
ground-up prior; and the ground-up blend, whose gain the gate attributes to "correcting the 2020
polling miss") while
mean_bias_correction.mdandbias_centre.mdconcluded it must not be, and the methodology page tells readers "the lean is therefore reported, not subtracted" (src/elab/registry/calibration.py:CORRECTION_ATTEMPTS). The public sentence is inaccurate while the intercept is applied; correcting it is a communications fix the coordinator should make whatever the model decision. - Generic-ballot bias:
national_environment.md(supported) againsthouse_generic_ballot_bias.md(refused), the same day, neither citing the other. - An explicit fundamentals/poll blend: ADR-0007 rejects the knob; the sparse blend and the statewide ground-up blend are both such blends, the latter on top of a fit that already uses the fundamentals prior (prior information possibly counted twice).
- Design effect: 1.6 in code; measured near one (
poll_dispersion*.json). The audit agrees with the measurement (removing it does not hurt). - Sparse-poll error inflation: shipped as a fix (FM-104) after the same change was refused as challenger c9; the audit reproduces the refusal on log loss (0 of 3 cycles) while coverage favours the fix.
- Fitting at three polls: c6 passed; the published path still requires four.
- Publishing unpolled races from a prior: FM-140 publishes them; c18 refused an unpolled prior.
Inconsistent estimates that are not opposite conclusions (sigma_nat quoted as 2.60, 2.49, 2.48 and 2.44 in different files; sigma_race as 4.94 to 5.23; the realised sparse-poll error as 8.44, 8.92 and about 9.98) are recorded in the audit's working notes and are not flagged by the check.
Correctness findings under policy (a)
- House: the uncontested-seat rule gave LA-2, a Democratic seat, to the Republican because
neither candidate's quality record was known (
forecast_house_ground_up.py,fixed_d). That is a check of an adjustment's application, not a specification question. Follow-up (2026-10-05): the data side had already been repaired by FM-155 (LA-2 has been simulated, D >99%, since 2026-10-03), so this note was stale for the published number; the rule itself still defaulted a seat with no nominee on record to the Republican and is fixed by FM-173 (section 5). - Texas governor, re-derived step by step with
--check-race US-TX governor: reproduced -5.99 against the stored -5.97 (P 0.181 both). The scale intercept moves it 1.40 points; sponsored polls 0.36; the ground-up blend 0.41. The registered adjustments were applied correctly; the number's size is a specification question (item 1), not a defect.
Could not be tested, and why
poll_selection_asof (a knowability rule: switching it off is leakage); lv_rv_handling (nothing is applied; 64% of historical polls record no population); min_polls_rule (a threshold with no off state); prior_only_publication (switching it off unpublishes 12 races rather than changing them); chamber_correlation (one chamber outcome per cycle); house_uncontested_rule (the gate assigned those seats from actual winners); and the inactive or rejected entries (shock rules, district polls, the mean-bias correction itself).
5. Verified-bug fixes
- FM-173 (2026-10-05): House one-party rule. A seat with no nominee on record
for either side was an automatic Republican seat; it is now simulated from its lean
(
split_one_party, shared by the published forecast, the audit and the counterfactual script). Effect today: none (no such seat). On the data that produced the LA-2 defect: 242.90 -> 243.89 Democratic seats, P(D majority) 0.980 -> 0.984.