The statistical model
Every distribution, every prior and every error term, with the assumption behind each stated beside it.
STATISTICAL MODEL
This document specifies the mathematics. It is written for a statistician who has not read the code and who intends to find the mistakes. Every constant that appears below is either estimated from data, read from configuration, or accompanied by a justification. There are no unexplained magic numbers.
Status: fitted, and validated where it says so. The model is fitted, walk-forward backtested and scored against certified results. The current model's scored record, and its comparison with a recency- and size-weighted polling average, are on the Accuracy page.
What remains unvalidated is named where it arises, not hedged here. The cross-race posterior
predictive check passes on all six cycles; the tightest is 2014, at tail p = 0.089 against a
next-closest 0.110 (experiments/phase5_exit_met.md). sigma_region fits at 0.00, and that is a bounded claim: measured on the real skeleton, the estimator finds a true 2.0-point regional term 98% of the time and a 1.0-point one 38% of the time, so a regional term of 2.0 points or more is ruled out and one of 1.0 or less is undetectable here. The forecast's own regional signal, debiased for the noise on each regional mean, is 0.72 points across all races and 0.00 among well-polled ones — below the floor, so the fitted zero and the slice's 3.33-point spread agree. A spread over four groups is about twice a standard deviation and each regional mean carries ±1.2 points of noise, which is where the apparent contradiction came from (experiments/region_tension.md). sigma_office fits at 0.00 on the same estimator and its floor also lands at 2.0 points, but more weakly — 82% detection at a true 2.0 against the regional 98% — so it bounds less. Where a parameterisation below is still a competing
candidate rather than a decision, it says so in place.
0. What is implemented, and what below is a proposal
The fitted statewide poll layer is src/elab/models/latent/single_race.py. It reads four things
per poll -- pollster, field midpoint, the raw two-candidate margin, sample size -- and fits one
scalar margin per race. It does not read population, polling mode, sponsorship or undecided
shares, and it does not share a latent trend across races. The House is a separate model. Which
polls are read is decided before the fit: the knowability rule, one version per release, and one
wave per firm where a firm's waves overlap (ObservationRepository.for_race). The table below is
the authoritative list. Read every later section against it; the active adjustments, with their
evidence and today's effect, are on the site's adjustments page
(data/model/adjustments.json).
| Term (section) | Status | Where |
|---|---|---|
| Latent random walk per race (§3) | implemented: one independent scalar random walk per race | single_race.py (drift_sd, innovations) |
| National latent environment η(t) and race elasticity β_r (§3) | not implemented: each race's random walk is independent | -- |
| Additive log-ratio / centred-log-ratio scale (§2) | not implemented: the fit works directly on the two-candidate margin | PollInput.margin |
| Pollster house effect h_p, sum-to-zero by volume (§4, §5) | implemented | single_race.py (house) |
| Pollster grouping by mode and partisanship (§6) | not implemented: house effects share one prior and one dispersion; excess variance shares one mean and one dispersion | single_race.py (house_sd, mu_tau, sigma_tau) |
| Pollster excess variance τ²_p, hierarchical (§4.1, §6) | implemented | single_race.py (mu_tau, tau_raw) |
| Student-t poll likelihood, ν estimated (§4.1) | implemented | single_race.py (nu) |
| Design effect on sampling error (§4.1) | implemented as one measured factor, walk-forward from the dispersion law (0.757 for 2026; 1.6 where no earlier cycles are available) | adjustments: design_effect; experiments/poll_dispersion_pre<cycle>.json |
| Field-period weighting (§4) | not implemented: plain field midpoint | PollInput.field_midpoint |
| Polling-mode effect m_mode (§4) | not implemented: mode is not read | -- |
| Population effect g_pop(t), LV/RV/adults (§4, §9) | not implemented: populations enter alike; one release keeps likely voters first | adjustments: lv_rv_handling |
| Sponsorship shift s_i·ψ (§4) | not implemented: sponsored polls are kept; the sponsor's lean is left to the pollster's house effect | adjustments: sponsor_handling |
| Overlapping tracker samples, ρ_overlap (§4.2.1) | canonical non-overlapping waves: of one firm's polls of the same question (candidates) with overlapping fieldwork, the newest is kept and earlier overlapping waves dropped; no overlap covariance is modelled. Exact duplicates are also removed at ingest | observations.nonoverlapping, ELAB_POLL_WAVES |
| Shared samples across races (§4.2.2) | not implemented | -- |
| Pollster families (§4.2.3) | not implemented in the fit; spelling variants of one firm are pooled | pollster_resolved (0057) |
| Undecided allocation (§9) | not implemented as a parameter: the raw margin is fitted with undecideds present, and the scale law carries it onto the all-vote target. Allocating undecideds in proportion to the two candidates before averaging is worse over 748 races (MAE +0.34, 90% CI +0.20 to +0.50) | adjustments: forecast_target |
| Turnout composition variance (§9) | not implemented as its own term; covered only inside the measured error terms | adjustments: turnout_models |
| Industry-wide mean bias b_cycle (§5) | not implemented: no mean-bias level is applied; the industry-wide error variance σ_nat is estimated from past cycles (§10, §11) | -- |
| Correlated error factors (§10) | national, Census region, office and race-idiosyncratic implemented; urbanicity, education, race/ethnicity, past politics, methodology mix and turnout environment not implemented | models/correlation/structure.py |
| Multivariate-t errors (§10) | implemented as a normalised shared-scale mixture; its ν is a moment estimate from the kurtosis of cycle means, clamped to [4, 20] | structure.estimate_structure, structure.draw_errors |
| Model uncertainty / stacking σ²_model (§14) | not implemented: var_model is a retired placeholder, null on new rows and absent from the live budgets |
simulate/variance.py |
The target. Every statewide product forecasts, and is scored on, the margin between the two contenders as shares of all votes cast (D% - R%, third parties and write-ins in the denominator) -- the way results are reported, and the scale the fundamentals prior, the scale law and the uncertainty terms are fitted on. The scored record is computed on this target (FM-197).
The House is a different model (house_ground_up): a gradient-boosted district lean, the
raters' average, a national term N3 and a national/state/district error model
(src/elab/models/district/model.py); none of the poll-level terms above apply to it. The lean
is the average of eight fits of the same learner that differ only in column order, which removes the
learner's tie-break between equally good splits from the answer (OrderAveragedGBM, FM-182). The
error model's national draw is normal; the House's state and district shocks are Student-t with
four degrees of freedom whose scale is 0.5 times the walk-forward sd
(experiments/house_spread_result.json: held-out district residuals have kurtosis near fourteen), not
normal. A t(4) has sd sqrt(4/2) = 1.414 times its scale, so the realised shock sd is 0.707 times
the walk-forward sd.
The House inputs are measured the same way in training and in prediction. The religion shares
(rel_*) come from the 2010 U.S. Religion Census for every cycle, 2012 to 2026, because the 2020
study counts some groups on another basis and a mixed panel would read the change of instrument as
a change in the electorate (elab.panel.religion.study_for; the statewide ground-up prior reads the
same rows). Past outside, party and PAC money counts only transactions dated before October of the cycle, in
filings the FEC had received by the twentieth of September (elab.panel.money). A filing whose image
number carries no date (the eleven-digit numbers used before mid-2015) is dated by the receipt date
the FEC records for its filing number, and one the FEC does not list is not counted.
Seats forecast from their own polls. Two House contests are not a Democrat against a Republican
and have polls of their own pair: AK-AL (an independent against a Republican, by ranked choice) and
CA-6 (a Democrat against a no-party candidate). Each is forecast from the two-way margin of its own
polls over a thirty-day window (elab.models.own_polls). The spread of these forecasts is the own-polls centre's measured past error, the walk-forward root mean square of its misses on every earlier polled
district-cycle (8.41 points for 2026, measured on 644 earlier district-cycles), drawn as a unit-variance t(4), the
House shock's shape; for ranked choice the ranking error below is added. On the 406 polled district-cycles of 2018-2024, each with its size measured on earlier cycles only,
the 90% and 80% ranges cover 0.901 and 0.768 (experiments/own_polls_shape_rebuilt_panel.md),
whereas the House model's full error, normal in shape, covered 0.983 at 90%. The shape was chosen on
the same rows, so this is a second look rather than a fresh test.
Ranked-choice final rounds. Alaska's Senate and governor races are published from their final round
(elab.models.ranked_choice); the Senate race's own two-way forecast is the reference the polling
error is read from. The centre is the scale law applied to the pollsters' average
final-round margin. The error has two parts. The polling component is a unit-variance t(5), the shape of every other
statewide race's election-day error, with its sd read from the two-way Senate forecast's 90% interval
under that shape. The ranking error is normal with sd 5.05 points, for ballots that exhaust or
transfer differently from what respondents said: the root mean square miss of predicting each of
Alaska's 2022-2024 final rounds from the round before it, with transfer rates from the other
contests. The two are convolved numerically
(RankedChoiceForecast.cdf).
The scandal rule. After a verified scandal about a candidate themselves (reported by two outlets doing their own
reporting; the events are listed by hand), a race's margin moves 3.0 points against that candidate from the event's
first date, with the spread unchanged (House: elab.models.district.scandal; Senate and governor:
scandal_shift in scripts/apply_statewide_prior.py). For a Senate or governor race the shift is made only while fewer than three polls of the forecast's
two contenders count: a poll counts when its fieldwork started after the event's first date, when the poll model
reads it, and when it is available at the forecast's own date. Each contender's event is counted on
its own (elab.models.shock.SCANDAL_POLLED_LIMIT). Polls taken after an event already carry most of its
effect. On the eve of past elections, a plain average of the post-event
polls missed the scandal-hit candidate by 0.5 points more than an ordinary race of the same year and margin.
The model's own number missed the scandal-hit candidate by 2.7 points
(experiments/scandal_investigation/), so a fixed
shift on top of three or more such polls would count the effect twice. The House rule has no
such limit.
Two conventions for a Student-t, one word each. The
statewide race model draws a unit-variance t (elab.simulate.forecast: the scale is
sd * sqrt((nu - 2) / nu)), so the statewide model's "sd" is the shock's sd. The House draws a
standard t times a scale (elab.models.district.model.simulate), so the House's state_sd and
idio_sd, as passed to the
simulation and as printed in ground_up_2026.json, are t scales; the shock's sd is the scale
times sqrt(df / (df - 2)). The multiplier k and the degrees of freedom are read together from the
same entry of house_spread_result.json (k_2026, shock_df), so a change of one changes the
other's meaning and is retested with it.
The nowcast (§11.0) is an estimate of today's two-candidate preference among the people polls sample -- likely voters, registered voters and adults alike, since population is not modelled -- not a forecast of an election held today with today's turnout.
1. Notation
| Symbol | Meaning |
|---|---|
r |
race (a contest in one geography for one office in one cycle) |
c |
candidate within a race; K_r candidates plus an "undecided/other" category |
t |
time, in days, with t = 0 at Election Day and t < 0 before it |
i |
an individual poll observation |
p(i) |
pollster of poll i; f(p) its pollster family |
n_i |
nominal sample size |
y_i |
reported vector of candidate shares (the fit reduces it to the two-candidate margin) |
θ_r(t) |
latent population support in race r at time t (the fit carries the scalar margin) |
D_X |
the information set knowable at calendar date X |
Pollster families f(p) appear in the specification only; the fit does not group firms into
families (§0).
Two quantities of interest, always distinct:
- Nowcast at date
X: the distribution of two-candidate support atXamong the people the polls sample, givenD_X; not the result of an election held onX(§11.0). - Forecast at date
X: the distribution of the result of the election held on Election Day, givenD_X.
Their poll-based variance budgets differ by one term, integrated future opinion drift (§11). The published election-day forecast also differs from the nowcast in its centre: for a race between a Democrat and a Republican it is blended with the ground-up estimate, and a listed scandal can shift it; neither touches the nowcast.
2. Scale
The published products are two-candidate margins: the Democratic share minus the Republican
share. The fitted statewide model works directly on that scalar (PollInput.margin), and the
House model works on the two-party share D/(D+R). The fit does not transform either onto a
simplex or a log-ratio scale.
The specification was written in additive log-ratio (ALR) space:
φ_rc(t) = log( θ_rc(t) / θ_r,ref(t) ), c = 1 … K_r - 1
with the leading major-party candidate as the reference, chosen from the fundamentals prior, and with the centred-log-ratio (CLR) parameterisation retained as the tested alternative. The motivation was that vote shares are compositional and that Gaussian noise on the raw simplex puts mass at the boundaries. That parameterisation is not implemented: it is a design alternative, not a description of the code. The fitted state variable is the scalar margin, and the scale law (§11) maps the poll margin onto the all-vote target.
Candidate alternative to test: a Dirichlet-multinomial observation model directly on the simplex. Selection would be empirical (ADR-0005), not aesthetic.
3. Latent opinion: the state process
For a race with four or more usable polls, the fitted statewide model is one scalar random walk
per race. Races with one to three polls use the precision blend in §7.2; a race with no polls is
not published as a race forecast. On the margin m_r(t) (the two-candidate margin in points),
m_r(t) = m_r(start) + Σ_{s = start+1}^{t} ω_s , ω_s ~ N(0, drift_sd²)
m_r(start)— race-level intercept. Its prior is the fundamentals prior (§7), and it is required: a fit without it is refused.drift_sd— the daily innovation scale, with aHalfNormal(0.30)prior in the fit. It is not estimated from prior cycles' polling trajectories; its posterior is updated by the current race's polls.- The innovations are independent across races. There is no national latent environment
η(t)and no race elasticityβ_rin the fit. The chamber simulation shares a national error draw across races (§10, §11), but that is a shock added to the stored race forecasts, not a latent trend estimated from polls.
3.1 Why a state-space model rather than a weighted average
A weighted average answers "what is the centre of recent polls". A state-space model answers "what is public opinion, given that polls are noisy, biased, and irregularly spaced". Within one race it borrows strength across time. It does not borrow across races in the fit: the cross-race borrowing is the shared error term in the chamber simulation (§10), not a latent national trend.
This still has to beat a recency-weighted average out of sample (§13). If it does not, the average ships.
4. Observation model for polls
For poll i fielded over [t_start, t_end] in race r, the fitted observation model is
ỹ_i ~ Student-t_ν ( μ_i , Σ_i )
μ_i = m_r(t̄_i) + h_{p(i)}
where
t̄_iis the field midpoint. A poll fielded over 10 days is not a measurement of its last day. The specification considered weighting the midpoint by field-period length; the fit uses the plain midpoint (§0).h_p— pollster house effect (§5, §6).
The specification also considered a polling-mode effect m_mode, a population effect
g_pop(t) for adults versus registered voters versus likely voters, and a sponsorship shift
s_i · ψ. None is in the fit: mode, population and sponsor are not read (§0).
4.1 Variance
Σ_i = DEFF · Σ_sampling(n_i) + τ²_{p(i)}
Σ_samplingis the sampling variance of the two-candidate margin implied byn_i. It is a floor, not the truth: reported margins of error systematically understate real error because weighting, clustering, and nonresponse inflate variance.DEFF— design effect. It is one scalar, measured walk-forward from the dispersion law and applied to every poll (0.757 for 2026; 1.6 where no earlier cycles are available). It is not estimated per mode or per pollster, and the fit does not assumeDEFF = 1.τ²_p— pollster excess variance beyond sampling (§6).- Student-t with estimated
νrather than Gaussian: individual polls produce outliers far more often than a normal allows. - What Σ_i is, exactly. The implementation passes
sqrt(DEFF · Σ_sampling + τ²)to the Student-t as its scale, not its standard deviation; a Student-t's sd is scale × √(ν/(ν−2)). Soτ(excess_sdin the code) is the excess scale, and the poll error's standard deviation is larger thansqrt(Σ_i)by that factor (5% at ν = 20, 29% at ν = 5). TreatingΣ_ias the variance instead scores slightly worse on the 2016-2024 record (log loss 0.1629, against 0.1626 for the parameterisation in use; experiments/student_t_scale_result.md), so the parameterisation stays as it is. No component outside the model is fitted around it: the uncertainty terms and the scale law are both fitted on the plain polling average. The fittedτabsorbs the difference, so the forecasts are calibrated as tested, butτis a scale and must never be read or reported as an sd.
4.2 Non-independence
Three mechanisms break the "one poll = one independent observation" assumption. The specification below handles each; the fitted model implements only the first, by keeping one firm's newest overlapping wave, with no overlap covariance, and the other two are not in the fit (§0):
- Tracking polls / rolling samples. Consecutive releases share respondents. Overlapping
releases from the same tracker are collapsed into non-overlapping windows, or given a
correlation term
ρ_overlapproportional to the fraction of shared field days. - Shared samples across questions. One survey reporting Senate and Governor toplines is one sample; its errors correlate across races. Modelled with a per-sample random effect shared by all races it reports.
- Pollster families. Firms sharing panels, vendors, or methodology (
f(p)) get a family-level house effect and family-level variance term, so releasing five polls under five brand names does not buy five independent observations.
5. Identification — the part most implementations get wrong
The decomposition μ_i = φ_r(t) + h_p is not identified without a constraint: adding a
constant to every h_p and subtracting it from every φ_r leaves the likelihood unchanged.
Polls alone can never tell us the absolute level of opinion. Only election results can. Therefore:
- Within a cycle, house effects are identified only relative to each other. We impose a
volume-weighted sum-to-zero constraint on
h_pwithin each race so the latent state is pinned to "the average pollster". - The industry-wide bias — the amount by which the average pollster misses the result — is
not applied as a level. No
b_cycleparameter exists in the fit, and neither the mean-bias correction nor the scale law's intercept is applied (§0). What is estimated is the variance of the industry-wide polling error,σ_nat, from realised results across past cycles (§10, §11). - Consequently, a model fitted on polls alone and reporting tight intervals is reporting the uncertainty of "where the polling average is", not "where the election lands". Conflating these is the single most common failure in public election models. The variance budget in §11 keeps them apart.
Number of effective observations of industry-wide error is small — roughly one per cycle,
on the order of ten to twenty-five for the modern polling era. σ_nat is therefore weakly
identified and carries a wide, honestly-reported posterior. This is a fundamental limit of the
problem, not of the implementation, and is recorded as failure mode FM-14.
6. Pollster quality, hierarchically
No subjective letter grades. For each pollster p, the fit is
house_raw_p ~ N(0, 1)
h_p = (house_raw_p - Σ_p w_p house_raw_p) · house_sd, w_p ∝ the firm's poll count
house_sd ~ HalfNormal(2.5)
log τ_p ~ N(μ_τ, σ_τ²)
μ_τ ~ N(log 2.0, 0.7)
σ_τ ~ HalfNormal(0.6)
The sum-to-zero constraint pins the latent level to the volume-weighted average pollster. The
house-effect and excess-variance priors are not grouped by polling mode or partisanship: there
is one house_sd, one μ_τ and one σ_τ across firms. A firm with three polls has a wide
posterior for both quantities; a firm with three hundred has a narrow one, because the
likelihood, not a group mean, does the shrinking. Pollster quality is itself a distribution,
and that distribution propagates into every forecast.
6.1 Accuracy vs. lean are different things
h_p (systematic lean) and τ_p (unpredictable noise) are separate parameters on purpose. A
pollster with a persistent +2.5 R lean and tiny τ_p is more useful than an unbiased pollster
with huge τ_p, because the lean is correctable and the noise is not. Any ranking the system
publishes must show both.
6.2 Open empirical questions, to be answered by backtest, not assumption
- Does
h_ppersist across cycles, and with what autocorrelation? Model ash_{p,cycle} = λ · h_{p,cycle-1} + e, estimateλ. Ifλ ≈ 0, historical house effects are useless for prediction and must not be applied. - Do pollsters improve or deteriorate? Test a trend term.
- Does a methodology change reset a pollster's history? Treat a documented methodology change
as a partial break: variance inflation on the carried-forward
h_p. - Survivorship: pollsters that quit after a bad cycle are missing from the evaluation set. Their absence biases estimated industry accuracy optimistically (FM-07).
None of these mechanisms is in the fit: house effects have one dispersion and excess variance one mean, with no cycle-to-cycle persistence term, trend or methodology-break term, and the evaluation set is not corrected for survivorship.
7. Fundamentals
A separate estimate gives the race intercept a prior before the polls arrive:
α_r ~ N( X_r' γ , σ_fund² )
In production the prior is the seat's previous margin, de-tided and re-levelled (§7.1); the candidate-feature relation below is the general form and, except where §7.1 says a feature is in production, is not. No feature outside §7.1 enters the published prior.
Candidate features (each a hypothesis, not an entitlement): partisan baseline of the geography (§8), incumbency and its interaction with tenure, open-seat status, previous margin, presidential approval, midterm penalty for the president's party, national economic indicators read from their first-published ALFRED vintages and state economic indicators read in their later-revised form (see below), candidate quality proxies (prior elected office), fundraising (§ build spec 50 — with reverse-causality controls), special-election swing (build spec 49), primary turnout differential, registration trends, and redistricting status.
Economic data. The national model's economic features are read from the ALFRED vintage in
force in mid-October of the cycle being scored (scripts/build_economic_vintages.py,
scripts/backtest_national.py --vintages, data/national/economic_vintages.json); before a
series' first vintage, its first available vintage is used, which for real disposable income
means the 2010 release. The state panel's economic features are read at their latest revision:
they do not carry a vintage_date filter, and src/elab/api/accuracy.py states that exception.
So the validation document's blanket statement that every economic read is filtered on
vintage_date <= :as_of is true of the national series and not of the state series.
Feature admission rule. A feature enters only if, in leave-one-cycle-out walk-forward evaluation, it improves out-of-sample log score by more than its standard error, across more than one held-out cycle, with a sign consistent with its hypothesised mechanism. Features that pass in-sample and fail out-of-sample are recorded as negative results in the experiment registry and not silently retried.
With ~35 Senate races per cycle and ~10 usable cycles, the fundamentals design matrix has on the order of 300–400 rows. This supports perhaps 5–10 real parameters, not 30. Regularised regression (horseshoe or hierarchical ridge) is mandatory; stepwise selection is forbidden.
7.1 What is actually fitted, and what was refused
In production: the seat's previous margin in the same seat, widened by measured volatility, with
its own cycle's national tide taken out and this cycle's estimated level put back. A 2026 Senate
prior drawn from 2020 otherwise carries 2020's national environment in full. For each office one
walk-forward regression of earlier cycles' mean margin on the realised national House margin (only
cycles before the target) is read twice: at the source cycle's realised House margin (the level
subtracted) and at the target cycle's N3 national estimate (the level added). The shift is the
slope times the difference of the two House margins, the tide alone; the source seat set's own
lean cancels (prior.detide_levels; experiments/detide_prior_preregistration.md, amendment 1).
Senate and governor only; the House keeps its previous margin. For 2026 it moves the Senate priors
toward the Democrats by 3.45 points on average and the governor priors by 6.07, before the polls
and the ground-up blend.
Tested on every election-eve backtest of 2016-2024 (256 races), the published forecast shows no measurable change in accuracy with the de-tiding, within the registered tolerances: log loss 0.1662 both, Brier 0.0518 with the de-tiding against
0.0517 without, 80%/90% coverage 0.840/0.945 against 0.836/0.945, mean absolute error 4.58 both
(experiments/detide_prior_result.md). The poll model alone is worse with the de-tiding (log loss
0.1781 against 0.1744); the blend with the ground-up prior absorbs that. On the 2026 forecasts the
de-tiding moves the published margins by 0.14 points on average over seventy-one races. The largest
change in a win probability is Oklahoma governor, where the Democratic chance is 17.0% with the
de-tiding and 15.8% without; no rating or chamber control changes. The de-tiding and the refitted uncertainty terms
(the latter described under the variance budget) form one model version, so a run that
switches the de-tiding off alone (ELAB_PRIOR_DETIDE=0) without the earlier terms files is
refused.
For a redistricted House seat, the district's own presidential margin under the new lines is used
in place of the old margin (BaselineRelation): mean absolute error 3.174 points using the
relation against 13.11 for the old-lines margin, on the live values in
experiments/baseline_relation.json (the file the parameters page reads).
Built, measured, and not in production: elab.models.fundamentals.lean fits, walk-forward,
lean_r = a + b1 * presidential_lean_r + b2 * previous_lean_r + b3 * held_r + b4 * incumbency_r
where a lean is a margin net of its own cycle's reference mean for that office, so the relation describes how a seat sits against the country and says nothing about where the country will sit -- that is the national term, supplied separately and carrying its own error. The recency half-life on training cycles is selected per target cycle by leave-one-cycle-out within the training cycles, because the presidential coefficient for the Senate rises from 0.29 on early cycles to 0.61 by 2024 as ticket splitting collapses, and a fixed half-life would be an assertion about how fast that moves.
At the prior level the fitted relation is better. Its mean absolute error is 10.58 points,
against 14.26 for the previous-margin prior, over 566 walk-forward seat-cycles; it is closer in
10 of 11 cycles, with standardised errors of sd 0.979 and coverage 0.571 / 0.813 / 0.943 against
a nominal 0.50 / 0.80 / 0.95 (experiments/lean_prior.md).
It is not in production because it did not clear the gate. On races with 1-3 polls its CRPS gain was
0.053 against a pre-registered threshold of 0.073 (c17_lean_prior, refused), because a poll
dominates the blend even after FM-104's correction. On races with no polls -- where the prior is
the whole forecast -- it won both folds it could reach by 1.66 and 3.88 points of CRPS, and the
pre-registration fixed three; 2004's drift law is unfittable and no unused cycle remains, so it was
recorded inconclusive (c18_lean_prior_unpolled). Under ADR-0016 there is no cycle left to register
this mechanism against, which makes "cannot be promoted on the evidence available" the answer rather
than a pending question. experiments/lean_prior.md and
experiments/preregistration_unpolled_prior.md carry both.
7.2 The prior for a race with one to three polls
A race below the fitting threshold is the prior and the polls combined by precision (c7_sparse_blend,
promoted). The polls' precision comes from the realised error of a sparse-race poll average,
pooled walk-forward from strictly earlier cycles, not from their sampling error. The row in force
on the parameters page is the 2024 cycle: realised 9.34 points against an assumed 4.33
(experiments/sparse_poll_variance.json), a ratio of 2.15. The pooled eleven-cycle measurement
is 8.92 against 4.04 (experiments/sparse_blend_poll_weight.md). On either, weighting by sampling
variance gave these polls about five times the weight the evidence supports and quoted an
interval about half as wide as the evidence supports. A cycle with nothing behind the measurement falls back to sampling error rather than
borrowing a number from its own future, and SparseEstimate.basis records which was used
(FM-104, experiments/sparse_blend_poll_weight.md). The national and race-specific parts of a sparse-race poll average's realised error are taken out before the blend (ELAB_SPARSE_SHARED_ONCE), because the simulation
adds both afterwards.
Two open questions about races with one to three usable polls are stated rather than hidden. First, such a race is
combined with the seat's prior here and then, like every race, with the ground-up prior
(scripts/apply_statewide_prior.py), so it takes a prior twice (FM-253). Counting it once
(ELAB_SPARSE_PRIOR_ONCE) improves the 2016-2024 record's log loss very slightly (0.16415 to
0.16400) but narrows the sparse races' 90% ranges, which already cover too few results (0.843 to
0.784), so it is held, to be made together with a correction of those races' range width. Second, the prior's
error and the polls' error in these races overlap: their correlation on the thirty-three sparse
races of 2020-2024 is 0.41 (95% interval 0.23 to 0.64), which the independent-errors formula
ignores. The exact posterior with that overlap and its own measured residual is the arm that experiments/sparse_error_accounting_preregistration.md scores against the published blend on
the 2026 results.
7.3 Blending fundamentals with polls
Not an ad-hoc weighted average. The fundamentals distribution is the prior on α_r; polls
update it through the likelihood. The effective weight on fundamentals therefore declines
automatically as polls accumulate — no hand-tuned decay schedule from fundamentals to polls is
needed, and none will be introduced.
8. Geographic baselines
A "partisan baseline" for a state or district must be constructed from results that are comparable across boundary changes.
- Districts are identified by
(cycle, state, district_number)plus a boundary hash. A district whose lines changed is a different unit even at the same number. - Baselines for redrawn districts are rebuilt from precinct-level results disaggregated onto new boundaries where precinct data exist; where they do not, the baseline carries an explicit larger uncertainty rather than a silent guess.
- Uncontested races produce missing, not zero, two-party margins; they are imputed with uncertainty from neighbouring-office results, and flagged.
- The production prior is not a weighted blend of presidential and down-ballot results. It is
the seat's previous margin, de-tided to the cycle's national level (§7.1); a redrawn district
takes its presidential margin inside the new lines instead (
BaselineRelation). A weighted blend with an estimated realignment drift is a specification alternative, not in production.
9. Likely voters, registered voters, undecideds, turnout
g_pop(t): the specification estimates an LV-versus-RV effect varying by cycle, office and calendar time. The fitted model does not: population is not read, and likely-voter, registered-voter and adult samples enter alike (§0). The folk belief of a fixed "LV premium" of a couple of points for one party is a hypothesis, not a constant.- Undecideds. The specification makes undecideds a modelled category, with their Election Day allocation a parameter informed by historical late-break behaviour. The fitted model has no such parameter: poll margins are read with undecideds left out, and the scale law carries them onto the result's scale (§0).
- Turnout composition error. The specification gives it a distinct variance component, because pollsters' likely-voter screens are themselves models that can fail in a correlated way across the industry. The fitted model has no separate term for it: it is covered inside the measured error terms (§0).
10. Correlated error — the term that dominates
Election outcomes are not independent Bernoulli draws. When polling misses, it misses many
places at once, in the same direction. The implemented joint error vector e over races at
Election Day has four terms:
e = σ_nat · 1 · z_0 + σ_region · L_region z_region + σ_office · L_office z_office
+ diag(σ_idio) z_idio
z_* ~ multivariate Student-t_ν (a normalised shared-scale mixture, not Gaussian)
The national term is one draw per simulated election, identical for every race. The region and
office terms are indicator loadings: a race loads on its census region and on its office. The
race-idiosyncratic term is the residual. σ_nat, σ_region, σ_office and σ_idio are
variance components estimated from the realised joint distribution of polling errors in
cycles strictly before the forecast cycle (structure.estimate_structure); they are not
assumed. The following factors were specified and are not implemented: urbanicity,
education, race/ethnicity, past politics, methodology mix, and turnout environment.
| Factor | Status |
|---|---|
| National | implemented: one draw per simulated cycle |
| Region | implemented: census-region indicator |
| Office type | implemented: office indicator |
| Urbanicity, education, race/ethnicity, past politics, methodology mix, turnout environment | not implemented |
Heavy tails are mandatory. In the chamber structure, ν is a moment estimate from the
kurtosis of cycle means, clamped to [4, 20] (structure.estimate_structure); it is not a sampled
parameter with a prior. A race page's own error is a unit-variance t with five degrees of
freedom, and the House's state and district shocks are t with four; those degrees of freedom are
constants, not fitted. A Gaussian tail assumption would have made the 2016 and 2020 polling
misses near-impossible events; history says they are not.
10.1 Sanity constraint
The resulting Σ must reproduce, in posterior predictive checks, the observed cross-race error
correlation in held-out cycles. If simulated cycles show less correlated error than history
does, the model is overconfident about chamber control even when individual races look fine.
The check is run by scripts/chamber_validation.py against the cycles 2014-2024 (FM-11); it is
separate from the per-fit convergence gate in §15.3.
11. The variance budget: nowcast vs forecast
11.0 What the nowcast actually estimates
"If voting occurred today" is not a well-defined quantity unless the electorate is specified. Two different questions hide inside the phrase:
| Estimand | Meaning |
|---|---|
| (a) | The election held today, voted by whoever would actually turn out today — a low-turnout, high-engagement electorate |
| (b) | The election held today, voted by the electorate we project for Election Day |
They differ by several points in a midterm. This system estimates neither exactly. It has no turnout model: likely-voter, registered-voter and adult samples enter alike (§0), so the nowcast is today's two-candidate preference among the people the polls sample. Where a pollster reports a likely-voter result it is used, and likely-voter screens are pollsters' attempts to model the November electorate, so the nowcast leans toward (b) to the extent the polls do. The methodology page describes it the same way.
Formally: the nowcast marginalises over measurement uncertainty about the polls and over the two realised polling-error terms, but conditions on latent opinion as of today, with no future drift. It carries no composition term, because the fit has no turnout model.
For race r at forecast date X, on the latent margin scale:
NOWCAST:
Var_now = Var[m_r(X) | D_X] # state uncertainty: polls, house effects, sparsity
+ σ²_nat # industry-wide error shared across races (§10)
+ σ²_race # race-specific residual polling error (§5, §10)
FORECAST:
Var_fc = Var_now + Var_drift(h) # integrated future drift to Election Day
Both halves of the variance-components model. The error model of section ten decomposes the
error of a polling average into mu_c ~ N(0, σ²_nat), one draw per cycle shared by every race in
it, and eps_{r,c} ~ N(0, σ²_race), the race-specific residual. For 2026, fitted on 1,226 races
over twenty-five cycles, σ_nat is 2.63 points and σ_race is 4.93 — the residual is the larger term. The
race-level budget carries both, as the chamber simulator does.
The residual is one fitted term, with no separate likely-voter, undecided or specification
placeholders beside it, because a residual already contains the error those would stand for;
carrying both would over-widen the range. Walk-forward, the budget with σ_race gives realised
coverage on 247 scored races of 0.474 / 0.777 / 0.895 against nominal 0.50 / 0.80 / 0.90, with
nothing tuned (FM-79). The budget contains no unfitted term, so the publishing path does not use
allow_guesses, and the simulator's own refusal of unfitted terms (elab.simulate.forecast)
guards the published forecast.
An independent check on an earlier thirteen-cycle fit, where σ_race was fitted at 5.03, measured an
rms of 5.42 for realised error with each cycle's mean removed, which is what eps is; the check
supports the term at that value, not directly the 4.93 of the twenty-five-cycle fit.
The drift term is estimated empirically: for each historical cycle, measure the realised
change in the latent state (or, failing that, in a consistently-computed polling average)
between horizon h and Election Day, pooled across races, and fit Var_drift(h). A pure
random walk implies Var ∝ h; history may show mean reversion (sublinear) or campaign-event
clustering (superlinear, with conventions and debates). We fit it; we do not assume ∝ h.
The files in force. σ_nat, σ_race and the drift law come from
experiments/uncertainty_terms_pre<T>.json, fitted by scripts/estimate_uncertainty_terms.py --before T on cycles before T only. The fit scores each race's final polling average against the
first round, the November vote the polls were taken for, rather than a runoff or a ranked-choice
count's final round. The fit reads live
result rows only and matches every party spelling (FM-232). For 2026, from 1,226 races over
twenty-five cycles: σ_nat 2.63, σ_race 4.93, drift a = 1.650 and b = 0.669.
Tested on every election-eve backtest of 2016-2024 against the terms fitted with the earlier result selection, the published forecast's log loss is 0.1641 against 0.1662, Brier 0.0516 against 0.0517, mean absolute error 4.57 against 4.58, and 80%/90% coverage 0.824/0.938 against 0.836/0.945 (experiments/uncertainty_terms_refit_result.md). That record covers election eve only, where the drift term is near zero, so the drift law is not tested at the horizons weeks out where it acts. The files for 2004-2014, which serve only the 2006-2014 backtests, use the earlier result selection.
Invariants enforced by test:
Var_fc ≥ Var_nowfor allh > 0.Var_fc → Var_nowash → 0.Var_fcis monotone non-decreasing inhunless a fitted mean-reversion term is justified by held-out evidence and documented.
This is the structural reason a race can be "D+6 today with an 88% chance if the election were today" and simultaneously "68% to win in November". Both numbers are always published.
11.1 The chamber products need their own drift, and the House's is national
The budget above is a race's. A chamber forecast built from priors rather than from race-level polling has a different structure, and needs a drift term of its own.
District polls (House). Where a district has polls, each pollster's latest poll, its sponsor's
measured bias removed, is averaged and combined with the district prior by inverse variance, the
polls weighted by s_o2 / n + s_c2 (elab.models.district.polls.shift_v2). This is an empirically
tested weighting with overlapping components, not an exact independent-plus-shared decomposition:
s_o2 is pooled over cycles and already contains the shared cycle error that s_c2 adds. The
clean split into independent and shared parts scores no better, because the shared component is
small: experiments/house_poll_variance_result.md.
Each district poll is also discounted by its age on a fixed schedule, not a fitted one
(elab.models.district.polls.age_factor in shift_v2, for every district and the seats forecast
from their own polls; adjustment house_poll_age_decay): full weight within sixty days of the
election, falling steadily to half at 210 days and a quarter beyond. On the 2016-2024 walk-forward
record the discount does no harm (squared-error difference -0.58, 95% interval -2.70 to +1.10;
better at 120, ninety and sixty days out, and neutral in the final month).
The House is 435 districts of which a handful are ever polled, so its forecast is each district's
prior shifted by the change in the national environment and spread by that district's
volatility net of the nation. The national shift is measured from current generic-ballot polls.
Neither the district spread, an election-to-election quantity, nor σ_nat, the cycle-level
polling error, allows for the movement of that environment between the forecast date and Election
Day. Without a term for that movement, a forecast made forty-one days out would carry the same national uncertainty as
one made on the morning of the election.
elab.models.fundamentals.environment_drift fits Var_env(h) = a·h^b from non-overlapping
increments of the generic-ballot path, four cycles of it, and the forecast adds Var_env(h) to
σ²_nat in quadrature. Three properties of the estimate are worth stating because they are easy to
get wrong:
-
The race-level drift law is the wrong instrument here. It is fitted on single races and therefore contains idiosyncratic movement, which is exactly what averages out of a national number. Applying it would overstate the term.
-
Measurement noise is subtracted. Each end of an increment is an estimate of the environment, so an observed difference is movement plus twice that estimate's sampling error. At short horizons the noise is the larger part -- 31% of the raw variance overall. It is also why every horizon exceeds the estimation window: overlapping windows share most of their polls, so their errors cancel rather than add, and the independent-sample correction then subtracts a variance that is not there.
Which effective sample size is a real choice, because the estimate has two (§11.1.1). The correction uses
sampling_n, the independence-assuming one. The firm-clustered figure is right for the level and wrong for a difference, and the data settles it rather than the argument: increments whose ends draw 97% of their weight from the same firms have a raw variance of 0.70 pts² at 21 days against a clustered noise estimate of 2.06 pts². A component three times the total it belongs to is refuted. House effects and panel persistence appear at both ends of such a difference and cancel. -
The fits are mean-reverting and the exponent is constrained. On one cycle's path the unconstrained fit gave
b = -0.78, which says a forecast three months out is more certain than one a month out.bis clamped into[0, 1.5]and the law records that it was clamped, so a clamped fit cannot be read as a measured one.
And a third national term: how well today is measured. Var_env(h) is the movement still to
come, and says nothing about the precision of the starting point. The estimate's own error is added
in quadrature, so a window of nine polls from five firms counts as less precise than a window of 165
polls from forty-three firms. This error term is the max of two figures, neither of which bounds the other (FM-74, FM-75, FM-76):
- the measured dispersion law (§11.1.2).
Var(poll) = floor + slope/nwith a house-effect dispersion across firms, fitted on same-week cross-firm pairs and walk-forward per cycle. The weighted mean's variance follows, and it has two parts: a per-poll part, and a house-mix part that depends on how concentrated the firm weights are because a firm's house effect is one number however many polls carry it. - the firm-level jackknife: delete one firm, re-estimate, take the cluster-robust spread. This is what this window's firms disagree about rather than what the corpus average says, and it is withheld below four firms rather than reported from two deviations.
1e4/independent_n is a printed diagnostic only. It is
blind to both dominant components: it said 0.63 points for the 2020 window where the dispersion law
says 1.84, and 0.63 for 2022 where the jackknife says 2.36, because Morning Consult held 54% of that
window's weight at D+3.19 while one 108,206-respondent SurveyMonkey panel held 20% at D−2.00. Each
route wins somewhere — the law in 2020, 2024 and the live 2026 window, the jackknife in 2022 — which is
why the term takes the larger rather than choosing between them.
The term is what makes an early, thinly polled or concentrated environment print a wider chamber interval than a late, broad one.
It double-counts whatever share of σ_nat is the generic ballot's own sampling noise. That share is
bounded by the same measurement: the cycles σ_nat is estimated from carried 0.6–0.9 points of it,
which is 0.4 of about 10 points², a 2% overstatement of the national sd in a well-polled window and
inside the resolution of an 11-cycle estimate. The direction is honest widening, and the alternative
in a thin window is claiming a precision the polls do not have.
11.1.1 Two effective sample sizes for the environment, always reported together
The national environment is a recency- and precision-weighted mean of generic-ballot polls, and how much information it rests on has two answers that differ by more than an order of magnitude:
sampling_n-- Kish's(sum w)² / sum(w²/nᵢ), the size that reproduces the weighted mean's sampling variance if the polls are independent samples. With recency weights this is larger thansum(rᵢ·nᵢ), because down-weighting a three-week-old poll gives up freshness, not respondents.independent_n-- each firm treated as one cluster whose precision is capped at its largest single wave. 59% of this corpus carries 538'strackingflag and 72% of the 2020 window's weight is two firms re-interviewing the same panels weekly, so respondents are being counted repeatedly.
In the 2020 window these are 695,887 and 26,032: a sampling sd for the margin of 0.12 points against
0.62. describe() prints both and sampling_sd quotes the clustered one. Neither is available
alone, because the gap between them is the substance. estimate() needs series_window( labelled=True) to compute the second at all, and returns None for it rather than a guess when the
pollster is not supplied.
The estimate is insensitive to this — one vote per firm instead of one vote per poll changes the
House seat error not at all (experiments/environment_weighting.md) — so the choice matters for the
advertised precision, not the estimate.
11.1.2 What a poll's precision is actually made of
Both sample sizes above assume a poll's variance is proportional to 1/n. Measured on 13,996 pairs of
generic-ballot polls fielded within three days of each other by different firms, with each firm's
house effect subtracted:
Var(A − B) = 2.37 + 7,571 · (1/n_A + 1/n_B)
These are the values in force (experiments/poll_dispersion_pre2026.json, which the parameters
page reads). Pairs rather than deviations from a consensus, because a consensus carries an error
of its own that is common to every poll at that date and would land in the intercept whether or
not the polls were clean.
- Sampling behaves about as stated. Slope 7,571 against the 10,000 a simple random sample of a two-party margin implies: a design effect of 0.757.
- Each poll carries 2.37 pts² = 1.54 points that no sample size reduces, and it is not opinion moving inside the pairing window — the earlier split by pairing gap gave 1.35 points at the same-day gap — nor a shared weekly fashion, since the within-date variance across 107 multi-poll dates is 5.99 pts².
- House effects are dispersed by 2.58 points across firms and do not average away within a firm.
The house-mix term dominates the environment's variance in every window measured. The implication for
the weights — that a 108,206-respondent panel is worth 4.1 times a 1,000-person poll rather than 108
times — is applied in the environment's weights (c11_inverse_variance_weights). The weighting's evidence is a firm-clustered 95% interval for the paired improvement in squared
error, [+0.063, +0.329] pts², measured on 3,765 held-out polls over 127 firms; the interval excludes zero, as the interval
measured at registration, [+0.059, +0.304], did. The metric is
leave-one-poll-out against other firms' polls in a twenty-one-day window, so it reads no election outcome
and uses no holdout under ADR-0010. experiments/preregistration_inverse_variance.md has the
decision. On a 2026 snapshot the weighting moves the expected Democratic House seats by about a
third of a seat, inside the registration's predicted range of environment movement, +0.02 to
+0.31 points; accuracy on elections is not the argument for the weighting.
11.1.3 Pollster house effects removed before averaging
Each firm's lean is measured against the consensus of the other firms (leave that firm out), shrunk
by k/(k+4) for a firm with k polls, centred so the firms average zero with equal weight per firm, and
subtracted from its polls before the national average is taken (elab.models.fundamentals.environment.house_effects,
read by forecast_house.py and build_national_inputs.py; adjustment generic_ballot_house_effects).
In a walk-forward test on 2004-2024 against the plain average, the correction has mean squared
error 7.84 against 8.82 and is better in eight of eleven cycles at thirty-one days and one day
before the election, where the registered rule requires at least seven; the bootstrap difference is
-0.98 (95% -1.78 to -0.44; experiments/generic_house_effects_result.json).
In the same test a variant anchored to media and academic firms has an even lower mean squared
error (6.07); the variant is registered for prospective scoring on
2026 only and does not enter
the forecast.
11.2 What is shared across races, and where it comes from
For a chamber, the question "which part of a race's uncertainty moves with every other race"
decides the answer, and the budget above already names it: σ²_syst is industry-wide polling
error, one number per cycle, identical across races by construction. Everything else in a
race's budget is that race's own.
The chamber simulation draws σ_syst once per simulated election from the stored budget of
the races it is combining, and draws every other term per race. It does not draw the
election-to-election national swing on top of races whose margins have already been measured
from polls: that counts the swing twice and makes the chamber interval far too wide (FM-66).
Unpolled races take the prior shifted by the swing the polled races imply, plus the
idiosyncratic part of a race's swing and the standard error of that swing estimate.
scripts/forecast_ec_from_stored.py applies the same decomposition to stored presidential
forecasts. The site publishes no 2026 presidential forecast and no electoral-college or
national-popular-vote product, so those are not aggregates of this model.
12. Simulation
- Draw
Ssimulation draws from the posterior. A race page usesS = 50,000(elab.simulate.forecast). The count is fixed; the Monte Carlo standard error on the win probability is reported beside the number (FM-36), rather thanSbeing chosen from an MCSE target. - For each draw, draw the correlated error vector
e(§10) and, for the election-day product, the empirical drift. - Apply the fitted scale law to the simulated margin and determine per-race winners.
- Aggregate: House seats, Senate seats (including holdover seats, independents' caucusing, and the Vice President's tie-break where a chamber simulation applies it), and governorships. No presidential contest is aggregated.
- Persist the full joint draw matrix, not just summaries — chamber-control probability is a functional of the joint distribution and cannot be recovered from marginals.
The two race products are labelled separately: the nowcast and the election-day forecast
(§11.0). The chamber products aggregate the election-day forecast. In the House, the Democratic
majority chance, the Republican majority chance and the no-majority outcome are counted from
the same simulated elections and shown as three separate figures; the no-majority share is never
added to a party's chance (elab.api.dashboard, elab.api.home).
Convergence is checked by Monte Carlo standard error on the reported quantities, and the simulation seed is recorded (N-01).
13. Baselines the model must beat
Every sophisticated component is evaluated against, at minimum:
| Baseline | Description | Status |
|---|---|---|
| B0 | Previous election result in the same geography | not implemented: the function raises rather than returning a number |
| B1 | Geography partisan baseline (§8), uniform national swing applied | not implemented: the function raises |
| B2 | Unweighted mean of last 21 days of polls | implemented (b2_unweighted_recent) |
| B3 | Exponentially recency-weighted poll average | implemented with a fixed 14-day half-life; the decay is not fitted by grid search |
| B4 | B3 + sample-size weighting | implemented as recency × capped √n (b4_recency_and_size); it has no house-effect correction |
| B5 | Fundamentals-only (§7) | not implemented as a published product; the prior alone is a chamber input |
| B6 | Simple 50/50 blend of B4 and B5 | not implemented |
Plus an uncertainty baseline: B2, B3 or B4 with an empirical error distribution taken from the historical distribution of (poll average − result). The published comparison uses B4 with that widened error. This is a surprisingly strong probabilistic baseline and beating it is the real test.
If the hierarchical model cannot beat B4 with that error out of sample on log score and calibration, the simple model ships and the complex one goes back to the lab. This is a rule, not a sentiment.
14. Model uncertainty (specification, not implemented)
Parameter uncertainty is not the same as being right about the model. The fitted variance
budgets carry no between-specification term: var_model is a retired placeholder, null on new
rows and absent from the live budgets (simulate/variance.py). The specification would carry
σ²_model by:
- Fitting a small set of structurally distinct specifications (different decay families, different correlation factor sets, different observation distributions).
- Evaluating each out of sample.
- Combining by stacking weights fitted on held-out log score — not by equal weights and not by in-sample marginal likelihood (which rewards overfitting under misspecification).
- Reporting the between-specification spread as an explicit variance component.
Stacking would itself need validation: an ensemble is a hypothesis too (build spec 27).
15. Estimation and diagnostics
15.1 Engine
The fit uses PyMC's NUTS, pm.sample(..., nuts_sampler="pymc")
(src/elab/models/latent/single_race.py). scripts/benchmark_samplers.py compares that sampler
with nutpie on a synthetic data-generating process by ESS per wall-clock second, divergence
behaviour and memory footprint. There is no CmdStan path in the code.
15.2 Priors
Every prior in the statewide fit, with its justification:
| Parameter | Prior | Justification |
|---|---|---|
start_margin |
Normal(fundamentals prior mean, sd) | the race prior from §7; required, with no default |
drift_sd |
HalfNormal(0.30) | daily random-walk innovation scale, in points per day |
house_sd |
HalfNormal(2.5) | dispersion of pollster house effects across firms, in points |
mu_tau |
Normal(log 2.0, 0.7) | geometric mean of the excess scale across firms, about 2 points |
sigma_tau |
HalfNormal(0.6) | dispersion of the log excess scale across firms |
nu |
Gamma(2.0, 0.1) | Student-t tail weight for a poll's error; support (0, ∞), mean 20 |
innovations, house_raw, tau_raw |
standard normal | non-centred parameterisations |
The specification also calls for a prior predictive check: simulate from the prior alone and confirm the implied elections are politically plausible (no 97%-of-vote Senate winners, no 40-point national swings in a week). That check is not implemented in the fit path: no code simulates from the prior alone.
15.3 Diagnostic gates (a fit that fails these is not used)
The gate covers every sampled parameter except the daily innovation and latent nuisance parameters, plus a separate gate on the one forecast-date latent margin. The nuisance parameters are hundreds of daily innovations, many on days with no poll; taking their maximum R-hat compares hundreds of values against one threshold. A systematic failure is caught by the fraction-of-parameters rule below, and the margin the forecast actually reads has its own gate. The diagnostics record which parameter is worst, so a failure names its cause.
-
R̂ < 1.01on every gated parameter -
Bulk ESS > 400 per gated parameter, and bulk and tail ESS > 400 on the forecast-date margin
-
Zero post-warmup divergences (or a documented, investigated exception)
-
No more than 5% of all sampled parameters over the R-hat threshold (a multiple-comparison guard)
Divergences are treated as a modelling defect, not a sampler setting: raising
target_acceptquiets the symptom while leaving the variance components wrong, and those are precisely the parameters this model exists to estimate (FM-29). The hierarchical non-centred parameterisation in §6,log τ_p ~ N(μ_τ, σ_τ²), is what avoids the funnel an independent scale per pollster produces. -
E-BFMI: not checked by
passes_gates -
Posterior predictive checks on held-out polls: not part of
passes_gates; the cross-race check is run separately (scripts/chamber_validation.py, §10.1) -
Synthetic-data parameter-recovery tests:
tests/synthetic/, a development check, not a live-fit gate
16. Known weaknesses, stated up front
- Few effective observations of systematic polling error (§5). Roughly ten to twenty-five
cycles.
σ_natis weakly identified and any claim to know it precisely is false. - Non-stationarity. Polling methodology changed radically between 2000 and 2026 (landline → online panel). Historical error estimates may not transfer. A concept-drift weighting is specified (build spec 82) but is not implemented; it would be a partial mitigation, not a solution.
- House data sparsity. Most of the 435 districts are never polled. Their forecasts are fundamentals-plus-national-swing, and their uncertainty must reflect that. The chamber distribution is therefore driven largely by the correlated error structure, which is the least well-identified part of the model.
- Candidate-level idiosyncrasy (gaffes, health events, unlisted scandals) has no term of its own. Such events are carried by three separate parts of the error
budget: the race-specific error,
σ_race; the drift still to come; and, once an event has been polled, the polls themselves. The one exception is an event listed under the scandal rule: the event moves the race's margin by a fixed 3.0 points against the scandal-hit candidate, with the spread unchanged, on the conditions given in section zero. That size comes from seventeen statewide and twelve House cases. The statewide measurement was set up to report the result rather than as a test to pass. The scandal shift's gain rests largely on one race (Missouri's 2012 Senate race), and the interval for the shift's change in CRPS (lower is better), -0.71 to +0.38, includes zero, whereas the House test's interval, -1.31 to -0.18, excludes zero. - Selection in the poll set. Polls are released, not sampled. Suppressed unfavourable internal polls bias the observed set in ways no weighting fixes (FM-08).
- Backtest reuse. Every pass over the same 10 cycles burns holdout. The holdout ledger (BACKTESTING.md §6) tracks the burn rate; when it is exhausted, honest evaluation requires waiting for a real election.