Restricting the Spec 3c clean comparator to media-only polls (dropping pollster-self) gives β = +8.15 pp (p = 0.036, n = 165 rows / 28 race-week cells) — if anything larger than the media-or-pollster-self spec. Pollster-self inclusion is not pulling the headline up.
Question
AN-002's Spec 3c uses a "clean comparator" pool of media + pollster-self
polls. Pollster-self polls are polls a pollster fields without a paying
sponsor — typically for stable-of-clients reputational reasons. The natural
worry is whether including pollster-self polls is doing load-bearing
work for the headline: a pollster fielding for itself might still have
indirect motivations that correlate with sponsor incentives. GPT-5-pro's
pre-submission review (2026-06-14) flagged this and asked for a
side-by-side spec restricting the comparator to media-only.
Design
Same as AN-002 Spec 3c, but the comparator pool is restricted to polls
whose sponsor_types == "media" (excluding pollster-self). After the
within-(race × week) cell filter, the sample drops from 60 cells (288
rows in the standard Spec 3c) to 28 cells (165 rows).
| Property | Spec 3c (AN-002) | Spec 3c media-only (this AN) |
|---|---|---|
| Comparator pool | media + pollster-self | media only |
| Cells (race × week) | 60 | 28 |
| Rows | 288 | 165 |
Results
| Spec | β (sponsored_by) | SE | p |
|---|---|---|---|
| Spec 3c (AN-002, media + pollster-self) | +8.05 | 2.60 | 0.003 |
| Spec 3c media-only (this AN) | +8.15 | 3.74 | 0.036 |
The media-only β is essentially unchanged from the standard 3c. The cell loss (60 → 28) widens the SE but the coefficient remains significant at the 5% level. If anything, dropping pollster-self pushes β up by 0.1 pp, so the standard 3c isn't averaging in a zero-effect pollster-self group.
Interpretation
The Spec 3c headline is not driven by including pollster-self in the
comparator pool. The conservative spec (media-only) gives the same
qualitative conclusion at less precision but a slightly higher point
estimate. Justifies the standard 3c choice of media + pollster-self
on power grounds without inflation risk.
Follow-ups
- The "within-firm" reputation reading (§sec:within-firm) is independent of this comparator-pool choice — pollster-self polls show up as the firm fielding for itself rather than for a candidate.
- If a referee insists on media-only as the primary spec, the cell count is the binding constraint, and the 2022 cycle extension (queued in todo.md) would help.