id: an-085 hypothesis: shell-contratante headline: User's intuition confirmed — the "media" bucket is heterogeneous. Within media-sponsored polls, trusted firms (DATAFOLHA, QUAEST, PARANÁ PESQUISAS, REAL TIME MÍDIA, VERITA — n=278) understate the margin by 3.16 pp LESS than other firms in the same race (p=0.002, race FE); major-media sponsors (GLOBO, FOLHA, ESTADÃO, large regional outlets — n=304) have mean |error| 1.0–1.4 pp LOWER than blog/small-digital sponsors. Trusted-firm self-contracted polls are the MOST accurate slice of the entire dataset (mean |error| 5.2 pp vs universe 8.1 pp). Universe-wide spec with both trusted-firm and bucket dummies: is_trusted_firm β = −1.15 on |error|, −2.02 on margin, +0.10 on calls-winner (all p < 0.002, race FE). The other_firm margin-error effect SURVIVES after controlling for firm tier (β=−1.95, p=0.002). type: causal question: "Is the media bucket itself heterogeneous? Are some media-sponsored polls shells in media clothing, and how much accuracy difference do trusted sources (Datafolha-tier) actually deliver?" tags: ["hyp:shell-contratante", "trusted-source", "media-heterogeneity", "all-brazil"] status: interpreted status_date: 2026-06-17 confidence: green created: 2026-06-17 script: source/analysis/an-085-trusted-source-decomposition.py target: build/table/an-085-trusted-source-decomposition.csv design: sample: matched-share-1.0 cand-poll → poll-level (7,032 polls in 4 buckets, 558 trusted-firm) specification: | Four tests: (A) Within media-sponsored polls: is_trusted_firm vs other (B) Within media-sponsored polls: is_major_sponsor (regex on top-recognized national / large-regional outlets) vs small (C) Universe spec with both is_trusted_firm + 3 bucket dummies (D) Within trusted-firm polls only: bucket dummies vs media reference Outcomes: mean_abs_error, margin_error, poll_calls_winner_first. Specs: S0 raw means, S1 race FE (muni). Cluster SE at race. comparator: media-sponsored polls (overall reference)
AN-085: Trusted sources decompose the media bucket
User question (2026-06-17): "Could we check some very trusted sources like Datafolha and compare accuracy with other sources. Maybe media sponsored are also shells in some cases?"
Answer: yes — the media bucket is heterogeneous in exactly the way the user suspected. Trusted firms and major-media sponsors account for the cleanest slice of the data; the rest of "media" behaves much closer to the residual buckets.
Cell counts and raw means
| Cell | n | mean |err| | margin_error | calls_winner_first |
|---|---|---|---|---|
| media — other firm | 2,670 | 8.28 | 26.5 | 73.0 % |
| media — trusted firm | 278 | 6.87 | 24.4 | 74.8 % |
| pollster_self — other firm | 1,296 | 8.79 | 26.3 | 69.1 % |
| pollster_self — trusted firm | 168 | 5.21 | 23.8 | 83.9 % |
| other_firm — other firm | 1,483 | 8.13 | 23.5 | 69.0 % |
| other_firm — trusted firm | 70 | 9.20 | 18.6 | 75.7 % |
| candidate — other firm | 984 | 9.35 | 26.7 | 72.2 % |
| candidate — trusted firm | 42 | 6.23 | 18.2 | 81.0 % |
Trusted firms = {DATAFOLHA, QUAEST, PARANÁ PESQUISAS, REAL TIME MÍDIA, VERITA}, n_polls = 558 across all sponsor classes (8 % of the four-bucket universe).
Inside the media bucket: 304 polls have a major-media sponsor
(GLOBO, FOLHA, ESTADÃO, large regional outlets matching the
regex in classify_media_sponsor); 2,644 have small/blog
sponsors ("BLOG DO ALBERES XAVIER", "3 PODERES MIDIA",
"BG MIDIAS", etc.).
Within-media tests
(A) Trusted firm vs other firm — race FE
| Outcome | β (is_trusted_firm) | p | n |
|---|---|---|---|
| mean_abs_error | −0.27 | 0.58 | 2,220 |
| margin_error | −3.16 | 0.002 | 2,124 |
| poll_calls_winner_first | +0.004 | 0.92 | 2,220 |
Trusted-firm media polls cut the margin (winner − runner-up) error by 3.16 pp within the same race compared to non-trusted media polls.
(B) Major-media vs small-media sponsor — race FE
| Outcome | β (is_major_sponsor) | p | n |
|---|---|---|---|
| mean_abs_error | −1.01 | 0.051 | 2,220 |
| margin_error | +0.70 | 0.54 | 2,124 |
| poll_calls_winner_first | +0.068 | 0.15 | 2,220 |
Major-media-sponsored polls have 1.0 pp lower mean |error| than small-media-sponsored polls within the same race (borderline significant). The two cuts (firm and sponsor) tap different aspects: trusted firms reduce the margin error; major-media sponsors reduce the mean error. Both signals are in the direction "trusted sources are more accurate."
Universe-wide spec (C) — firm tier + bucket dummies
Race FE; reference = media-sponsored + other firm:
| Coefficient | β | SE | p |
|---|---|---|---|
| is_trusted_firm | −1.15 | 0.32 | 0.0004 |
| is_candidate | +0.29 | 0.46 | 0.52 |
| is_pollster_self | +0.21 | 0.30 | 0.49 |
| is_other_firm | +0.09 | 0.29 | 0.76 |
…on mean_abs_error. Firm tier dominates; the bucket dummies
shrink to null. On margin_error:
| Coefficient | β | SE | p |
|---|---|---|---|
| is_trusted_firm | −2.02 | 0.63 | 0.001 |
| is_candidate | −2.53 | 0.93 | 0.007 |
| is_pollster_self | −0.91 | 0.61 | 0.14 |
| is_other_firm | −1.95 | 0.62 | 0.002 |
Three significant negative coefficients — the trusted-firm,
candidate-linked, and other_firm dummies all UNDERSHOOT the
margin vs the media-other-firm reference. The other_firm
margin_error signal (the AN-084 headline) survives the firm-
tier control at full magnitude (β = −1.95) — it's not just
that shells use shadier firms. On poll_calls_winner_first:
| Coefficient | β | SE | p |
|---|---|---|---|
| is_trusted_firm | +0.099 | 0.029 | 0.0005 |
| is_candidate | +0.024 | 0.032 | 0.45 |
| is_pollster_self | +0.027 | 0.022 | 0.23 |
| is_other_firm | −0.018 | 0.024 | 0.44 |
Trusted-firm polls call the winner right 9.95 pp more often
than other-firm media polls within race. The AN-082 result on
is_other_firm (less accurate rank) was being partially driven
by firm tier — once firm tier is in, the is_other_firm rank
effect shrinks to −1.8 pp (p=0.44).
Within-trusted-firm spec (D)
Restricting to the 558 polls produced by the 5 trusted firms, how do their sponsor classes compare?
Race FE, reference = media-sponsored trusted-firm poll:
| Outcome | candidate β (p) | pollster_self β (p) | other_firm β (p) |
|---|---|---|---|
| mean_abs_error | −0.09 (0.90) | −2.80 (0.0001) | +0.76 (0.56) |
| margin_error | −2.94 (0.10) | −2.77 (0.017) | −1.63 (0.25) |
| poll_calls_winner_first | +0.009 (0.95) | +0.098 (0.063) | −0.089 (0.34) |
The pollster_self-trusted-firm cell (Datafolha / Quaest / Real Time doing showcase self-contracted polls) is the most accurate slice of the entire dataset: mean |error| 5.21 pp vs universe 8.13 pp. Within trusted firms, their pollster_self polls outperform their own media-sponsored polls by 2.8 pp on |error|.
This is the structural reading of AN-082's surprise finding (pollster_self firms call the winner 7.4 pp more often within race × week): it's mostly trusted firms doing brand-protection showcase polls, not the IPOP-style 2020 fraud channel that nowadays lives in other_firm.
Interpretation
The "media" bucket is heterogeneous. Trusted-firm + major- media-sponsor polls (the headline-quality slice) sit clearly apart from the bulk. The bulk media-sponsored pool is mostly small digital outlets and blogs — by sponsor volume the top spots are "3 PODERES MIDIA" (276), "BG MIDIAS" (158), "BLOG DO ALBERES XAVIER" (130). Globo (50), Folha (46) are way down the list.
The user's "media might be shells too" hypothesis is empirically grounded. Within-media-bucket within-race margin- error differs by 3.16 pp between trusted and non-trusted firms, and mean |error| differs by 1.0 pp between major and small media sponsors. Either some "media-sponsored" polls are functionally candidate-money rerouted through small digital outlets, or the small-digital outlets just are systematically worse pollsters' clients. Either way the media bucket is not the clean reference AN-082 implicitly treated it as.
Two structural findings emerge:
Trusted-firm dominance. Five firms (Datafolha, Quaest, Paraná Pesquisas, Real Time Mídia, Verita) account for 8 % of polls but consistently produce the most accurate predictions by every measure. Their self-contracted polls (n=168) are the cleanest data we have.
The other_firm margin-error signal survives firm tier control. Even after adding is_trusted_firm, the is_other_firm coefficient on margin_error remains −1.95 pp (p=0.002). The shell-style margin error isn't just "shells use bad firms" — it's something about how shell sponsors themselves distort what gets registered.
Implications for the headline framing
AN-082 used media as the implicit-honest reference. AN-085
shows that reference was 90 % small / non-trusted media. The
"other_firm vs media" contrast is more truthfully "other_firm
vs mostly-small-blog-sponsored media" — both groups are below
the trusted-firm slice. The trusted-firm slice is the right
reference for "what does the disclosure regime let an honest
poll look like."
A cleaner AN-082 re-spec would:
- restrict the media reference to trusted-firm media-sponsored polls (n=278) and re-run the bucket dummies, or
- include is_trusted_firm as an orthogonal dummy throughout the paper's specs (the AN-085 spec C / D version).
Caveats
Trusted-firm list is judgment-based. Five firms by name pattern. Industry recognition matters; the list could reasonably include AtlasIntel, Vox Populi, IPEC (most absent or near-absent from the matched mayoral sample). Including large-volume "second-tier" firms (DATATRENDS n=203, VOX BRASIL n=167, etc.) would dilute the trusted-firm advantage we see. Different operationalizations (Q5 firm size, Q1 firm size, pre-registration in CONSAS, etc.) would give different trusted-firm definitions worth exploring.
Major-media sponsor list is a regex. Misses cases: smaller-but-recognized regional papers, magazine sponsors, TV affiliates not in the pattern. The 304 major-media-sponsored polls are a lower bound; the actual share of major-sponsor polls in the media bucket may be larger by ~50 %.
Endogeneity of who hires whom. Trusted firms charge more, so they're disproportionately hired in big-stakes races where the candidates are well-known. Race FE absorbs the static part but not the "trusted firms get harder races" → race margin attenuation. Direction is conservative (against finding trusted-firm accuracy), but the effect magnitude is biased.
n=42 trusted-candidate-sponsored polls is small. The −2.94 pp margin coefficient at p=0.10 is directionally consistent with the universe but underpowered.
Paraná Pesquisas inclusion is contested. The Folha-2022 article in the project corpus alleges Paraná Pesquisas received R$ 2.7M from the PL party. Including it in "trusted" is the industry-recognition convention but not necessarily the right cut substantively. Re-running with PARANA excluded gives is_trusted_firm β ≈ −1.0 pp on |error| (vs the −1.15 with it) — a modest sharpening.
Follow-ups
Re-spec AN-082 with the trusted-firm orthogonal dummy in all of the four-bucket regressions. This is the cleanest way to disentangle "media bucket is heterogeneous" from "other_firm bucket is slanted."
Audit the small-media-sponsor names. Within the 2,644 small-media polls, run a /llmkit classification on sponsor names: which are legitimate small digital outlets vs which are themselves shell entities. The "3 PODERES MIDIA / 3 PODERES" (276 polls) is the biggest single sponsor in the media bucket — that's worth one targeted look.
Add a "second-tier trusted" cut. Run with the 5-firm set, the 10-firm set (adding DATATRENDS, VOX BRASIL, etc. by volume), and the 20-firm set. The robustness of the "is_trusted_firm" coefficient to definition tells us how crisp the tier distinction is.
Within-firm × cross-bucket within-race × week. The most demanding within-firm test we can run is restricting to the five trusted firms and asking: within Quaest's polls in a given race × week, does the sponsor bucket matter? Sample size is the bottleneck (n=558 in 4 cells in a few hundred races) but the test is the cleanest possible identifier.
Wire to the paper. AN-085 reframes how §Results presents the bucket contrasts. Add a panel "trusted-firm decomposition" to the headline regression table; mention the media-bucket-heterogeneity caveat in §Setting or §Identification.
Artifacts
- Script:
source/analysis/an-085-trusted-source-decomposition.py - Spec-level coefficients:
build/table/an-085-trusted-source-decomposition.csv - Cell-level raw means:
build/table/an-085-trusted-source-decomposition__cells.csv - Headline JSON:
build/table/an-085-trusted-source-decomposition.json