SIGNAL Ep 1 — independent citation validation (live test, 2026-08-23)

An agent with only web access re-validated every load-bearing claim in the live public episode

The Signal, Week Ahead in AI (2026-07-20), 34 days after air.

This is the platform's proposed Judge gate, run for real against real content. Full verdict table below.

Scorecard

• 7 claims fully SUPPORTED — including all three that the pre-air Council judge had forced citations into (the Grok Build leak, the Claude Code source leak + permission flaw). Those verify at full precision 34 days later.
• 4 PARTLY (nuance lost) — paraphrase drift: "(soon to be) Open Model" aired as "weights you can download and run"; New York's one-year hyperscale permit pause aired as "banned datacenters."
• 1 UNSUPPORTED as framed — Mira Murati's model was framed as upcoming; Thinking Machines had shipped Inkling five days before air.
• 1 unverifiable (rolling feed, no archive permalink), 1 stale-since-air.

The structural finding

Most on-air attributions are podcast mentions ("per All-In, July 18"), not resolvable sources — hearsay with a timestamp. Three stories credited to a recap podcast actually broke in primary press days earlier, and the recap is where "permit pause" became "ban." Zero of ~14 claims carry a citation on any public surface: the three real citations live only in an unpublished provenance.json, whose piece-ID doesn't match the shipped episode's. The homepage still says "IN PRODUCTION" while the mp3 and RSS item are live.

Why this is the proof point

The gate already worked once: the pre-air judge went RED on the security segment, Byron intervened, and the fixed claims are the only ones that verify cleanly today. The gate raised accuracy exactly where it fired — it just never fired on the other five threads. And the validator correctly distinguished "no live source located" from "contradicted," which is what makes a gate a gate rather than a filter.

The mechanical fixes (cheap, high-leverage)

1. Publish citations into the feed's show notes (content:encoded) — they already exist on disk.
2. Require a fetchable URL per claim; a podcast name + date is not a citation.
3. Run the judge on every thread, not only the one that looks risky.
4. Fix the provenance piece-ID linkage and the stale "IN PRODUCTION" homepage state.

(Full 14-row verdict table with live source URLs is preserved in the project archive: ~/arnao-fm/tmp/validation-full.md)

Full verdict table