-
-
Held-out test: 15/20 correct with 0 false receipts, over a public-domain audiobook with game noise.
-
Receipt found: the streamer's own words and real voice, checked in 288 to 452 ms on the live broadcast.
-
Lower-third overlay capped at 34% of the frame, so the game stays visible.
-
NO MATCH on a confident miss: chat loses a life, while CAN'T TELL punishes nobody.
-
A promise from mid-fight, found from a viewer's plain-words claim and shown in the streamer's own words.
-
!replay brings back a played receipt from the hall of shame, using the real clip.
Inspiration
"Damn, they got the receipts." Every corner of the internet knows that moment: someone denies saying something, and someone else pulls up the proof. On Twitch, chat lives that moment constantly. Streamers make promises live, then deny them a minute later. Chat remembers, but it can't prove anything. We wanted every viewer to be able to pull up the receipts.
What it does
A viewer types !receipts <claim> in plain words, such as !receipts you said you would play with no armor if you died. Receipts answers with one of three outcomes:
- RECEIPT FOUND: the stream overlay stamps the screen, shows the verbatim quote with word highlighting, and plays the streamer's actual voice from a local rolling audio buffer.
- NO MATCH: the claim conflicts with what was said. Chat loses a life, so spamming false claims has a cost.
- CAN'T TELL: the evidence is weak. Nobody is punished.
The streamer opts in. A moderator or the broadcaster can veto a receipt during a short hold before playback.
Then it turns into a show:
- Announcer: a stock voice calls every outcome, ending on "Damn. They got the receipts." It plays six fixed lines and never speaks a quote.
- Hall of shame:
!replaybrings back any played receipt, using the real clip. - ON RECORD: the verbatim quote stays on screen until a mod or the broadcaster types
!settled.
Innovation
Clip tools need someone to remember the moment and find it. Receipts turns memory into a game between chat and streamer: a plain-language claim gets checked against what was actually said, and the proof is the streamer's own voice. Chat's lives make it a contest instead of a harassment tool, and "can't tell" means uncertainty costs nobody anything.
Potential impact
It is built for any streamer on a Windows PC who can add an OBS browser source. It needs no Affiliate status, cloud AI or paid API, and audio stays on the streamer's PC. Viewers get a reason to listen closely and call things out, and the streamer gets a running bit with built-in consent and a moderator veto.
Technological implementation
- Node.js and TypeScript server, OBS overlay, and a Python sidecar on one Windows PC.
- Twitch EventSub WebSocket (
channel.chat.message), the Send Chat Message API, and Device Code Grant sign-in. - Local faster-whisper with word timestamps.
- Two-stage matching: hybrid lexical and embedding retrieval, then a local NLI cross-encoder that checks whether the words entail the claim, with a negation guard.
- Span refinement picks the smallest passage that still clears the verifier, so clips are short.
Results are in eval/RESULTS.md:
- 15/20 fresh held-out claims correct, with 0 false receipts on that set, over a public-domain audiobook with game noise added. The claims came from a Codex turn, before the verifier ran on them.
- 15/16 on synthetic streamer text, with one false receipt. It is not infallible, which is why the veto exists.
- Mean claim-to-decision 214 ms on replay (p95 254 ms). In the live broadcast shown in the video, the three decisions took 288 to 452 ms.
- WER 11.53%.
Design
The overlay is a lower third capped at 34% of the frame, so the game stays visible. A large stamp, word highlighting and a compact lives strip are designed to read without sound. The veto and opt-in live on a local control page.
How it was built with Codex
All product code came from a single Codex thread (session ID in the form), twelve turns in all, one of them a blocked first attempt. Working turns followed the same pattern: prompt, Codex edits, typecheck and npm test. Codex's sandbox can't write to git, so Claude Code committed each passing turn. Claude Code also wrote the prompts, ran the evals and produced the demo video.
- An early matcher scored 3/30 because true and false claims overlapped in score. Codex rebuilt it as retrieve-then-verify (commit cdd3971).
- A real take produced a quote fragment and a 26-second clip. Codex added word timestamps and span refinement.
- A test at 1920x1080 showed the overlay filling the frame. Codex moved it to a lower third.
What's real in the video
The Twitch channel, EventSub chat, bot replies, overlay and matching are real, on a real broadcast at twitch.tv/dankam0001. Chat came from the developer's own account.
The streamer is scripted: the voice is a voice clone reading a script. The narration is the same clone, and the announcer is a stock AI voice. Announcer audio was added in the edit at the moments the overlay played it, because the broadcast capture does not include overlay audio. The Minecraft footage is real Minecraft, but the fight was pre-recorded and played by automated inputs plus a data-pack timeline that scripts the aim, the health loss and the death. That recording is what went out on the broadcast under the live overlay. On-screen captions label the scripted voice, the pre-recorded fight on automated inputs, and the announcer.
Built With
- api
- claude
- code
- codex
- elevenlabs
- eventsub
- faster-whisper
- node.js
- obs
- openai
- python
- twitch
- typescript