Inspiration
Joining a game stream halfway through can mean missing the context that makes the community fun. CatchUp gives newcomers a way to catch up at their own pace, without calling them out or making the broadcaster repeat the story.
What it does
CatchUp samples the broadcaster's selected OBS game source and turns images into short, evidence-backed progress notes. A Twitch viewer can type !recap or !回顧 to receive the latest shared summary. A read-only viewer page presents screenshots, timestamps, event cards and uncertainty notes. No microphone input is required in this version.
The prototype captures a frame every ten seconds and analyzes up to six frames together. It stores compact state and chapter notes rather than reprocessing the entire broadcast for each request. Chat cooldowns, delivery deduplication, usage accounting and session budget reservations control spam and cost.
How I built it with Codex
Codex helped translate the specification into native Node.js modules for authenticated OBS WebSocket capture, OpenAI Responses API vision analysis, Twitch OAuth and EventSub, the Send Chat Message API, a local control page and a read-only recap page. The app uses Node.js 24 built-ins and has no external package-install step.
I used Codex iteratively to inspect errors, modify the implementation, run tests and verify actual behavior. The demo includes the real Codex development session and source code. Primary development session: 01a1982f-5c5c-76f2-877c-43edfe99fe17.
Challenges
Real OBS returned image/jpg data URLs while the initial adapter expected image/jpeg. We fixed and tested normalization. Another edge case appeared when the model understood the current scene but confirmed no new event: the chat formatter incorrectly said it was still building the recap. A regression test now covers state-only summaries.
Game screenshots can also be ambiguous. TapTapLoot has a mostly dark captured background and a narrow playfield. The model was instructed to describe only visible evidence and avoid inventing progress, actions or victories.
What I verified
A real TapTapLoot OBS source fed six screenshots into one visual analysis. In the actual Twitch channel, two !recap commands produced one successful reply and one cooldown suppression. The chat requests caused zero additional AI calls. The analysis cost approximately US$0.004137, calculated from token usage and configured prices, not invoice-reconciled.
This test used the broadcaster account in an offline channel; it was not a public on-air broadcast or a separate-viewer-account test. Nine automated tests also passed, including from an anonymously downloaded copy of the public GitHub repository.
What I learned
A useful recap must distinguish observed facts from unknown details. Shared summaries make costs depend mainly on sampling frequency and stream length rather than the number of viewers asking for context. Real integration checks caught issues that mocked tests alone did not.
What is next
Test longer changing-game sessions, a separate viewer account and an on-air broadcast; improve game-specific event detection; and add public HTTPS hosting and automatic publishing. The viewer page is currently local. Static export is implemented, but public hosting is not configured. Audio is not analyzed, and sampling may miss brief events.
Reproduce and review
Source and English instructions: https://github.com/snivy1827/catchup-twitch-recap
Run the clearly labeled offline fixture without API keys using node src/server.mjs. It demonstrates the workflow and does not claim AI accuracy. Live mode requires the operator's own OBS, OpenAI and Twitch configuration; no credentials are published. See docs/TWITCH_VALIDATION.md for real test results and limits.
Public demo: https://www.youtube.com/watch?v=hYGgMk6Mikw
The approximately 2:45 video contains real OBS footage, Twitch verification screenshots, Codex development footage and English captions. Narration was synthesized locally; no background music or secrets are included.
Log in or sign up for Devpost to join the conversation.