Inspiration

After a stream, creators often remember a moment but not its timestamp. Finding it again, making a cover, and clipping it can consume the time they wanted to spend creating. SemVOD gives the streamer control over what counts as a moment: search for the dog, the glowing blue doorway, or a phrase from chat.

What it does

SemVOD is Ctrl+F for your stream: a local app captures small frames from the OBS program scene and Twitch chat through EventSub. Search in plain words, explore a visual-relevance heatmap beside chat energy, select a frame, download a 1280×720 cover with optional title, and create a clip from the matching Twitch archive. A VOD timestamp link is the fallback when Twitch declines clipping.

The included illustrated sample has original procedural artwork, synthetic chat, and clearly labeled bundled results. Judges can try it with no accounts or keys. Actual AI searches use the creator’s OpenAI key. Phrase search uses recorded chat, not audio transcription.

How it connects to Twitch

OBS WebSocket v5 provides screenshots because Twitch’s metadata API does not supply VOD video files. Twitch OAuth requests user:read:chat and channel:manage:clips. EventSub records channel.chat.message. The clip flow matches the recording’s stream ID and time to its archive, calculates the end offset required by Create Clip From VOD, and polls for confirmation.

How we built it with Codex

This project was built in one Codex thread. Codex checked the official API contracts, wrote the native OBS and EventSub adapters, built the interface and cache, generated original sample artwork through code, and tested the integration. Human feedback pushed toward fewer dependencies, smaller images, memoization, privacy, and adversarial verification.

Codex session ID: 01a19d28-f0f4-7e53-8bda-dd046fd0c3a8

Cost and implementation

Exact duplicate frames share scores. Numbered contact sheets batch up to 48 frames; short recordings preserve more detail. A refinement pass checks the strongest unique candidates. Scores persist by normalized query, image content, model, and prompt version; concurrent identical work is coalesced. At most three sheets run concurrently. Moderation precedes live search. There is no hosted backend or database, and keys stay in private local files.

A measured fresh search on our 63-frame, eight-unique-image test used 1,855 input and 278 output tokens: about $0.00119 at published GPT-4.1-mini list prices. Repeating it used zero vision calls and took 17 ms. This is a short-session measurement, not a claim about every stream. We compared the typed Decisions API against Responses on the same eight frames. Decisions was faster on the initial tiny fixture, but an adversarial cat-versus-dog check exposed false positives. We kept the tested Responses engine and published the failure alongside the benchmark receipts.

What we verified and learned

Real OBS capture, Twitch OAuth, a public artwork-only broadcast, a confirmed 30-second app-created Twitch clip, actual EventSub message delivery, OpenAI moderation and vision, cover download, and cache reuse all passed. Twenty-two tests cover protocols, caching, HTTP protections, clip offsets, lifecycle races, and reconnect recovery.

Adversarial tests reproduced startup/stop and late-chat-event defects before the fixes, then passed afterward. Temporary JSON files now have owner-only permissions from creation. The first compact sheet missed a small dog; short sheets now retain 384-pixel detail, and refinement corrections update every duplicate timestamp. An absent elephant returned no hits, and the real chat phrase ranked the frame nearest its timestamp. Receipts and limits are documented in the repository.

Semantic timelines

The gallery compares real searches for dog, cat, and brightest blue across the same recorded stream. Every query uses the same fixed 0–10 scale: no per-query stretching that turns weak evidence into a bright peak. A stacked view adds the scores divided by ten; these are relevance signals, not probabilities. The absent cat query stays flat with the release engine. Raw score arrays and the deterministic renderer are in the repo. This comparison is a published analysis graphic; the app currently explores one query at a time.

What’s next

A moderator replay queue, stream chapters, and vertical covers. Still-image sampling trades motion and audio detail for lower cost; longer real-gameplay benchmarks and streamer feedback are the next validation step. The creator stays in control.

Source and verification: https://github.com/OttObottr/semvod

Built With

Share this project:

Updates

Submission history