StreamLore

StreamLore “What did I miss & what is that?” for every viewer An AI agent & twitch integration that keeps track of the stream’s current objective, major moments, and inside jokes.

Comment

Inspiration Joining a Twitch stream late means missing more than the action. You also miss the inside jokes, recurring characters, and stories that make the community feel like a community.

StreamLore started with two questions: “What did I miss?” and “What does that mean?” We wanted newcomers to understand the conversation without making streamers and regular viewers repeat the same explanations. What it does StreamLore combines a stream catch-up companion with a lasting library of channel lore.

Viewers can read the current objective, a short recap, and a timeline of important moments. They can search community phrases and ask privately about unfamiliar jokes in the Twitch panel. Answers use approved channel lore or clearly labeled starter references. Unknown phrases return an honest no-match response.

Creators and moderators can record moments, review lore drafts, attach origin clips, and approve entries directly or nominate them for community voting. Approved lore can also appear through an on-stream overlay.

Chat commands such as !catchup, !lore, and !lorevote connect these features to the conversation. Optional AI helps summarize supplied context and draft lore for review. How we built it We built the companion with React, TypeScript, and Vite, alongside native Twitch panel, mobile, configuration, and live management views.

The backend supports Node.js with Express and SQLite, plus a Cloudflare Workers deployment using SQLite-backed Durable Objects. Twitch OAuth connects broadcaster accounts, EventSub brings in chat events, and signed Extension tokens establish viewer roles and channel identity.

Optional AI features use the OpenAI Responses API with structured output grounded in supplied captions, transcripts, and stream context. AI-generated lore stays private until a creator reviews it.

OpenAI Codex was our development collaborator, helping implement the interface, Twitch integrations, moderation workflows, packaging, and automated checks. Challenges we ran into One challenge was separating temporary stream context from permanent community history. A new broadcast should reset the catch-up timeline while preserving the jokes and traditions viewers return for.

Another was making explanations trustworthy. Repeated chat messages can suggest that something matters, but they do not prove where a joke originated. We built review controls, source references, and explicit unknown responses around that distinction.

Twitch integration also required careful coordination between Extension authentication, broadcaster OAuth, hosting, and permissions. Private messaging introduced additional delivery constraints, so we made private panel answers a core part of the experience.

Finally, we had to make the interface useful within a narrow Twitch panel while keeping creator controls accessible. Accomplishments that we're proud of

  • Bringing stream catch-up and community lore together in one experience.
  • Building a complete draft, review, nomination, and voting workflow.
  • Reaching Twitch Hosted Test with StreamLore active on TheDarthh’s channel.
  • Exercising the private question flow inside Twitch and connecting broadcaster authorization and the chat webhook.
  • Adding persistent channel storage, encrypted credentials, and channel isolation.
  • Building automated coverage for authentication, moderation, voting, private answers, and Extension packaging. What we learned We learned that useful AI needs clear boundaries. A plausible explanation is not enough when describing a community’s history. Evidence and creator review make the feature more useful.

We also learned to distinguish local correctness from live integration. Mocked tests help validate behavior, while actual Twitch testing reveals authentication, hosting, and delivery issues.

Most of all, we learned that onboarding is about belonging. Understanding one recurring joke can give a newcomer a way into the conversation. What's next for StreamLore Our immediate priorities are completing viewer testing, verifying real Whisper receipt, validating mobile behavior, and finishing the materials required for Twitch review and public release.

We also want to expand the sourced lore library, simplify creator onboarding, and finish validating the personal viewer add-on for explanations across channels.

Longer term, we want to explore caption and transcription integrations that reduce manual context entry while preserving source attribution and creator control.

Built With

Share this project:

Updates

Submission history